Commit Graph

815 Commits

Author SHA1 Message Date
A.J. Beamon 1b81e72604 Add a Tuple::makeTuple function to easily construct a tuple. Update Tuple to allow all types to be passed via .append() so they can be used with makeTuple. 2022-07-19 11:50:58 -07:00
A.J. Beamon c7c7bd5f17 Undo change to mark tenant modes non-experimental. Other minor cleanup. 2022-07-19 09:18:53 -07:00
A.J. Beamon 8f77048e95 Add the ability to configure tenant groups in a metacluster 2022-07-18 11:24:49 -07:00
A.J. Beamon 860d3843cc Merge remote-tracking branch 'origin/feature-tenant-groups' into feature-metacluster 2022-07-16 19:33:26 -07:00
A.J. Beamon c08592368f Add support for tenant groups 2022-07-15 21:18:41 -07:00
A.J. Beamon 0fe6f6ad36 Merge remote-tracking branch 'origin/update-prefix-encoding-in-tenant-metadata-json' into feature-metacluster
# Conflicts:
#	bindings/python/tests/fdbcli_tests.py
#	fdbcli/TenantCommands.actor.cpp
#	fdbclient/include/fdbclient/TenantManagement.actor.h
#	fdbclient/include/fdbclient/TenantSpecialKeys.actor.h
#	fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-14 14:32:29 -07:00
Jon Fu 7f3b51f31a initial commit to introduce tenant renaming to special keys 2022-07-13 15:59:52 -07:00
A.J. Beamon cb499fa5db Merge branch 'main' into feature-metacluster 2022-07-13 15:28:34 -07:00
A.J. Beamon 91ccdbcb03 Use a more standard encoding in the JSON metadata reported for tenants. 2022-07-13 15:22:19 -07:00
A.J. Beamon a3f970cf3a Parse integer options in fdbcli rather than passing the bytes unparsed. This allows specifing the string "1" instead of "\x01\x00\x00\x00\x00\x00\x00\x00". 2022-07-11 16:39:59 -07:00
A.J. Beamon 712f40b727 Add support for tenant groups 2022-07-08 15:59:25 -07:00
A.J. Beamon b9cc5389b1 Undo some changes that aren't needed 2022-07-07 14:48:36 -07:00
A.J. Beamon a32cc5218c Fix some merge issues 2022-07-07 11:55:31 -07:00
A.J. Beamon 1519f24f77 Merge branch 'main' into feature-metacluster 2022-07-07 09:35:40 -07:00
Jon Fu df1f108b78 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename 2022-07-06 14:10:22 -07:00
A.J. Beamon aea4d802c6 Merge branch 'main' into feature-metacluster 2022-06-30 15:08:09 -07:00
A.J. Beamon 2f67328a0c Update the tenant special keys submodule to support multiple sub-ranges. This will enable future work that allows configuring tenants at the same time as creating them. 2022-06-30 15:03:37 -07:00
Jon Fu 5e7bb0aa21 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename 2022-06-29 13:20:19 -07:00
Trevor Clinkenbeard db769667ae
Merge pull request #7112 from sfc-gh-tclinkenbeard/global-tag-throttling3
Create Global Tag Throttler
2022-06-29 10:06:00 -07:00
sfc-gh-tclinkenbeard 2134fa9a0c Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-28 19:51:41 -07:00
A.J. Beamon df90d87903 Add missing return statement 2022-06-28 16:03:08 -07:00
A.J. Beamon e1a93988ef Merge branch 'main' into feature-metacluster 2022-06-28 14:58:07 -07:00
Jon Fu 5fee5e1a73 add renameTenant back to TenantAPI after rebase 2022-06-28 11:54:03 -07:00
Jon Fu 869c6fcac5 reintroduce renameTenantCommandActor as a pass-through function 2022-06-28 11:35:40 -07:00
Jon Fu 904ac358ce call directly into management api and remove extra function 2022-06-28 11:35:38 -07:00
Jon Fu 5f4f95db4d fix unreachable code 2022-06-28 11:34:54 -07:00
Jon Fu ddfdbdcb09 initial commit introducing renameTenant function 2022-06-28 11:34:44 -07:00
sfc-gh-tclinkenbeard 086e4bff06 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-28 10:18:13 -07:00
Markus Pilman fc2f695b15 Merge remote-tracking branch 'origin/main' into features/cmake-refactor-dirs 2022-06-28 10:41:07 -06:00
A.J. Beamon fcd91f676a
Merge pull request #7436 from sfc-gh-ajbeamon/tenant-reorganization
Tenant code refactoring
2022-06-28 09:16:19 -07:00
Markus Pilman 1bdbd89839 Merge remote-tracking branch 'origin/main' into features/cmake-refactor-dirs 2022-06-28 08:52:03 -06:00
Markus Pilman 4ac236eeba fdbcli compiling 2022-06-27 19:20:18 -06:00
Markus Pilman 03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
A.J. Beamon f990fba44d Add support for getting tenant metadata as a JSON document. 2022-06-27 12:34:40 -07:00
A.J. Beamon 4bafe77889 Some refactoring of tenant code:
* extract tenant management into its own file and namespace
* rename the tenant management workload source file
* extract tenant special keys functions to a separate file
* extract some helper functions to GenericTransactionHelper.h
* convert StringRef -> TenantNameRef
* move some TenantMapEntry implementation into the cpp file
* add some helper functions to decode/encode a tenant mode
2022-06-27 12:32:49 -07:00
A.J. Beamon eccf244ba6 Add an index that allows us to allocate tenants from the non-full cluster with the most tenants already assigned to it. 2022-06-27 12:28:55 -07:00
A.J. Beamon 4b69723a2f Cleanup metadata associated with a cluster when force removing it. Move some metacluster metadata into the normal key-space. Cleanup some todos. 2022-06-24 14:55:57 -07:00
Markus Pilman d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
A.J. Beamon 9f3819752f Change the command to create a metacluster from using 'configure tenant_mode=management' to 'metacluster create <NAME>'. Distribute this name to all processes in a metacluster. Eliminate the tenant mode entirely from metacluster clusters, instead relying on a metacluster registration key. 2022-06-22 12:15:43 -07:00
A.J. Beamon 96fad987eb Add new command to get total metacluster capacity. Fix bug where removing a data cluster could make it impossible to create new tenants on it. Add knobs for max data clusters and max tenants. 2022-06-17 10:42:03 -07:00
sfc-gh-tclinkenbeard 308e0113a8 Update fdbcli quota usage output 2022-06-16 16:26:45 -07:00
sfc-gh-tclinkenbeard 99d243197e Add fdbcli quota command 2022-06-16 14:07:16 -07:00
A.J. Beamon 2907aafa60 Merge branch 'main' into feature-metacluster 2022-06-02 12:18:42 -07:00
Jingyu Zhou db5508520c
Merge pull request #7249 from johscheuer/fix-fdbcli-exclude-check
Check if all specific processes are excluded
2022-05-26 15:01:01 -07:00
Johannes M. Scheuermann 8277965158 Correct variable for for loop to iterate over inprogress exclusions 2022-05-25 17:21:15 +01:00
Johannes M. Scheuermann 0bc8e2cea6 Check if all specific processes are excluded 2022-05-25 17:08:02 +01:00
A.J. Beamon 9773261a03 Expand support for tenant groups. Track where each tenant group is assigned and used groups to help assign tenants. Fix tenant state tracking bugs. Add help hints for metacluster remove FORCE. 2022-05-24 11:58:40 -07:00
Xiaoxi Wang fd35fde481 Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-05-23 15:09:03 -07:00
Lukas Joswiak 475b157310 Add `get` command to versionepoch help text 2022-05-23 11:45:18 -07:00
Xiaoxi Wang 73624bcd2a Merge remote-tracking branch 'upstream/main' into readaware 2022-05-23 11:17:38 -07:00
A.J. Beamon d784173f7f Add first implementation of tenant creation and deletion in a metacluster 2022-05-20 15:21:21 -07:00
Chaoguang Lin 5e8c3130e6 Extend the reboot interface to accept comma-delimited string;
Only fetch requested worker interfaces in rebootWorkerActor
Add killall test to multiprocess_fdbcli_tests
Make the rebootWorker only sends requests if all the requested workers are valid
Add comments; Organize print texts; Fix the bug clear the map every time run kill/suspend/expensive_data_check
2022-05-20 12:06:33 -07:00
A.J. Beamon 69261f9f10 Merge branch 'main' into feature-metacluster
# Conflicts:
#	fdbclient/MultiVersionTransaction.actor.cpp
#	fdbclient/MultiVersionTransaction.h
#	fdbserver/tester.actor.cpp
2022-05-19 09:11:30 -07:00
A.J. Beamon 367e59dc33 Add support for getting tenant and cluster metadata in fdbcli in a JSON format 2022-05-18 14:53:25 -07:00
A.J. Beamon 98c3813431 Refactor the register logic so that only two write transactions take place. The first is to the data cluster, the second to the management cluster. 2022-05-18 11:21:39 -07:00
A.J. Beamon e8e26c9f7c Remove the internal special key-space mechanism (at least for now), instead relying on the local logic to implement the metacluster functions. Add ability to force remove a cluster. Improve idempotency of register operation. 2022-05-18 08:18:25 -07:00
Xiaoxi Wang 382f0fc4a2 merge upstream/main 2022-05-17 10:20:51 -07:00
Evan Tschannen 583120514f fixed compilation errors 2022-05-12 17:19:25 -07:00
Evan Tschannen c51ad847d6 revert "kill" command to previous 7.0 behavior because the current implementation is killing processes one at a time 2022-05-12 13:12:23 -07:00
A.J. Beamon 85019612d4 Remove subordinate mode and instead use a key in the system key-space to track metacluster membership. Use this key to determine if a cluster is already part of a metacluster or can be configured away from required mode. Disallow configuring to or from a management cluster that has data. 2022-05-11 17:10:55 -07:00
Vishesh Yadav f14baf2af8 clang-format changes 2022-05-09 14:54:51 -07:00
Vishesh Yadav 9173e2e19b Move GlobalConfig to DatabaseContext 2022-05-09 14:54:51 -07:00
Vishesh Yadav 7578d5ebc7 Create GlobalConfig object for each database instance
Currently, GlobalConfig is a singleton that means for each process there is only
one GlobalConfig object. This is bug from clients perspective as a client can
keep connections to several databases. This patch tracks GlobalConfig for each
database using an unordered_map in flowGlobals.

We discovered this bug while testing multi-version client, where the client got
stuck. This was lucky, as normally it'd just write down config to the wrong
database.
2022-05-09 14:54:51 -07:00
A.J. Beamon 80a3c8dd50 All metacluster operations to the management cluster go through the management cluster special key space 2022-05-09 14:50:07 -07:00
A.J. Beamon 7c3b99f5bd Add support for connection to clusters via a connection string. Start work on multi-cluster operations with data cluster registration. 2022-05-06 09:53:53 -07:00
Xiaoxi Wang 992ff59bfb Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-05-05 09:57:22 -07:00
A.J. Beamon 0c41225d76 Add support for setting and storing the tenant group for a tenant 2022-05-05 08:57:01 -07:00
Trevor Clinkenbeard 0069ce51a5
Merge pull request #7054 from sfc-gh-tclinkenbeard/remove-non-tls-support
Remove `DISABLE_TLS` CMake option
2022-05-03 22:13:07 -07:00
Xiaoxi Wang 7c37d172b9 solve some comments 2022-05-03 17:21:08 -07:00
Xiaoxi Wang 269d85daa8 Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-05-03 13:37:56 -07:00
A.J. Beamon 7e860dc92c Some initial metacluster implementation 2022-05-03 12:53:35 -07:00
sfc-gh-tclinkenbeard 225146176d Apply clang-format to fdbcli.actor.cpp and Net2.actor.cpp 2022-05-03 12:13:09 -07:00
sfc-gh-tclinkenbeard 06825775db Fix formatting of lines with TLS_OPTION_FLAGS 2022-05-02 22:56:06 -07:00
sfc-gh-tclinkenbeard 7f05221cfe Removed TLS_DISABLED macro 2022-05-02 22:15:27 -07:00
Xiaoxi Wang 69985ba251 Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-05-02 10:53:22 -07:00
A.J. Beamon 43c2ca35a5 Move fdbcli command and hint generators into the files implementing the command. 2022-05-02 08:39:59 -07:00
Renxuan Wang c69a07a858
Check in the new Hostname logic. (#6926)
* Revert #6655.

20220407-031010-renxuan-c101052c21da8346           compressed=True data_size=31004844 duration=4310801 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=1:04:15 sanity=False started=100047 stopped=20220407-041425 submitted=20220407-031010 timeout=5400 username=renxuan

* Revert #6271.

20220407-051532-renxuan-470f0fe6aac1c217           compressed=True data_size=30982370 duration=3491067 ended=100002 fail_fast=10 max_runs=100000 pass=100002 priority=100 remaining=0 runtime=0:59:57 sanity=False started=100141 stopped=20220407-061529 submitted=20220407-051532 timeout=5400 username=renxuan

* Revert #6266.

Remove resolving-related functionalities in connection string. Connection string will be used for storing purpose only, and non-mutable.

20220407-175119-renxuan-55d30ee1a4b42c2f           compressed=True data_size=30970443 duration=5437659 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=0:59:31 sanity=False started=100154 stopped=20220407-185050 submitted=20220407-175119 timeout=5400 username=renxuan

* Add hostname to coordinator interfaces.

* Turn on the new hostname logic.

* Add the corresponding change in config txns.

The most notable change is before calling basicLoadBalance(), we need to call tryInitializeRequestStream() to initialize request streams first.

Passed correctness tests.

* Return error when hostnames cannot be resolved in coordinators command.

* Minor fixes.
2022-04-27 21:54:13 -07:00
Xiaoxi Wang 0639810b66 merge upstream/main 2022-04-22 11:09:15 -07:00
Zhe Wang 6c9ff6ee5e
Add sharded rocksdb type (#6862)
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
2022-04-21 22:53:14 -04:00
Xiaoxi Wang d17b36acf8 Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-04-15 11:10:09 -07:00
Andrew Noyes 29cf5f1fbf
Fix an ASSERT when an fdbcli command times out (#6857)
* Re-throw operation_cancelled

There's a few places in fdbcli where we don't rethrow operation
cancelled but wait on a future. It's very unusual that you don't want to
rethrow operation_cancelled.

* Update ASSERT

It's possible to get error_code_broken_promise here if the network has
already shutdown.
2022-04-14 12:09:25 -07:00
Xiaoxi Wang 8d3f851495 merge upstream/mainA 2022-04-12 17:03:09 -07:00
Xiaoxi Wang ed97a35dc0 Merge branch 'main' into readaware 2022-04-12 16:47:15 -07:00
Xiaoxi Wang 61a1f7683b fix dd command line read special key space error 2022-04-11 22:49:21 -07:00
Aaron Molitor cbaef8f03b update version to 7.2.0 2022-04-11 23:23:27 -05:00
Xiaoxi Wang 6c841cd32b merge readaware-better 2022-04-11 17:09:39 -07:00
Xiaoxi Wang 82e5859e03 allow safeThreadFutureToFuture 2022-04-11 14:35:12 -07:00
Lukas Joswiak 73a7c32982
Add fdbcli command to read/write version epoch (#6480)
* Initialize cluster version at wall-clock time

Previously, new clusters would begin at version 0. After this change,
clusters will initialize at a version matching wall-clock time. Instead
of using the Unix epoch (or Windows epoch), FDB clusters will use a new
epoch, defaulting to January 1, 2010, 01:00:00+00:00. In the future,
this base epoch will be modifiable through fdbcli, allowing
administrators to advance the cluster version.

Basing the version off of time allows different FDB clusters to share
data without running into version issues.

* Send version epoch to master

* Cleanup

* Update fdbserver/storageserver.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Jump directly to expected version if possible

* Fix initial version issue on storage servers

* Add random recovery offset to start version in simulation

* Type fixes

* Disable reference time by default

Enable on a cluster using the fdbcli command `versionepoch add 0`.

* Use correct recoveryTransactionVersion when recovering

* Allow version epoch to be adjusted forwards (to decrease the version)

* Set version epoch in simulation

* Add quiet database check to ensure small version offset

* Fix initial version issue on storage servers

* Disable reference time by default

Enable on a cluster using the fdbcli command `versionepoch add 0`.

* Add fdbcli command to read/write version epoch

* Cause recovery when version epoch is set

* Handle optional version epoch key

* Add ability to clear the version epoch

This causes version advancement to revert to the old methodology whereas
versions attempt to advance by about a million versions per second,
instead of trying to match the clock.

* Update transaction access

* Modify version epoch to use microseconds instead of seconds

* Modify fdbcli version target API

Move commands from `versionepoch` to `targetversion` top level command.

* Add fdbcli tests for

* Temporarily disable targetversion cli tests

* Fix version epoch fetch issue

* Fix Arena issue

* Reduce max version jump in simulation to 1,000,000

* Rework fdbcli API

It now requires two commands to fully switch a cluster to using the
version epoch. First, enable the version epoch with `versionepoch
enable` or `versionepoch set <versionepoch>`. At this point, versions
will be given out at a faster or slower rate in an attempt to reach the
expected version. Then, run `versionepoch commit` to perform a one time
jump to the expected version. This is essentially irreversible.

* Temporarily disable old targetversion tests

* Cleanup

* Move version epoch buggify to sequencer

This will cause some issues with the QuietDatabase check for the version
offset - namely, it won't do anything, since the version epoch is not
being written to the txnStateStore in simulation. This will get fixed in
the future.

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-04-08 12:33:19 -07:00
Xiaoxi Wang 5113277e9a Merge branch 'main' into readaware 2022-04-08 11:50:01 -07:00
Xiaoxi Wang 411550580e merge 2022-04-08 11:49:33 -07:00
neethuhaneesha 5311d93c72
Merge pull request #6783 from neethuhaneesha/rocksdbEngineType
Adding warning on configuring rocksdb storage until production tested.
2022-04-07 13:29:12 -07:00
Xiaoxi Wang 150b4318aa refactor datadistribution command; try dual-mode code 2022-04-06 22:10:23 -07:00
Yi Wu 994b8c92f8
Add option to limit resident memory and remove default memory limit (#6719)
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.

Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.

closes #6671, closes #6672
2022-04-06 20:06:24 -07:00
Neethu Haneesha Bingi 5d3981821e Adding warning on configuring rocksdb storage until production tested. 2022-04-06 13:24:57 -07:00
Xiaoxi Wang aba9d85560 merge main 2022-04-06 09:57:52 -07:00
Jingyu Zhou f68fd28d73 Refactor duplicated code into IKnobCollection::setupKnobs() 2022-04-05 02:06:38 -07:00
Xiaoxi Wang d6d4596c35
Merge pull request #6709 from sfc-gh-ajbeamon/fix-fdbcli-bugs
Fix issues with command completion for exclude and storage_migration_type
2022-03-30 19:53:49 -07:00
Andrew Noyes d727e7648e Fix a few memory issues found by ASAN 2022-03-29 12:39:12 -07:00
A.J. Beamon 23fcd8c076 Fix issues with command completion for exclude and storage_migration_type. Add missing documentation for tenant_mode in one spot. 2022-03-29 09:07:33 -07:00
Xiaoxi Wang d93b57dd88 conflict solving 2022-03-24 20:45:51 -07:00
Xiaoxi Wang 1b631a9263 solve conflict with main 2022-03-24 16:29:11 -07:00
Josh Slocum f27475e2f4 Merge branch 'main' into blob_integration 2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon fd81ef99c0
Merge pull request #6561 from sfc-gh-ajbeamon/fdb-tenant-fdbcli
Add fdbcli support for tenants
2022-03-21 11:26:10 -07:00
sfc-gh-tclinkenbeard be875e040d Prevent infinite loop on invalid input to 'throttle off' command 2022-03-18 20:22:49 -07:00
sfc-gh-tclinkenbeard 004e73b2f1 Add TagSet::toString method 2022-03-18 20:10:56 -07:00
sfc-gh-tclinkenbeard fdab73d7b3 Rename TagSet variables in throttleCommandActor.
Previously these were named "tags", conflicting with a state variable,
and creating confusing issues when they go out of scope after a wait
statement.
2022-03-18 19:20:25 -07:00
sfc-gh-tclinkenbeard 6eaa20078f Add ContainsRecommended boolean parameter 2022-03-18 18:12:50 -07:00
sfc-gh-tclinkenbeard 63ec6d5f5b Add static defaultThrottleListLimit in ThrottleCommand.actor.cpp 2022-03-18 17:14:34 -07:00
Josh Slocum 37e7c80f26 Merge branch 'main' into blob_integration 2022-03-17 18:45:42 -05:00
A.J. Beamon a23add6bc4 Add fdbcli test for tenants. Add documentation for new fdbcli tenant commands. Various output cleanup. Fix limit parsing bug in listtenants command. Update gettenant output format. 2022-03-17 12:10:39 -07:00
A.J. Beamon bd64781ad9 Use special keys to manage tenants 2022-03-17 12:10:39 -07:00
A.J. Beamon 4aa053d342 Update tenant mode options in fdbcli 2022-03-17 12:10:39 -07:00
A.J. Beamon f8cddd3842 Add fdbcli support for tenants 2022-03-17 12:10:39 -07:00
Xiaoxi Wang d38b827906 fix DD ignore bug 2022-03-13 23:47:07 -07:00
sfc-gh-tclinkenbeard 8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
Renxuan Wang 92ce0de404 When storing coordinators string, we should concatenate by comma.
We are splitting by comma (402fa4dd9e/fdbclient/SpecialKeySpace.actor.cpp (L1686)). So when we concatenating, if we use ", ", there will be an extra space, causing hostname parsing error. NetworkAddress happens not to have the same issue because it uses sscanf (402fa4dd9e/flow/network.cpp (L119)).
2022-03-09 14:28:22 -08:00
Josh Slocum e71b3533f9 Merge branch 'main' into blob_integration 2022-03-09 08:59:56 -06:00
Jingyu Zhou 1a5bf25b5c Update code base to use fmt 8.1.1 2022-03-04 15:52:06 -08:00
Xiaoxi Wang 01c347d45c format code; special key space notice update 2022-03-03 15:38:28 -08:00
Xiaoxi Wang a28b0f4361 Merge branch 'main' of https://github.com/apple/foundationdb into readaware 2022-03-03 11:15:18 -08:00
Xiaoxi Wang 887d5a25cb add storage wiggle to status 2022-03-02 11:31:55 -08:00
Xiaoxi Wang e73c0a31e6 add wiggle_server_ids and wiggle_server_addresses in status json 2022-03-02 10:03:23 -08:00
Renxuan Wang fa2eee8279 Use PRId64 and PRIu64 to replace lld and llu.
This fixes https://github.com/apple/foundationdb/pull/6148#issuecomment-1054575608.
2022-02-28 21:30:14 -08:00
Renxuan Wang 233c918ffb Replace printf() and fprintf() with fmt::print(). 2022-02-25 19:06:57 -08:00
Renxuan Wang f7eb66441d Try eliminating warnings in macOS and Windows CI builds.
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
Xiaoxi Wang 40a1f562a7 temporary change special key for data distributor 2022-02-25 11:01:23 -08:00
Renxuan Wang 06b1d06d38 Support hostname in coordinators commands. 2022-02-24 23:02:29 -08:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Renxuan Wang 622d89b552 Rebase on main.
Since we changed ClusterConnectionString's status flag from boolean to enum in #6422, we need to update this PR correspondingly.
2022-02-22 16:29:59 -08:00
Renxuan Wang 481587a8c6 Turn on hostname logic. 2022-02-22 16:29:59 -08:00
Xiaoxi Wang 59ac1dffb4 change storage_migration_mode to storage_migration_type 2022-02-22 09:36:30 -08:00
Josh Slocum 38a75a8b89 Merge branch 'main' into blob_integration 2022-02-17 17:47:38 -06:00
A.J. Beamon 5e7982fdd7
Merge pull request #6350 from sfc-gh-ajbeamon/improve-exclude-help
Improve the help for fdbcli exclude
2022-02-11 13:41:59 -08:00
Renxuan Wang e962d8184c Remove the deprecated code.
They were deleted in #5593 but accidentally added back in #5589.
2022-02-08 17:01:29 -08:00
A.J. Beamon 2973130443 Improve the help for fdbcli exclude. 2022-02-07 10:14:42 -08:00
Xiaoxi Wang 6dc5921575
createdTime based storage wiggler (#6219)
* add storagemetadata

* add StorageWiggler;

* fix serverMetadataKey bug

* add metadata tracker in storage tracker

* finish StorageWiggler

* update next storage ID

* change pid to server id

* write metadata when seed SS

* add status json fields

* remove pid based ppw iteration

* fix time expression

* fix tss metadata nonexistence; fix transaction retry when retrieving metadata

* fix checkMetadata bug when store type is wrong

* fix remove storage status json

* format code

* refactor updateNextWigglingStoragePID

* seperate storage metadata tracker and store type tracker

* rename pid

* wiggler stats

* fix completion between waitServerListChange and storageRecruiter

* solve review comments

* rename system key

* fix database lock timeout by adding lock_aware

* format code

* status json

* resolve code format/naming comments

* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>

* fix omit start rount

* format code

* status json reset

* solve status json format

* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions

* status timestamp
2022-02-04 15:04:30 -08:00
Ray Jenkins dd45805312
Merge branch 'apple:main' into threadname-issue-6064 2022-02-01 17:40:07 -06:00
A.J. Beamon 65f2c14f31
Merge pull request #6320 from johscheuer/add-fdbcli-loggroup
Add LogGroup support to fdbcli
2022-02-01 09:13:18 -08:00
A.J. Beamon 6ca8f08da8
Merge pull request #6293 from sfc-gh-ajbeamon/fdbcli-print-at-col-stream
Support other output streams in fdbcli printAtCol
2022-01-31 13:16:27 -08:00
Johannes M. Scheuermann 3b0feebd5d Add LogGroup support to fdbcli 2022-01-31 17:47:39 +00:00
Ray Jenkins 783cbb0aea Merge branch 'main' into threadname-issue-6064 2022-01-27 09:57:11 -06:00
A.J. Beamon 2a83e99d6f Support other output streams in fdbcli printAtCol 2022-01-26 14:50:23 -08:00
A.J. Beamon 7989b83450 Use printable when formatting change feed ranges 2022-01-26 14:41:47 -08:00
Ray Jenkins 2c41c4841d Rename linenoise thread name. 2022-01-25 13:21:14 -06:00
Ray Jenkins e9e0d26867 Add thread name for FlowLineNoise. 2022-01-24 19:43:40 -06:00
Ray Jenkins ff0d1e5efa add back special_keys_api_failure error check 2022-01-24 13:32:44 -08:00
Ray Jenkins d3055cc59a Use single transaction for setProcessClass and add fdbcli unit test. 2022-01-24 13:32:44 -08:00
Ray Jenkins d1fe48ce84 Format fix. 2022-01-24 13:32:44 -08:00
Ray Jenkins 9f577a8675 Add state prefix to ThreadFuture. 2022-01-24 13:32:44 -08:00