Jon Fu
0487aa5445
Merge branch 'main' of github.com:apple/foundationdb into jfu-metacluster-rename
2022-08-10 14:07:43 -07:00
A.J. Beamon
cd82f65ef5
Change the data cluster entry to use a state field rather than a locked boolean
2022-08-09 11:53:57 -07:00
Jon Fu
74df84f686
Merge branch 'main' of github.com:apple/foundationdb into jfu-metacluster-rename
2022-08-08 17:49:15 -07:00
Vaidas Gasiunas
79571dd2b4
Testing upgrades to a future version of FDB ( #7780 )
...
* Enable configuring the next future protocol version as the current protocol version in FDB client, fdbserver, and fdbcli
* Auto format python files used in upgrade tests
* Add a test for upgrading to a future FDB version
* Emphasize that the options for using future protocol version are intended for test purposes only
* Make the global variable for current protocol version visible only locally
* Refactirng to avoid using currentProtocolVersion() in static intialization
* Update go bindings
2022-08-08 17:29:49 +02:00
Jon Fu
cc85973957
Merge branch 'main' of github.com:apple/foundationdb into jfu-metacluster-rename
2022-08-05 12:38:43 -07:00
Hui Liu
29ad2c0654
fdbcli: show status details about # works and # key ranges if blob granules enabled ( #7792 )
...
* fdbcli: show status if blob granules is enabled
* fdbcli: show status details for blob granules for # works and # key ranges
2022-08-05 12:33:57 -05:00
Josh Slocum
ddbf32d35e
adding knob and fixing blobrange cli check bug with too many granules ( #7793 )
2022-08-05 11:19:17 -05:00
Jon Fu
7a45aba74d
Merge branch 'main' of github.com:apple/foundationdb into jfu-metacluster-rename
2022-08-04 16:36:09 -07:00
A.J. Beamon
ff23d5994e
Merge pull request #7729 from sfc-gh-ajbeamon/feature-metacluster
...
Metacluster
2022-08-04 15:29:44 -07:00
Hui Liu
4f75f01882
fdbcli: show status if blob granules is enabled ( #7784 )
2022-08-03 19:15:34 -05:00
Josh Slocum
1cda8a2fc1
More blob granule operational stuff ( #7783 )
...
* More blob manager metrics
* making blobrange check command work for large ranges
2022-08-03 18:11:25 -05:00
Jon Fu
bceb44f5a1
address code review comments
2022-08-03 12:03:44 -07:00
Jon Fu
5dc8e930a9
remove whitespace
2022-08-01 11:06:04 -07:00
Jon Fu
d9dc1999c6
metacluster rename before using metacluster operation ctx
2022-07-29 13:43:47 -07:00
A.J. Beamon
a69164d160
Merge branch 'main' into feature-metacluster
2022-07-29 11:53:06 -07:00
Josh Slocum
cfc13e7018
Adding more blobrange cli commands and a couple other tweaks ( #7727 )
2022-07-29 08:20:45 -07:00
A.J. Beamon
e5bf838c83
Fix metacluster register command hints, fix bug in cluster registration on retries, extend the length of the tenant management concurrency test
2022-07-28 21:02:12 -07:00
A.J. Beamon
65cfc839a6
Some various cleanup
2022-07-28 13:35:24 -07:00
A.J. Beamon
7f53b340f5
Merge branch 'main' into feature-metacluster
...
# Conflicts:
# bindings/python/tests/fdbcli_tests.py
# fdbcli/TenantCommands.actor.cpp
# fdbclient/ClientKnobs.cpp
# fdbclient/include/fdbclient/ClientKnobs.h
# fdbclient/include/fdbclient/Tenant.h
# fdbserver/BlobManager.actor.cpp
# fdbserver/BlobWorker.actor.cpp
# fdbserver/TenantCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/TenantManagementWorkload.actor.cpp
# flow/include/flow/error_definitions.h
2022-07-28 11:43:21 -07:00
A.J. Beamon
7c6b3fb0b8
Merge branch 'main' into feature-metacluster
2022-07-27 08:55:10 -07:00
Jon Fu
6dbaf46ec8
restructure test workload impl and map
2022-07-26 13:53:04 -07:00
Jon Fu
21309c2af8
Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys
2022-07-26 11:15:56 -07:00
A.J. Beamon
59ccdc35ab
Disallow configuring the same tenant parameter more than once at a time. Made some slight changes to serialization code for tenants.
2022-07-26 09:04:29 -07:00
A.J. Beamon
f2a8af9155
Better API version handling for fdbcli tenant commands
2022-07-26 09:04:29 -07:00
A.J. Beamon
a64693518a
Add support for tenant groups
2022-07-26 09:04:29 -07:00
A.J. Beamon
2990294f6a
Make the metacluster experimental by updating the name of the command used to configure one.
2022-07-24 19:11:25 -07:00
Jon Fu
84269bac3e
Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys
2022-07-22 16:42:27 -07:00
A.J. Beamon
2b6f349649
Better API version handling for fdbcli tenant commands
2022-07-22 15:37:48 -07:00
Jon Fu
43c7146d86
Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys
2022-07-22 15:21:13 -07:00
A.J. Beamon
b308aa6604
Merge branch 'feature-tenant-groups' into feature-metacluster
2022-07-22 11:38:23 -07:00
A.J. Beamon
237081905d
Add support for tenant groups
2022-07-22 09:27:31 -07:00
A.J. Beamon
17146c484b
Use key-backed types for tenants. Add a tenant state field that will be used in upcoming work. Some other tenant related refactoring.
2022-07-21 20:33:28 -07:00
A.J. Beamon
3e1763e7ac
Merge branch 'main' into feature-metacluster
2022-07-21 14:48:53 -07:00
A.J. Beamon
410f27412b
Merge pull request #7620 from sfc-gh-ajbeamon/make-tuple
...
Add a Tuple::makeTuple function to easily construct a tuple
2022-07-20 17:09:10 -07:00
A.J. Beamon
279296c29f
Merge branch 'tenant-metadata-change' into feature-metacluster
...
# Conflicts:
# fdbclient/SystemData.cpp
# fdbclient/Tenant.cpp
# fdbclient/include/fdbclient/SystemData.h
# fdbclient/include/fdbclient/Tenant.h
# fdbclient/include/fdbclient/TenantManagement.actor.h
# fdbserver/TenantCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-20 09:18:27 -07:00
A.J. Beamon
190ad8c7e9
Convert existing tuple usages to use Tuple::makeTuple()
2022-07-19 13:45:59 -07:00
Markus Pilman
1de37afd52
Make TEST macros C++ only ( #7558 )
...
* proof of concept
* use code-probe instead of test
* code probe working on gcc
* code probe implemented
* renamed TestProbe to CodeProbe
* fixed refactoring typo
* support filtered output
* print probes at end of simulation
* fix missed probes print
* fix deduplication
* Fix refactoring issues
* revert bad refactor
* make sure file paths are relative
* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
A.J. Beamon
1b81e72604
Add a Tuple::makeTuple function to easily construct a tuple. Update Tuple to allow all types to be passed via .append() so they can be used with makeTuple.
2022-07-19 11:50:58 -07:00
A.J. Beamon
c7c7bd5f17
Undo change to mark tenant modes non-experimental. Other minor cleanup.
2022-07-19 09:18:53 -07:00
A.J. Beamon
8f77048e95
Add the ability to configure tenant groups in a metacluster
2022-07-18 11:24:49 -07:00
A.J. Beamon
860d3843cc
Merge remote-tracking branch 'origin/feature-tenant-groups' into feature-metacluster
2022-07-16 19:33:26 -07:00
A.J. Beamon
c08592368f
Add support for tenant groups
2022-07-15 21:18:41 -07:00
A.J. Beamon
0fe6f6ad36
Merge remote-tracking branch 'origin/update-prefix-encoding-in-tenant-metadata-json' into feature-metacluster
...
# Conflicts:
# bindings/python/tests/fdbcli_tests.py
# fdbcli/TenantCommands.actor.cpp
# fdbclient/include/fdbclient/TenantManagement.actor.h
# fdbclient/include/fdbclient/TenantSpecialKeys.actor.h
# fdbserver/workloads/TenantManagementWorkload.actor.cpp
2022-07-14 14:32:29 -07:00
Jon Fu
7f3b51f31a
initial commit to introduce tenant renaming to special keys
2022-07-13 15:59:52 -07:00
A.J. Beamon
cb499fa5db
Merge branch 'main' into feature-metacluster
2022-07-13 15:28:34 -07:00
A.J. Beamon
91ccdbcb03
Use a more standard encoding in the JSON metadata reported for tenants.
2022-07-13 15:22:19 -07:00
A.J. Beamon
a3f970cf3a
Parse integer options in fdbcli rather than passing the bytes unparsed. This allows specifing the string "1" instead of "\x01\x00\x00\x00\x00\x00\x00\x00".
2022-07-11 16:39:59 -07:00
A.J. Beamon
712f40b727
Add support for tenant groups
2022-07-08 15:59:25 -07:00
A.J. Beamon
b9cc5389b1
Undo some changes that aren't needed
2022-07-07 14:48:36 -07:00
A.J. Beamon
a32cc5218c
Fix some merge issues
2022-07-07 11:55:31 -07:00
A.J. Beamon
1519f24f77
Merge branch 'main' into feature-metacluster
2022-07-07 09:35:40 -07:00
Jon Fu
df1f108b78
Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename
2022-07-06 14:10:22 -07:00
A.J. Beamon
aea4d802c6
Merge branch 'main' into feature-metacluster
2022-06-30 15:08:09 -07:00
A.J. Beamon
2f67328a0c
Update the tenant special keys submodule to support multiple sub-ranges. This will enable future work that allows configuring tenants at the same time as creating them.
2022-06-30 15:03:37 -07:00
Jon Fu
5e7bb0aa21
Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-rename
2022-06-29 13:20:19 -07:00
Trevor Clinkenbeard
db769667ae
Merge pull request #7112 from sfc-gh-tclinkenbeard/global-tag-throttling3
...
Create Global Tag Throttler
2022-06-29 10:06:00 -07:00
sfc-gh-tclinkenbeard
2134fa9a0c
Merge remote-tracking branch 'origin/main' into global-tag-throttling3
2022-06-28 19:51:41 -07:00
A.J. Beamon
df90d87903
Add missing return statement
2022-06-28 16:03:08 -07:00
A.J. Beamon
e1a93988ef
Merge branch 'main' into feature-metacluster
2022-06-28 14:58:07 -07:00
Jon Fu
5fee5e1a73
add renameTenant back to TenantAPI after rebase
2022-06-28 11:54:03 -07:00
Jon Fu
869c6fcac5
reintroduce renameTenantCommandActor as a pass-through function
2022-06-28 11:35:40 -07:00
Jon Fu
904ac358ce
call directly into management api and remove extra function
2022-06-28 11:35:38 -07:00
Jon Fu
5f4f95db4d
fix unreachable code
2022-06-28 11:34:54 -07:00
Jon Fu
ddfdbdcb09
initial commit introducing renameTenant function
2022-06-28 11:34:44 -07:00
sfc-gh-tclinkenbeard
086e4bff06
Merge remote-tracking branch 'origin/main' into global-tag-throttling3
2022-06-28 10:18:13 -07:00
Markus Pilman
fc2f695b15
Merge remote-tracking branch 'origin/main' into features/cmake-refactor-dirs
2022-06-28 10:41:07 -06:00
A.J. Beamon
fcd91f676a
Merge pull request #7436 from sfc-gh-ajbeamon/tenant-reorganization
...
Tenant code refactoring
2022-06-28 09:16:19 -07:00
Markus Pilman
1bdbd89839
Merge remote-tracking branch 'origin/main' into features/cmake-refactor-dirs
2022-06-28 08:52:03 -06:00
Markus Pilman
4ac236eeba
fdbcli compiling
2022-06-27 19:20:18 -06:00
Markus Pilman
03d913a1de
Flow compiling
2022-06-27 17:05:55 -06:00
A.J. Beamon
f990fba44d
Add support for getting tenant metadata as a JSON document.
2022-06-27 12:34:40 -07:00
A.J. Beamon
4bafe77889
Some refactoring of tenant code:
...
* extract tenant management into its own file and namespace
* rename the tenant management workload source file
* extract tenant special keys functions to a separate file
* extract some helper functions to GenericTransactionHelper.h
* convert StringRef -> TenantNameRef
* move some TenantMapEntry implementation into the cpp file
* add some helper functions to decode/encode a tenant mode
2022-06-27 12:32:49 -07:00
A.J. Beamon
eccf244ba6
Add an index that allows us to allocate tenants from the non-full cluster with the most tenants already assigned to it.
2022-06-27 12:28:55 -07:00
A.J. Beamon
4b69723a2f
Cleanup metadata associated with a cluster when force removing it. Move some metacluster metadata into the normal key-space. Cleanup some todos.
2022-06-24 14:55:57 -07:00
Markus Pilman
d35445a868
enforce include modularization in cmake
2022-06-23 14:37:35 -06:00
A.J. Beamon
9f3819752f
Change the command to create a metacluster from using 'configure tenant_mode=management' to 'metacluster create <NAME>'. Distribute this name to all processes in a metacluster. Eliminate the tenant mode entirely from metacluster clusters, instead relying on a metacluster registration key.
2022-06-22 12:15:43 -07:00
A.J. Beamon
96fad987eb
Add new command to get total metacluster capacity. Fix bug where removing a data cluster could make it impossible to create new tenants on it. Add knobs for max data clusters and max tenants.
2022-06-17 10:42:03 -07:00
sfc-gh-tclinkenbeard
308e0113a8
Update fdbcli quota usage output
2022-06-16 16:26:45 -07:00
sfc-gh-tclinkenbeard
99d243197e
Add fdbcli quota command
2022-06-16 14:07:16 -07:00
A.J. Beamon
2907aafa60
Merge branch 'main' into feature-metacluster
2022-06-02 12:18:42 -07:00
Jingyu Zhou
db5508520c
Merge pull request #7249 from johscheuer/fix-fdbcli-exclude-check
...
Check if all specific processes are excluded
2022-05-26 15:01:01 -07:00
Johannes M. Scheuermann
8277965158
Correct variable for for loop to iterate over inprogress exclusions
2022-05-25 17:21:15 +01:00
Johannes M. Scheuermann
0bc8e2cea6
Check if all specific processes are excluded
2022-05-25 17:08:02 +01:00
A.J. Beamon
9773261a03
Expand support for tenant groups. Track where each tenant group is assigned and used groups to help assign tenants. Fix tenant state tracking bugs. Add help hints for metacluster remove FORCE.
2022-05-24 11:58:40 -07:00
Xiaoxi Wang
fd35fde481
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-05-23 15:09:03 -07:00
Lukas Joswiak
475b157310
Add `get` command to versionepoch help text
2022-05-23 11:45:18 -07:00
Xiaoxi Wang
73624bcd2a
Merge remote-tracking branch 'upstream/main' into readaware
2022-05-23 11:17:38 -07:00
A.J. Beamon
d784173f7f
Add first implementation of tenant creation and deletion in a metacluster
2022-05-20 15:21:21 -07:00
Chaoguang Lin
5e8c3130e6
Extend the reboot interface to accept comma-delimited string;
...
Only fetch requested worker interfaces in rebootWorkerActor
Add killall test to multiprocess_fdbcli_tests
Make the rebootWorker only sends requests if all the requested workers are valid
Add comments; Organize print texts; Fix the bug clear the map every time run kill/suspend/expensive_data_check
2022-05-20 12:06:33 -07:00
A.J. Beamon
69261f9f10
Merge branch 'main' into feature-metacluster
...
# Conflicts:
# fdbclient/MultiVersionTransaction.actor.cpp
# fdbclient/MultiVersionTransaction.h
# fdbserver/tester.actor.cpp
2022-05-19 09:11:30 -07:00
A.J. Beamon
367e59dc33
Add support for getting tenant and cluster metadata in fdbcli in a JSON format
2022-05-18 14:53:25 -07:00
A.J. Beamon
98c3813431
Refactor the register logic so that only two write transactions take place. The first is to the data cluster, the second to the management cluster.
2022-05-18 11:21:39 -07:00
A.J. Beamon
e8e26c9f7c
Remove the internal special key-space mechanism (at least for now), instead relying on the local logic to implement the metacluster functions. Add ability to force remove a cluster. Improve idempotency of register operation.
2022-05-18 08:18:25 -07:00
Xiaoxi Wang
382f0fc4a2
merge upstream/main
2022-05-17 10:20:51 -07:00
Evan Tschannen
583120514f
fixed compilation errors
2022-05-12 17:19:25 -07:00
Evan Tschannen
c51ad847d6
revert "kill" command to previous 7.0 behavior because the current implementation is killing processes one at a time
2022-05-12 13:12:23 -07:00
A.J. Beamon
85019612d4
Remove subordinate mode and instead use a key in the system key-space to track metacluster membership. Use this key to determine if a cluster is already part of a metacluster or can be configured away from required mode. Disallow configuring to or from a management cluster that has data.
2022-05-11 17:10:55 -07:00
Vishesh Yadav
f14baf2af8
clang-format changes
2022-05-09 14:54:51 -07:00
Vishesh Yadav
9173e2e19b
Move GlobalConfig to DatabaseContext
2022-05-09 14:54:51 -07:00
Vishesh Yadav
7578d5ebc7
Create GlobalConfig object for each database instance
...
Currently, GlobalConfig is a singleton that means for each process there is only
one GlobalConfig object. This is bug from clients perspective as a client can
keep connections to several databases. This patch tracks GlobalConfig for each
database using an unordered_map in flowGlobals.
We discovered this bug while testing multi-version client, where the client got
stuck. This was lucky, as normally it'd just write down config to the wrong
database.
2022-05-09 14:54:51 -07:00
A.J. Beamon
80a3c8dd50
All metacluster operations to the management cluster go through the management cluster special key space
2022-05-09 14:50:07 -07:00
A.J. Beamon
7c3b99f5bd
Add support for connection to clusters via a connection string. Start work on multi-cluster operations with data cluster registration.
2022-05-06 09:53:53 -07:00
Xiaoxi Wang
992ff59bfb
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-05-05 09:57:22 -07:00
A.J. Beamon
0c41225d76
Add support for setting and storing the tenant group for a tenant
2022-05-05 08:57:01 -07:00
Trevor Clinkenbeard
0069ce51a5
Merge pull request #7054 from sfc-gh-tclinkenbeard/remove-non-tls-support
...
Remove `DISABLE_TLS` CMake option
2022-05-03 22:13:07 -07:00
Xiaoxi Wang
7c37d172b9
solve some comments
2022-05-03 17:21:08 -07:00
Xiaoxi Wang
269d85daa8
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-05-03 13:37:56 -07:00
A.J. Beamon
7e860dc92c
Some initial metacluster implementation
2022-05-03 12:53:35 -07:00
sfc-gh-tclinkenbeard
225146176d
Apply clang-format to fdbcli.actor.cpp and Net2.actor.cpp
2022-05-03 12:13:09 -07:00
sfc-gh-tclinkenbeard
06825775db
Fix formatting of lines with TLS_OPTION_FLAGS
2022-05-02 22:56:06 -07:00
sfc-gh-tclinkenbeard
7f05221cfe
Removed TLS_DISABLED macro
2022-05-02 22:15:27 -07:00
Xiaoxi Wang
69985ba251
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-05-02 10:53:22 -07:00
A.J. Beamon
43c2ca35a5
Move fdbcli command and hint generators into the files implementing the command.
2022-05-02 08:39:59 -07:00
Renxuan Wang
c69a07a858
Check in the new Hostname logic. ( #6926 )
...
* Revert #6655 .
20220407-031010-renxuan-c101052c21da8346 compressed=True data_size=31004844 duration=4310801 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=1:04:15 sanity=False started=100047 stopped=20220407-041425 submitted=20220407-031010 timeout=5400 username=renxuan
* Revert #6271 .
20220407-051532-renxuan-470f0fe6aac1c217 compressed=True data_size=30982370 duration=3491067 ended=100002 fail_fast=10 max_runs=100000 pass=100002 priority=100 remaining=0 runtime=0:59:57 sanity=False started=100141 stopped=20220407-061529 submitted=20220407-051532 timeout=5400 username=renxuan
* Revert #6266 .
Remove resolving-related functionalities in connection string. Connection string will be used for storing purpose only, and non-mutable.
20220407-175119-renxuan-55d30ee1a4b42c2f compressed=True data_size=30970443 duration=5437659 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=0:59:31 sanity=False started=100154 stopped=20220407-185050 submitted=20220407-175119 timeout=5400 username=renxuan
* Add hostname to coordinator interfaces.
* Turn on the new hostname logic.
* Add the corresponding change in config txns.
The most notable change is before calling basicLoadBalance(), we need to call tryInitializeRequestStream() to initialize request streams first.
Passed correctness tests.
* Return error when hostnames cannot be resolved in coordinators command.
* Minor fixes.
2022-04-27 21:54:13 -07:00
Xiaoxi Wang
0639810b66
merge upstream/main
2022-04-22 11:09:15 -07:00
Zhe Wang
6c9ff6ee5e
Add sharded rocksdb type ( #6862 )
...
* add-sharded-rocksdb-type
* address comments
Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
2022-04-21 22:53:14 -04:00
Xiaoxi Wang
d17b36acf8
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-04-15 11:10:09 -07:00
Andrew Noyes
29cf5f1fbf
Fix an ASSERT when an fdbcli command times out ( #6857 )
...
* Re-throw operation_cancelled
There's a few places in fdbcli where we don't rethrow operation
cancelled but wait on a future. It's very unusual that you don't want to
rethrow operation_cancelled.
* Update ASSERT
It's possible to get error_code_broken_promise here if the network has
already shutdown.
2022-04-14 12:09:25 -07:00
Xiaoxi Wang
8d3f851495
merge upstream/mainA
2022-04-12 17:03:09 -07:00
Xiaoxi Wang
ed97a35dc0
Merge branch 'main' into readaware
2022-04-12 16:47:15 -07:00
Xiaoxi Wang
61a1f7683b
fix dd command line read special key space error
2022-04-11 22:49:21 -07:00
Aaron Molitor
cbaef8f03b
update version to 7.2.0
2022-04-11 23:23:27 -05:00
Xiaoxi Wang
6c841cd32b
merge readaware-better
2022-04-11 17:09:39 -07:00
Xiaoxi Wang
82e5859e03
allow safeThreadFutureToFuture
2022-04-11 14:35:12 -07:00
Lukas Joswiak
73a7c32982
Add fdbcli command to read/write version epoch ( #6480 )
...
* Initialize cluster version at wall-clock time
Previously, new clusters would begin at version 0. After this change,
clusters will initialize at a version matching wall-clock time. Instead
of using the Unix epoch (or Windows epoch), FDB clusters will use a new
epoch, defaulting to January 1, 2010, 01:00:00+00:00. In the future,
this base epoch will be modifiable through fdbcli, allowing
administrators to advance the cluster version.
Basing the version off of time allows different FDB clusters to share
data without running into version issues.
* Send version epoch to master
* Cleanup
* Update fdbserver/storageserver.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
* Jump directly to expected version if possible
* Fix initial version issue on storage servers
* Add random recovery offset to start version in simulation
* Type fixes
* Disable reference time by default
Enable on a cluster using the fdbcli command `versionepoch add 0`.
* Use correct recoveryTransactionVersion when recovering
* Allow version epoch to be adjusted forwards (to decrease the version)
* Set version epoch in simulation
* Add quiet database check to ensure small version offset
* Fix initial version issue on storage servers
* Disable reference time by default
Enable on a cluster using the fdbcli command `versionepoch add 0`.
* Add fdbcli command to read/write version epoch
* Cause recovery when version epoch is set
* Handle optional version epoch key
* Add ability to clear the version epoch
This causes version advancement to revert to the old methodology whereas
versions attempt to advance by about a million versions per second,
instead of trying to match the clock.
* Update transaction access
* Modify version epoch to use microseconds instead of seconds
* Modify fdbcli version target API
Move commands from `versionepoch` to `targetversion` top level command.
* Add fdbcli tests for
* Temporarily disable targetversion cli tests
* Fix version epoch fetch issue
* Fix Arena issue
* Reduce max version jump in simulation to 1,000,000
* Rework fdbcli API
It now requires two commands to fully switch a cluster to using the
version epoch. First, enable the version epoch with `versionepoch
enable` or `versionepoch set <versionepoch>`. At this point, versions
will be given out at a faster or slower rate in an attempt to reach the
expected version. Then, run `versionepoch commit` to perform a one time
jump to the expected version. This is essentially irreversible.
* Temporarily disable old targetversion tests
* Cleanup
* Move version epoch buggify to sequencer
This will cause some issues with the QuietDatabase check for the version
offset - namely, it won't do anything, since the version epoch is not
being written to the txnStateStore in simulation. This will get fixed in
the future.
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-04-08 12:33:19 -07:00
Xiaoxi Wang
5113277e9a
Merge branch 'main' into readaware
2022-04-08 11:50:01 -07:00
Xiaoxi Wang
411550580e
merge
2022-04-08 11:49:33 -07:00
neethuhaneesha
5311d93c72
Merge pull request #6783 from neethuhaneesha/rocksdbEngineType
...
Adding warning on configuring rocksdb storage until production tested.
2022-04-07 13:29:12 -07:00
Xiaoxi Wang
150b4318aa
refactor datadistribution command; try dual-mode code
2022-04-06 22:10:23 -07:00
Yi Wu
994b8c92f8
Add option to limit resident memory and remove default memory limit ( #6719 )
...
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.
Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.
closes #6671 , closes #6672
2022-04-06 20:06:24 -07:00
Neethu Haneesha Bingi
5d3981821e
Adding warning on configuring rocksdb storage until production tested.
2022-04-06 13:24:57 -07:00
Xiaoxi Wang
aba9d85560
merge main
2022-04-06 09:57:52 -07:00
Jingyu Zhou
f68fd28d73
Refactor duplicated code into IKnobCollection::setupKnobs()
2022-04-05 02:06:38 -07:00
Xiaoxi Wang
d6d4596c35
Merge pull request #6709 from sfc-gh-ajbeamon/fix-fdbcli-bugs
...
Fix issues with command completion for exclude and storage_migration_type
2022-03-30 19:53:49 -07:00
Andrew Noyes
d727e7648e
Fix a few memory issues found by ASAN
2022-03-29 12:39:12 -07:00
A.J. Beamon
23fcd8c076
Fix issues with command completion for exclude and storage_migration_type. Add missing documentation for tenant_mode in one spot.
2022-03-29 09:07:33 -07:00
Xiaoxi Wang
d93b57dd88
conflict solving
2022-03-24 20:45:51 -07:00
Xiaoxi Wang
1b631a9263
solve conflict with main
2022-03-24 16:29:11 -07:00
Josh Slocum
f27475e2f4
Merge branch 'main' into blob_integration
2022-03-22 11:41:58 -05:00
sfc-gh-tclinkenbeard
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
A.J. Beamon
fd81ef99c0
Merge pull request #6561 from sfc-gh-ajbeamon/fdb-tenant-fdbcli
...
Add fdbcli support for tenants
2022-03-21 11:26:10 -07:00
sfc-gh-tclinkenbeard
be875e040d
Prevent infinite loop on invalid input to 'throttle off' command
2022-03-18 20:22:49 -07:00
sfc-gh-tclinkenbeard
004e73b2f1
Add TagSet::toString method
2022-03-18 20:10:56 -07:00
sfc-gh-tclinkenbeard
fdab73d7b3
Rename TagSet variables in throttleCommandActor.
...
Previously these were named "tags", conflicting with a state variable,
and creating confusing issues when they go out of scope after a wait
statement.
2022-03-18 19:20:25 -07:00
sfc-gh-tclinkenbeard
6eaa20078f
Add ContainsRecommended boolean parameter
2022-03-18 18:12:50 -07:00
sfc-gh-tclinkenbeard
63ec6d5f5b
Add static defaultThrottleListLimit in ThrottleCommand.actor.cpp
2022-03-18 17:14:34 -07:00
Josh Slocum
37e7c80f26
Merge branch 'main' into blob_integration
2022-03-17 18:45:42 -05:00
A.J. Beamon
a23add6bc4
Add fdbcli test for tenants. Add documentation for new fdbcli tenant commands. Various output cleanup. Fix limit parsing bug in listtenants command. Update gettenant output format.
2022-03-17 12:10:39 -07:00
A.J. Beamon
bd64781ad9
Use special keys to manage tenants
2022-03-17 12:10:39 -07:00
A.J. Beamon
4aa053d342
Update tenant mode options in fdbcli
2022-03-17 12:10:39 -07:00