* Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (no need to guard)
* Enable the relaxed special key access in transactions that need to use special key-space APIs introduced in 7.2
* Encryption data at-rest db-config
Description
diff-1: Handle 'force' updates to encryption_at_rest db-config
Major changes proposed:
1. Introduce 'encryption_data_at_rest_mode" 'configure new'
option to enable Encryption data at-rest. The feature is disabled
by default.
2. The configuration is meant to be set at the time of database
creation, addition checks will be done to avoid updating the config
in subsequent PR.
3. DatabaseConfiguration validity check to account for "tenant_mode"
set to `required` if Encryption data at-rest is selected given
EncryptionDomain matches Tenant boundaries.
Testing
devCorrectness - 100K
Description
Improve FDB architecture documentation to explicitly mention that
Stateless processes may get recruited on a single cluster process
Testing
* Add the verify option for \xff\xff/worker_interfaces
* Remove unused code
* update documentations
* update documentations
* solve comments from review
* update some of the comments to be more clear
* throttle the cluster when blob workers fall behind
* do not throttle on blob workers if they are not enabled
* remove an unnecessary actor
* fixed a compile error
* fetch blob worker metrics at the same interval as the rate is updated, avoid fetching the complete blob worker list too frequently
* fixed another compilation bug
* added a 5 second delay before bw throttling to prevent false positives caused by the 100e6 version jump during recovery. Lower the throttling thresholds to react much quicker to bw lag.
* fixed a number of problems
* changed the minBlobVersionRequest to look at storage server versions since this will be a lot more efficient
* fix: do not let desired go backwards
* fix: track the version of notAtLatest changefeeds for throttling
* ratekeeper now throttled blob workers by estimating the transaction per second throughput of the blob workers
* added metrics for blob worker change feeds
* added a knob to disable bw throttling
* fixed the transaction options in blob manager
Currently GRV is reporting proxy_memory_limit_exceeded error which has
error message claiming Commit proxy failing. This split should remove
such confusion.
* Add getRange test coverage for SpecialKeyRangeAsyncImpl
* Fix the bug in SpecialKeyRangeAsyncImpl found by the test
* Refactor ConflictingKeysImpl::getRange to use containedRanges to simplify the code
* Fix file format
* Initialize SpecialKeyRangeAsyncImpl cache with correct end key
* Add release notes
* Revert "Refactor ConflictingKeysImpl::getRange to use containedRanges to simplify the code"
This reverts commit fdd298f469.
* Add an internal C API to support memory connection records
* Track shared state in the client using a unique and immutable cluster ID from the cluster
* Add missing code to store the clusterId in the database state object
* Update some arguments to pass by const&
1. Support virtual hosting endpoint.
2. On-premise s3 compatible storage service may use IP instead of s3 form domain name,
especially for development/test environment.
Instead of parsing service and region from domain name,
1). Hard code "s3" as service name in v4 signature
2). Add new parameter to allow pass region name from url
3. Fix creating bucket issue on aws, adding request body.
Since memory is now limited with RSS size, add RSS size in status json for
reporting. Also change how available_bytes is calculated from:
(available + virtual memory) * process_limit / machine_limit
to:
(available memory) * process_limit / machine_limit
Use CompletableFuture::join instead of CompletableFuture::get
when blocking on futures in java tutorial snippets. This avoids
the unreported checked exceptions which prevent the snippets from
compiling (and is in line with the full tutorial code).
Changing `memory` option to limit resident memory instead of virtual memory, in config file and fdbserver/fdbbackup/fdbcli command-line argument. Since `rlimit` doesn't support limiting virtual memory, the current implementation have both of fdbmonitor and the fdbserver/fdbbackup process checking process RSS periodically and kill and restart the process if the limit is exceeded.
Adding a new `memory_vsize` option to limit virtual memory, if backward-compatible behavior is desired.
closes#6671, closes#6672
Jon Fu