* added dynamic write amp calculations for blob granule compaction
* changing blob worker parallelism counts to bytes budget to handle less uniform operation sizes
* more snapshotting parallelism for behind feeds
* add a bit of observability when this happens
* adding knobs
* typo
* adjusting some knobs up with buggified granule size
* fixing bugs in dynamic write amp
* fixing formatting
* fixing bug in knob buggification
* fix formatting
The metacluster status command in fdbcli currently reports some useful metacluster information when run on a
management cluster. We should update this command to report a status even on data clusters of a metacluster and
standalone clusters that do not belong to any metacluster.
- On data clusters, this would report that the cluster is a data cluster as well its name and the name of the metacluster it is a part of.
- On standalone clusters, status should report that the cluster is not part of a metacluster.
Test plan:
- CI
- Manual test
- Added new test `metacluster_fdbcli_tests.py` that can be run with ctest `ctest -R metacluster_fdbcli_tests`
Changes:
1. Update `ConfigureDatabase` workload to test with Redwood, while previously it does not. Also when encryption is enabled, only test with Redwood and not test the migration to other storage engine types, as currently only Redwood supports encryption.
2. Update multiple restart tests so that when testing upgrade from/downgrade to 7.2, disable encryption. This is due to recent change to make encryption controlled by DB config instead of a knob make 7.3 encryption incompatible with 7.2 encryption, and 7.2 encryption is considered an incomplete feature. This is done by splitting `from_7.1.0` test directory into `from_7.1.0_until_7.2.0`/`from_7.2.0_until_7.3.0`/`from_7.3.0`, duplicating every test to these directories and add `disableEncryption=true` when needed. Except...
3. For tests that run `SnapTest`, keep them in `from_7.1.0_until_7.2.0` directory. These tests could fail with 7.2/7.3 restart test, and due to separate Joshua issue, these failures are not exposed when they stay in `from_7.1.0` directory. The plan is to only keep these snapshot tests in `from_7.3.0` (or the directory for restart from the latest version) once issues are fixed.
* encrypted change feeds, still missing re-encryption of mutations which have been modified
* removed inverted filtering and reduced the number of comparisons for calculating clear intersection
* fix merge conflict
* fixed another merge conflict
* encrypt mutations which have been modified
* renamed encrypted to encryptedMutation
* fix formatting
* format nativeAPI
* format code
* more formatting fixes
* fix: encrypted clear ranges which have been modified by shard boundaries need to be re-encrypted
* fix formatting
Otherwise, the Attrition can RebootAndDelete tlogs in remote DC such that the
remote is unusable and blocking recovery to fully_recovered state. In fact,
the FirstCycleTest can only reach the accepting_commits state.
In the part 2 of the restarting test, the runTests() wait for quietDatabase()
to reach fully fully_recovered state, but was stuck in the accepting_commits
state.
Note: Server processes started getting reported as clients since 7.1.0
(not sure if this change in behavior was intentional or not), and this
breaks the operator upgrade logic.
Jon Fu