Disable sharded rocks storage for downgrade tests where we
Need to keep knob "shard_encode_location_metadata" so that downgrade tests
can pass the second phase.
Changes:
1. Update `ConfigureDatabase` workload to test with Redwood, while previously it does not. Also when encryption is enabled, only test with Redwood and not test the migration to other storage engine types, as currently only Redwood supports encryption.
2. Update multiple restart tests so that when testing upgrade from/downgrade to 7.2, disable encryption. This is due to recent change to make encryption controlled by DB config instead of a knob make 7.3 encryption incompatible with 7.2 encryption, and 7.2 encryption is considered an incomplete feature. This is done by splitting `from_7.1.0` test directory into `from_7.1.0_until_7.2.0`/`from_7.2.0_until_7.3.0`/`from_7.3.0`, duplicating every test to these directories and add `disableEncryption=true` when needed. Except...
3. For tests that run `SnapTest`, keep them in `from_7.1.0_until_7.2.0` directory. These tests could fail with 7.2/7.3 restart test, and due to separate Joshua issue, these failures are not exposed when they stay in `from_7.1.0` directory. The plan is to only keep these snapshot tests in `from_7.3.0` (or the directory for restart from the latest version) once issues are fixed.
Otherwise, the Attrition can RebootAndDelete tlogs in remote DC such that the
remote is unusable and blocking recovery to fully_recovered state. In fact,
the FirstCycleTest can only reach the accepting_commits state.
In the part 2 of the restarting test, the runTests() wait for quietDatabase()
to reach fully fully_recovered state, but was stuck in the accepting_commits
state.
* Fix Redwood tree height overgrowth when EaR and tenant page split are enabled, by removing the buildNewSubtree() logic.
* Fixing incorrect page upper bound for the last page created by writePages() without the buildNewSubtree() logic.
* Enable tenant page split if encryption mode is domain-aware encryption.
* Related test fixes:
- In simulation, pass encryption mode to storage/Redwood via knobs. This is a workaround to enable testing with Redwood encryption before we correctly pass the encryption mode via db config. Also temporarily disable tenant page split for restart tests.
- Disable raw access in FuzzApiCorrectness test if domain-aware encryption is enabled, to avoid test timeout
- Disable encryption for DrUpgradeRestart test, which is likely to fail due to a rare EKP deadlock issue blocking recovery. Will re-enable after the deadlock issue is fixed.
Originally, storageEngineExcludeTypes was storageEngineExcludeType, and
-1 signified "don't ignore any storage engines". This is no longer
meaningful now that it's a list.
Using the feature to set knobs from test spec files appears to require
using toml, so also rewrite CycleTestRestart-{1,2}.txt to
CycleTestRestart-{1,2}.toml
Previously to get around the issue that EKP is not present when restart test switching encryption from on to off and read encrypted data, EKP was made to start in simulation regardless of encryption knob. This PR revert that change, and instead force restart test not to change encryption knob, by passing previous encryption knob through restartInfo.ini file. Also since we don't allow downgrading an encrypted cluster to previous version, disable encryption in downgrade tests.
Also adding an assert to allow reading encrypted mutations only if encryption knob is on. We may reconsider allowing switching encryption on/off for existing cluster, but for now we don't allow it.
Yi Wu