Commit Graph

7458 Commits

Author SHA1 Message Date
Jingyu Zhou f86058fba6
Remove the usage of txsTag (#11688)
* Add assertions to code paths with txsTag

txsTag should be obsolete by now, since it's used in 6.1, which is no longer
supported for upgrade.

* Actually remove txsTag usage

20240926-225930-jzhou-7ed3304c415ae65e

* Remove more code

20240926-235242-jzhou-7ed3304c415ae65e

* Disable two verbose trace events

They can cause TraceTooManyLines errors.
2024-09-30 07:53:37 -04:00
neethuhaneesha 5637f23231
Increasing minimum age to wiggle to avoid re-wiggling migrated rocksdb storage servers (#11683) 2024-09-26 12:59:37 -07:00
Yao Xiao 83dd1f202e
Fix block cache size error and improve logging. (#11681) 2024-09-24 13:08:31 -07:00
Jingyu Zhou b872a8ea57
Merge pull request #11637 from neethuhaneesha/direct_io_enabling
Enabling rocksdb direct_io and wiggle knobs
2024-09-19 10:12:14 -07:00
Jingyu Zhou 712f88a1ff More protocol version related code removal
Removed code handle old protocol versions, i.e., before 7.1
2024-09-18 13:28:06 -07:00
Jingyu Zhou fc30fc269e Remove dead code after removing tagLocalityUpgraded usage
20240918-170752-jzhou-33111b2c3e6776aa
2024-09-18 11:23:09 -07:00
Jingyu Zhou 7b76561bb9 Remove tagLocalityUpgraded usage at various places
Since we have removed old tlog implementation, so the code path using this tag
can be deleted to simplify the code.
2024-09-18 11:23:09 -07:00
Jingyu Zhou 80ca71833b Make xxhash checksum the default for TLog
Update downgrade tests to use the xxhash.
2024-09-17 12:46:42 -07:00
Keijo Kapp b9926aefe5
Fix the key range affected by setting version stamped key (#11424)
When doing version stamped key operation, the affected key range should
start from the next read version, not the current one.
2024-09-14 14:52:01 -07:00
Syed Paymaan Raza e1c7cdd3e3
[CC+Worker] Enable WORKER_HEALTH_MONITOR related knobs in simulation tests (#11657) 2024-09-12 23:27:47 -07:00
Jingyu Zhou 2313fdaa0e
Add rocksdb, sharded rocksdb to configure workload (#11654)
* Add rocksdb, sharded rocksdb to configure workload

Also remove mentioning of ssd-redwood-1-experimental.

* Fix test failure when SHARD_ENCODE_LOCATION_METADATA is off
2024-09-12 21:03:06 -07:00
Yao Xiao 289d02899f
Add knobs for caching index blocks. (#11650) 2024-09-11 22:58:26 -07:00
neethuhaneesha 8ff623e523 Enabling rocksdb direct_io and wiggle knobs 2024-09-11 09:28:44 -07:00
Sepeth 3854dbfe4d
Upgrade fmt from 8.1.1 to 11.0.2 (#11601)
And, added via cmake FetchContent, and removed contrib/fmt-8.1.1
2024-09-10 14:42:43 -07:00
Jingyu Zhou d730db521a
Fix a Valgrind error (#11645)
buggifyShortReadWindow used unitialized variable ENABLE_VERSION_VECTOR.
2024-09-10 15:29:36 -04:00
Dan Lambright 5eafd46351
Disable version vector on batches with backed up mutations (#11634)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-09-09 21:18:59 -04:00
Zhe Wang 5ee0db13e6
Fix external timeout with ShardedRocksDB and re-enable ShardedRocksDB in simulation tests (#11638)
* speedup sharded rocksdb in simulation

* re-enable shardedrocksdb and disable physical shard move
2024-09-08 10:57:55 -07:00
hao fu f092e19026 address comments 2024-09-05 16:07:01 -07:00
hao fu 5295920ded Check bucket exist or not, rather than listBucket 2024-09-05 16:06:18 -07:00
hao fu 640b0fe7f3 Finish testing, set default to false 2024-09-05 15:17:34 -07:00
hao fu 04e02c2908 Retry with dryrun in the presence of s3 token error
s3 token is from local disk and might be expired or invalid,
before this change backup retries to upload data to s3 indefinitely,
thus it is a waste of network bandwidth.

Now retry with a get request of list all buckets in the case of
s3 token error, and only retry the upload when token error disappears.
2024-09-05 15:17:27 -07:00
Syed Paymaan Raza 48064f6cf1
Make some codeprobes rare (#11607)
* Make BlobGranule code probes rare

* Make encryption related code probes rare

* fixup! Fix formatting
2024-08-26 22:33:38 -07:00
Jingyu Zhou cf188a99d4 Convert most actors in Watches workload into coroutines
There is one left that doesn't seem to have a good way for conversion. To make
sure the converted code is behaving correctly, I added a few CodeProbes to
ensure code coverage.
2024-08-23 12:11:57 -07:00
Zhe Wang 3305d2e3ee
fix storage engine selection (#11586) 2024-08-20 09:24:30 -07:00
Jingyu Zhou 5d5f1a2dc7
Merge pull request #11575 from brownleej/backup-transaction-options
Capture default database options in fdbbackup in a local variable.
2024-08-16 14:16:38 -07:00
Zhe Wang 6c502e9707
Solve RocksDB external timeout error and re-enable RocksDB simulation tests (#11577)
* init knob tune

* include rocksdb in tests

* probably reuse rocksdb iterator in simulation

* clear unnecessary knob change
2024-08-16 12:37:18 -07:00
John Brownlee cd2962f10c Rename fields in new trace events to match formatting standards. 2024-08-16 11:44:25 -07:00
John Brownlee 860963ba05 Reformat changes. 2024-08-15 12:31:17 -07:00
John Brownlee cd4eb794b5 Add debug logging to help validate the transaction options set in fdbbackup. 2024-08-15 12:01:56 -07:00
Jingyu Zhou bd2e108531
Merge pull request #11555 from jzhou77/fix
Reduce chance of running rare tests
2024-08-06 10:13:03 -07:00
Syed Paymaan Raza 392bad2bd3
More copyright end year updates (#11556) 2024-08-05 14:00:32 -07:00
Jingyu Zhou 5d2deddb7d Reduce the chance to run some rare tests
E.g., StatusBuilderPerf and TLogVersionMessagesOverheadFactor are more like
performance tests, which shouldn't be running so many times.

Without the change, a 100k-run has this many for these tests:

   1318 tests/rare/CycleWithKills.toml
   1591 tests/rare/TLogVersionMessagesOverheadFactor.toml
   1647 tests/rare/ConfigDBUnitTest.toml
   1839 tests/rare/StatusBuilderPerf.toml

After the change, a 100k-run has:

    129 tests/rare/TLogVersionMessagesOverheadFactor.toml
    151 tests/rare/CycleWithKills.toml
    160 tests/rare/StatusBuilderPerf.toml
    375 tests/rare/ConfigDBUnitTest.toml
2024-08-02 17:24:30 -07:00
Syed Paymaan Raza c3e7542cda Update end year in copyright header 2024-08-02 09:40:11 -07:00
Zhe Wang a245b9622c
Fix a couple of simulation failures (#11543)
* Add usable region check per shard for encode shard location metadata

* nits

* nit

* address comments

* fix SS assertion failed for a wrong data move type generated by an old binary which does not encode the data move type in the data move id

* fix ClientTransactionProfilingCorrectness 7.3 upgrade test considering physical shard move compatibility

* code clean

* split CycleTestRestart in upgrading test from release-7.3

* address comments

* nits
2024-08-01 22:32:32 -07:00
Jingyu Zhou 995eec702b
Fix ApplyMutationsError test failure due to hot shard throttling (#11551)
In the restore test, there are hot ranges with writes to customized range with
prefix "BeforeRestart" or "AfterRestart". As a result, FDB can return
transaction_throttled_hot_shard errors and cause test failure.
2024-08-01 22:28:22 -07:00
Zhe Wang def1f0edc8
fix dd stuck due to long fetch shard (#11537) 2024-07-29 21:35:49 -07:00
Zhe Wang 81dde9216e
fix bulkload valgrind issue (#11527) 2024-07-25 00:29:33 -07:00
Giuseppe 1eaf861c3a
Documentation fix: datacenter_id does not need to be hexadecimal (#11519) 2024-07-24 12:39:53 -07:00
Xiaoge Su b324bfb9b9
When a commit is cancelled, record all unresponsive/failed TLogs (#11459)
* When a commit is cancelled, record all unresponsive/failed TLogs

* fixup!

* fixup!

* fixup!
2024-07-23 17:26:11 -07:00
Zhe Wang 74990e44bd
Bulk Loading Framework (#11369) 2024-07-23 14:57:28 -07:00
Dan Lambright 1e834f84c8
Add dynamic knob to disable gray failure recoveries. (#11509)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-07-20 14:35:21 -04:00
Jingyu Zhou a733e48048
Merge pull request #11517 from jzhou77/release-notes
Fix Wunused-but-set-variable warnings
2024-07-18 13:51:21 -07:00
Yao Xiao c630fa2296
Fix wait (#11474) 2024-07-18 11:28:34 -07:00
Jingyu Zhou d9e4c49503 Fix more -Wunused-variable warnings 2024-07-17 15:35:49 -07:00
Xiaoge Su ecdd78f65e Add extra informations when checkExclusion receives a problematic status json 2024-07-12 17:51:49 -07:00
Vishesh Yadav 6cd5ad2ffe Add code back 2024-07-10 18:52:14 -07:00
Vishesh Yadav 591efa1d1b Remove swift 2024-06-29 11:11:48 -07:00
Sreenath Bodagala 93b62f6299
- Cleanup error/trace messages logged in the context of replica comparison (#11467) 2024-06-18 17:33:19 -04:00
Sreenath Bodagala d7eb028b2a
Enable replica consistency check on data movement (#11415)
* - Enable replica consistency check on data movement (and, randomly, on
all reads)

* - Address PR review comments
2024-06-17 17:07:32 -04:00
Xiaoge Su 3e3eee98fc fixup! Reformat source 2024-06-17 11:41:06 -07:00