Commit Graph

11039 Commits

Author SHA1 Message Date
Evan Tschannen f7c7677bb9
Merge pull request #3393 from Daniel-B-Smith/rocksdb-fixes
A bunch of small RocksDB fixes
2020-07-02 14:21:56 -07:00
Daniel Smith bca740342b Merge remote-tracking branch 'upstream/release-6.3' into rocksdb-fixes 2020-07-02 20:34:50 +00:00
Xin Dong 58df3ba29f
Merge pull request #3443 from dongxinEric/bugfix/use-correct-range-begin-and-end-key
Use the actual range begin and end key instead of the shard begin and…
2020-07-02 09:32:03 -07:00
Steve Atherton acddb68aba
Merge pull request #3381 from Daniel-B-Smith/rocksdb-corruption
Allow detection of storage engines by presence of directory.
2020-07-01 22:11:13 -07:00
Xin Dong ae63304392 Address Meng's comment 2020-07-01 20:23:18 -07:00
Xin Dong 3ecdaf24ce Use the actual range begin and end key instead of the shard begin and shard end key 2020-07-01 16:07:59 -07:00
Jingyu Zhou a793386fe9
Merge pull request #3427 from bchavez/master
Corrects spelling mistakes in various markdown files.
2020-06-30 21:11:54 -07:00
Jingyu Zhou 89c8764cd6
Merge pull request #3432 from xumengpanda/mengxu/fr-applier-perf-circus-PR
Fast Restore: Improvements based on circus performance test
2020-06-30 09:10:48 -07:00
Meng Xu efb61bcac0 Rename knob to FASTRESTORE_TXN_EXTRA_DELAY 2020-06-29 21:16:30 -07:00
Meng Xu a5febfbc75 Remove spurious comments 2020-06-29 10:26:01 -07:00
Meng Xu 8f5ee1f4aa Refine trace events for better perf diagnosis 2020-06-29 10:22:07 -07:00
Meng Xu 97e26d8eb0 FastRestore:Count appliedBytes 2020-06-29 10:18:18 -07:00
Meng Xu 4b35685e99 FastRestore:Adjust knob based on circus test 2020-06-29 10:13:18 -07:00
Daniel Smith ab56afbaee Merge branch 'rocksdb-corruption' into rocksdb-fixes 2020-06-29 16:30:55 +00:00
Daniel Smith 9c2937d4d0 Only check for files/directories when needed 2020-06-29 16:25:36 +00:00
bchavez ecf340da88
Corrects spelling mistakes in various markdown files. 2020-06-28 21:30:34 -07:00
Meng Xu a708e8e09a
Merge pull request #3422 from jzhou77/fdbconvert
Fix a spurious SevError and a spammy trace event
2020-06-28 17:42:49 -07:00
Meng Xu 82dfb5ce3f FastRestore:Update process metrics for restore master 2020-06-28 12:37:04 -07:00
Meng Xu bc98c84346 RestoreLoader release data early and revert Lower priority for RestoreApplierReceiveMutations actor
A quick evalution shows lowering priority for receive mutation actor does not help restore speed but hurt it.
2020-06-28 11:12:06 -07:00
Jingyu Zhou 8263c25336 Really ignore the error from uploading actor 2020-06-27 22:36:53 -07:00
Jingyu Zhou d883426c6a Fix spammy GotBackupProgress events
Only print this types of events during master recovery and don't log them for
backup workers.
2020-06-27 21:30:38 -07:00
Jingyu Zhou b8c77ead43 Fix spurious SevError from backup workers
While displaced backup workers wait for uploading to finish, it can get
connection_failed error, which caused spurious SevError of BackupFailed. Fix
by ignoring any errors from the uploading actor.
2020-06-27 21:24:22 -07:00
Meng Xu 026a735703 Test perf without gracefully clear memory 2020-06-27 16:15:56 -07:00
Meng Xu d6381de16f RestoreLoader:Clear memory per loading param 2020-06-27 15:51:19 -07:00
Meng Xu ca7beb5a26 Fix compilation 2020-06-27 15:21:06 -07:00
Meng Xu f18a0a242d FastRestore:Loader frees mutation memory immediately after send mutations 2020-06-27 15:19:08 -07:00
Meng Xu e57dba00bd FastRestore:Lower priority for RestoreApplierReceiveMutations actor 2020-06-27 15:16:38 -07:00
Meng Xu 78c45c1200 Knob for txn delay and add back FlowLock to control txn concurrency 2020-06-27 10:13:34 -07:00
Meng Xu ecd2d8b239 FastRestore:Add counters for applier and disable FlowLock on applyStagingKeysBatch 2020-06-27 00:20:54 -07:00
Jingyu Zhou 8f842a66e4
Merge pull request #3397 from xumengpanda/mengxu/fr-addPrefix-PR
Fast Restore: Support and test addPrefix and removePrefix feature
2020-06-25 14:06:39 -07:00
Meng Xu 43d55fb73f AtomicRestore:Fix typo in comments 2020-06-25 13:24:58 -07:00
Steve Atherton af6b141736
Merge pull request #3407 from sfc-gh-tclinkenbeard/fix-vectorref-memory-leaks
Fix memory leaks in VectorRef
2020-06-25 00:54:33 -07:00
Meng Xu 5860a5b4db FastRestore:Suppress or mute spammy trace events 2020-06-24 22:10:54 -07:00
Meng Xu c13ba7aeb4 FastRestore:Cosmetic code style improvement
Also mute some trace events
2020-06-24 21:25:37 -07:00
Meng Xu 94b1e17a96 FastRestore:Fix:Avoid reset txn backoff time in sanity check 2020-06-24 21:10:47 -07:00
Meng Xu d2b71e77c2 FastRestore:Fix:checkDatabaseLock tr should not reset 2020-06-24 20:21:49 -07:00
Jingyu Zhou 5cd3dad986
Merge pull request #3409 from sears/assert_strings
Print out the text (to stderr and as trace) of the failed condition o…
2020-06-24 14:58:12 -07:00
Meng Xu a0edd1d4eb FastRestore:Disable addPrefix test 2020-06-24 14:26:33 -07:00
Russell Sears 422003df34 Print out the text (to stderr and as trace) of the failed condition on assert failure 2020-06-24 14:21:48 -07:00
sfc-gh-tclinkenbeard 3f6222a04d Mark DebugEntryRef trivially destructible 2020-06-24 14:08:22 -07:00
Daniel Smith 69139e3593 Remove unneeded option 2020-06-24 20:30:18 +00:00
sfc-gh-tclinkenbeard 8de70432a6 Fix memory leaks in VectorRef
VectorRef<T> relies on T being trivially destructible, but this was not
yet enforced. By statically asserting that T is trivially destructible,
we avoid leaking memory by not calling the destructor for non-trivially
destructible types
2020-06-24 10:54:19 -07:00
Daniel Smith dd42d11f1b Revert "Revert "Re-use RocksDB iterators for scans""
This reverts commit 00102dc6f8.
2020-06-24 16:01:00 +00:00
Meng Xu b69c36f0d6 transformDatabaseContents:cosmetic change 2020-06-23 19:27:58 -07:00
Meng Xu 198696bc1e Move transformRestoredDatabase from server to client
AtomicRestore workload turns out to rely on the FileBackupAgent
client. Keeping transformRestoredDatabase in server makes linking harder.
2020-06-23 15:48:43 -07:00
Evan Tschannen 0000d81595
Merge pull request #3377 from yliucode/fix-recent-requests
Fix GetReadVersionReply.recentRequests to reflect a proxy's number of recent requests in the commit stats.
2020-06-23 15:05:57 -07:00
Meng Xu 6eb8dc58b6 FastRestore:addPrefixTest:Fix calling transformRestoredDatabase 2020-06-23 15:02:38 -07:00
Meng Xu 84a640e4ee AtomicRestore:Add addPrefix test for fast restore 2020-06-23 14:52:27 -07:00
Meng Xu 4e27fd34e5 Refactor transformDatabaseContents into RestoreCommon
Prepare to enable addPrefix for atomicRestore
2020-06-23 14:33:13 -07:00
Meng Xu f6954ef038 FastRestore:Test both addPrefix and not add it 2020-06-23 13:09:02 -07:00