Commit Graph

5655 Commits

Author SHA1 Message Date
Jon Fu 6ed330572e Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version 2020-09-24 12:09:17 -04:00
Xin Dong 34d1d04904 Commit pipelining attempt #2 2020-09-23 20:03:04 -07:00
Xin Dong de5b0abb92
Merge pull request #3806 from xumengpanda/mengxu/fix-typo-PR
Fast Restore: Fix a typo in FastRestoreApplerPhaseApplyTxnStart event name
2020-09-23 17:11:59 -07:00
Meng Xu 5214becaa8 FR:Fix typo for event FastRestoreApplerPhaseApplyTxnDone 2020-09-23 16:43:35 -07:00
Xin Dong feb3bda79e
Merge pull request #3797 from xumengpanda/mengxu/fr-write-traffic-control-PR
Fast Restore: Add write rate control
2020-09-23 15:50:08 -07:00
Meng Xu 262307d557 FR:Change applierRemainMB map to unordered_map 2020-09-23 15:39:01 -07:00
Meng Xu aa683c0d26 FRApplier:Fix applyingDataBytes accounting at exception
When exception is thrown out after txnSize is calculated but before
it is accounted into applyingDataBytes, we will decrease applyingDataBytes in the
error handling block incorrectly.
2020-09-23 15:19:02 -07:00
sfc-gh-tclinkenbeard d13a9d3c91 Make EBackupState an enum class 2020-09-23 15:14:09 -07:00
Xin Dong 77048c3d0f Handle possbile timeout when getting a read version. Updated documentation of the status json format 2020-09-23 13:50:59 -07:00
Jon Fu 7553daba20 change waitVersion to waitForBackup 2020-09-23 16:13:30 -04:00
Jon Fu 69580593dd Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version 2020-09-23 15:35:05 -04:00
Xiaoge Su 581427c880 fixup! Remove "\n" and extra spaces in printUsage 2020-09-23 12:03:09 -07:00
Jon Fu 89c334eb53 Merge branch 'master' of https://github.com/apple/foundationdb into jfu-ibackup-test-timeout 2020-09-23 14:30:46 -04:00
Jon Fu 285e2594ef adjusted delay and timing of version gate 2020-09-23 14:28:47 -04:00
Xiaoge Su e2ea6bbdd0 fixup! Use currLine.size() for the first currLength in printOptionUsage 2020-09-23 11:19:02 -07:00
Xin Dong 50f681cd32
Apply suggestions from code review
Co-authored-by: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-09-23 10:54:49 -07:00
Meng Xu a4aad591fd FR:Remove applyStagingKeysBatchLock flow lock 2020-09-23 08:47:37 -07:00
Evan Tschannen 850ec6aa43 fixed compile error 2020-09-22 20:27:10 -07:00
Evan Tschannen 814df222b9 testRywLifetime fails below API version 630 2020-09-22 20:18:15 -07:00
Meng Xu 6da7e56e03 FastRestore:Add more trace events 2020-09-22 19:57:19 -07:00
Meng Xu 08e243a07e Fix assert failure at addCallbackAndDelFutureRef 2020-09-22 15:41:07 -07:00
Meng Xu c16d76745d FastRestore:small fix compilation error 2020-09-22 13:36:26 -07:00
Meng Xu f8c5d85b79 FastRestore:Catch controller error and fix unhandled error occurred in updateHeartbeatTime 2020-09-22 13:32:26 -07:00
Meng Xu 17ece3d477 FastRestore:Fix FastRestoreApplierTransactionRateControl events 2020-09-22 13:25:43 -07:00
Meng Xu b4254473d7 FastRestore:Add transaction rate info tracer 2020-09-22 09:10:11 -07:00
Meng Xu 002b1bec4c FastRestore:Control write traffic at each applier
Controller assigns each applier a write rate.
Applier keeps the write-rate worth of transactions outstanding to DB.

This is to avoid heavily overloading DB while still keep enough
traffic to DB to get a good write throughput.
2020-09-22 08:14:28 -07:00
Xiaoge Su 2bf4ebbd51 fixup! Add missing comma 2020-09-21 20:43:07 -07:00
Xiaoge Su 3c505786ca Refactor fdbserver.actor.cpp 2020-09-21 20:19:47 -07:00
Jon Fu 7780c24cbe Attempt to use version instead of test duration to have backup capture all changes 2020-09-21 14:45:29 -04:00
Jon Fu 9b7d75b6dd WIP 2020-09-21 13:41:35 -04:00
sfc-gh-tclinkenbeard 0814841827 Replace NULL with nullptr in fdbserver 2020-09-20 11:31:49 -07:00
Andrew Noyes a52ef0acb8
Merge pull request #3677 from sfc-gh-clin/add-setclass-to-special-key-space
Add setclass to special keys
2020-09-19 14:55:03 -07:00
sfc-gh-tclinkenbeard dbeb8772f0 Fix isMetadataMutation 2020-09-19 09:40:40 -07:00
Chaoguang Lin 4d36ed7ba3 Rename traces and add comments when getting empty worker list 2020-09-18 19:28:32 -07:00
Chaoguang Lin 443385b052 Skip the case where the test client gets zero workers in setclass test 2020-09-18 18:20:47 -07:00
Evan Tschannen 12b07d024f
Merge pull request #3775 from sfc-gh-xwang/fdbtest
add knob MIN_TAG_WRITE_PAGES_RATE for write tag-throttling
2020-09-18 10:40:28 -07:00
Young Liu 6032a21d30 Fix comment 2020-09-17 18:02:23 -07:00
Young Liu 60ccaa2ae9 Support finding max restorable version for key ranges when keyspace's max restorable version is unavailable 2020-09-17 17:58:58 -07:00
Young Liu d4e3e98957 Use new getRestoreSet in RestoreController 2020-09-17 17:56:36 -07:00
Young Liu 71309a79f9 Merge branch 'release-6.3' into range 2020-09-17 17:54:33 -07:00
Meng Xu cf69f455a9
Merge pull request #3785 from apple/release-6.3
Merge Release 6.3 to master
2020-09-17 14:43:56 -07:00
Jon Fu 3746d52fb8 apply symmetrical clear to txnStateStore for writeRecoveryKey 2020-09-17 15:38:09 -04:00
Xin Dong 4df0f60729 Instead of using fully_recovered, use accepting_commits as a singal of DB turned available. Also add the number of old generations into status 2020-09-17 09:55:25 -07:00
A.J. Beamon 0c98a145b9
Fix whitespace 2020-09-17 08:09:21 -07:00
A.J. Beamon 303e1dc0c6 Fix fall through in switch statement 2020-09-17 08:07:53 -07:00
Jingyu Zhou ea27e4bd9c
Merge pull request #3772 from yliucode/rename
Rename master proxy as commit proxy
2020-09-16 20:06:50 -07:00
Evan Tschannen 1fe31aa9e8
Merge pull request #3706 from yliucode/tlogs
Report missing old tlogs in recovery before storage servers are fully recovered
2020-09-16 14:07:06 -07:00
Young Liu cc5bc16bd8 Rename more places from proxy to commit proxy 2020-09-15 22:29:49 -07:00
Trevor Clinkenbeard 3a68c89281
Remove initialisms
Co-authored-by: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-09-15 16:32:24 -07:00
Jon Fu 37d77ecb64 WIP of adding tests 2020-09-15 13:31:06 -04:00
Meng Xu ed2d261266 Add MinV2 and AndV2 into AtomicOps test workload 2020-09-15 10:01:36 -07:00
Meng Xu 8224e17a08 Integrate extended getRestoreSet API into fast restore
The extended getRestoreSet provides a much smaller set of backup files for small keyrange restore;
This commit integrate it into fast restore so that fast restore does not have to filter out unneeded backup files.
2020-09-15 09:38:43 -07:00
Jingyu Zhou 8afd041184
Merge pull request #3776 from sfc-gh-tclinkenbeard/update-checkunreadability
Remove unnecessary parameter from checkUnreadability
2020-09-14 13:55:30 -07:00
Trevor Clinkenbeard 94e7e11664
Merge pull request #3779 from ajbeamon/fix-write-tag-test-memory-error
Fix memory issue in write tag throttling test
2020-09-14 10:59:57 -07:00
Meng Xu a3c861b59d
Merge pull request #3777 from apple/release-6.3
Merge Release 6.3 into master
2020-09-14 10:32:50 -07:00
A.J. Beamon 268eb43cc2 Fix memory issue in test 2020-09-14 10:29:40 -07:00
sfc-gh-tclinkenbeard 09b3b7a325 Remove unnecessary parameter from checkUnreadability 2020-09-13 21:32:47 -07:00
Xin Dong 3c7bd3549a Fix compile errors 2020-09-11 14:23:27 -07:00
XiaoxiWang 52bd86ad42 update knob 2020-09-11 20:50:19 +00:00
Xin Dong f2f3351560 Only report if the field FullyRecoveredAtVersion exists. 2020-09-11 13:44:17 -07:00
Xin Dong 2619e4d3df Use version clock to mitigate network clock skew. 2020-09-11 13:39:16 -07:00
Jon Fu 62c81e03c7 changed incremental backup workload to have an option to check system keys for version 2020-09-11 16:02:13 -04:00
XiaoxiWang d5fba9a69b add write-tag throttling 2020-09-11 19:10:35 +00:00
Jon Fu 260c8d9568 Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version 2020-09-11 15:05:58 -04:00
Jon Fu 22996284c7 added changes to allow writing of last epoch end version to special keys when performing recovery due to snapshot 2020-09-11 15:00:11 -04:00
Xin Dong 224f23b0f8 Rely on MasterRecoveryState message since we only care about the current generation. 2020-09-11 11:45:02 -07:00
XiaoxiWang 195c55df4b add comment 2020-09-11 16:48:21 +00:00
XiaoxiWang 9947b416ee Merge branch 'master' of https://github.com/apple/foundationdb into fdbtest 2020-09-11 15:27:35 +00:00
XiaoxiWang 084c69b2ad fix reason serialization bug 2020-09-11 07:21:57 +00:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Trevor Clinkenbeard 732a457ba4
Merge pull request #3720 from sfc-gh-xwang/mako
update mako and WriteTagThrottling workload so as to trigger tag throttling event
2020-09-10 15:21:00 -07:00
Young Liu 1867ee1f5f Change cli output format 2020-09-09 22:34:36 -07:00
Chaoguang Lin d270b15c89 Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space 2020-09-09 16:21:19 -07:00
Chaoguang Lin 2c224de2f8 Update test of setclass 2020-09-09 16:19:55 -07:00
Meng Xu ce92f1a224 FR:Init StagingKey when created
The key field was used in various places, such as figuring out the conflict key range.
We should not leave it empty
2020-09-09 16:12:32 -07:00
Meng Xu 5c5abd7afa FastRestoreApplier:Calculate conflict range in applyStagingKeysBatch 2020-09-09 15:00:25 -07:00
Meng Xu 2febbe74ce FastRestoreApplier:Fix conflict range inverted due to invalid memory access
Rerpot error on loader and applier if not error_code_operation_cancelled error ever happens
2020-09-09 14:40:21 -07:00
A.J. Beamon 60efc5d44b Don't set SPECIAL_KEY_SPACE_ENABLE_WRITES randomly in Fuzz API test 2020-09-09 12:40:05 -07:00
Meng Xu f10e9ea679 FastRestoreApplier:Add write conflict range 2020-09-09 12:12:14 -07:00
Young Liu 1155d015c9 fetch current log generation as well 2020-09-09 11:54:58 -07:00
Meng Xu e69ee87f33 Refactor blob failure to simulateBlobFailure function 2020-09-09 10:54:21 -07:00
Meng Xu c9171e15b8 Change FastRestoreApplierPrecomputeResultSkipUnexpectedBackupMutation to SevError 2020-09-09 10:42:51 -07:00
sfc-gh-tclinkenbeard 2ce81a3c44 Make WATCH_OVERHEAD_BYTES constexpr 2020-09-09 10:35:42 -07:00
Meng Xu 5880bd9fa7 BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error 2020-09-09 08:47:43 -07:00
Meng Xu 83d1350d8d FastRestore:Handle retriable blob error 2020-09-09 07:29:17 -07:00
Meng Xu 6d58064964 Buggify _parsePartitionedLogFileOnLoader with blob http error 2020-09-09 06:46:08 -07:00
Meng Xu cffb54ba0b Buggify decodeLogFileBlock with blob http error 2020-09-09 06:44:02 -07:00
Meng Xu f3986d201d Buggify backup decodeRangeFileBlock with blob connection error 2020-09-09 06:40:41 -07:00
Meng Xu 9eeed8144e FastRestore:Fix bug in precomputeResult 2020-09-08 16:37:12 -07:00
XiaoxiWang 916c4ef628 Merge branch 'master' of https://github.com/apple/foundationdb into fdbtest 2020-09-08 23:29:39 +00:00
Trevor Clinkenbeard 62dd1f7234
Merge pull request #3696 from sfc-gh-xwang/tag-report
report busiest write tag of each storage server
2020-09-08 15:21:14 -07:00
sfc-gh-tclinkenbeard 90984467c8 Add comments for WATCH_OVERHEAD_BYTES 2020-09-08 15:19:31 -07:00
Jingyu Zhou a60342a5ba
Merge pull request #3757 from yliucode/grv-metrics
Fix status fetcher for GrvProxyStats
2020-09-08 15:15:50 -07:00
XiaoxiWang 2935d3d4f6 change workload; solve some comments 2020-09-08 21:47:49 +00:00
Xin Dong 4363dd0f25 This resolves issue #3739 by exposing time since last full recovery. 2020-09-08 14:26:01 -07:00
Young Liu 23e1ff694c Report missing old tlogs in recovery between accepting commits and storage recovered 2020-09-08 13:35:42 -07:00
Trevor Clinkenbeard 223c419f35
Merge pull request #3676 from sfc-gh-jfu/jfu-incremental-backup-only
incremental backup and restore
2020-09-08 12:06:25 -07:00
Chaoguang Lin cda9e93fe0 Clang-format 2020-09-08 11:37:27 -07:00
Young Liu 6c3d919295 Fix status fetcher for GrvProxyStats 2020-09-08 11:11:45 -07:00
Chaoguang Lin 43e3e320e3 Refactor getrange for read-write module and add a test to make sure we have consistent results 2020-09-08 11:08:48 -07:00
A.J. Beamon a188f91fe6
Merge pull request #3738 from dongxinEric/misc/remove-unused-client-knob
The actual BYTE_LIMIT_UNLIMITED and ROW_LIMIT_UNLIMITED are defined i…
2020-09-08 10:46:27 -07:00
Jon Fu 0eb7092faa Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-09-08 12:53:29 -04:00
XiaoxiWang c3e153b5d9 Merge branch 'master' of https://github.com/apple/foundationdb into tag-report 2020-09-08 15:35:22 +00:00
Meng Xu d9bfdd0196
Merge pull request #3755 from apple/release-6.3
Merge Release 6.3 into master
2020-09-07 18:05:46 -07:00
Xin Dong 1ee40848df
Merge pull request #3753 from xumengpanda/mengxu/fr-fix-trace-PR
FastRestore:Fix trace event overflow in traceRoleVersionBatchProgress
2020-09-07 00:08:57 -07:00
Xin Dong b9e43bca5d
Merge pull request #3754 from xumengpanda/mengxu/fr-valgrind-asset-PR
FastRestore: Fix valgrind error
2020-09-07 00:06:06 -07:00
Meng Xu fc01b7558c FastRestore:fix uninitialized variable in param.asset 2020-09-06 22:40:57 -07:00
Meng Xu 684048f449 FastRestore:Fix trace event overflow in traceRoleVersionBatchProgress 2020-09-06 21:05:16 -07:00
Young Liu 1ad5e17458 add support for comparing original and current impls 2020-09-05 11:14:59 -07:00
Lukas Joswiak efde86340a Add knob to disable span serialization 2020-09-04 17:37:34 -07:00
Lukas Joswiak 783e6a170e Add code coverage 2020-09-04 17:36:56 -07:00
Lukas Joswiak 7dc55fdffd Revert state 2020-09-04 15:36:47 -07:00
Lukas Joswiak b96dbc45cb Update formatting 2020-09-04 15:36:47 -07:00
Lukas Joswiak 00d3aa3acc Update formatting 2020-09-04 15:36:47 -07:00
Lukas Joswiak 53b7721d6c Add additional trace information 2020-09-04 15:36:47 -07:00
Lukas Joswiak 9398025f6a Remove test 2020-09-04 15:36:47 -07:00
Lukas Joswiak f896c68996 Cleanup 2020-09-04 15:36:47 -07:00
Lukas Joswiak 2a58e775d2 Add original changes 2020-09-04 15:36:47 -07:00
Lukas Joswiak 1ca7fe1a05 Add span metadata message 2020-09-04 15:36:47 -07:00
XiaoxiWang a0204caccc Merge remote-tracking branch 'myrepo/mako' into fdbtest 2020-09-04 19:07:52 +00:00
XiaoxiWang 7a8cedd42b format 2020-09-04 18:58:46 +00:00
XiaoxiWang ecf2c0109c more concise status json 2020-09-04 18:40:45 +00:00
XiaoxiWang 5b5087c566 format 2020-09-04 16:34:05 +00:00
XiaoxiWang 7660fb3beb report busiest tags in status json 2020-09-04 16:33:59 +00:00
XiaoxiWang f94a78ff03 update TagThrottleApiWorkload 2020-09-04 16:31:10 +00:00
Xiaoxi Wang ec56e1f286 merge with master 2020-09-04 16:11:42 +00:00
Xiaoxi Wang 341feb2b6f fix compare bug 2020-09-04 07:17:35 +00:00
Evan Tschannen ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
Xin Dong 4e511295a6 Use the correct ROW_LIMIT_UNLIMITED value 2020-09-03 15:08:41 -07:00
Young Liu 42298647e3 change ProxyStats to GrvProxyStats 2020-09-02 17:09:09 -07:00
Young Liu 87693cae81 merge master branch and resolve conflicts 2020-09-02 13:44:33 -07:00
A.J. Beamon b4c96cadc7 Merge branch 'release-6.3' into merge-release-6.3-into-master 2020-09-02 12:45:57 -07:00
Jon Fu d334b6484e attempt to write to system keys with snapshot 2020-09-02 15:17:54 -04:00
Jon Fu 0c508660dc Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-09-02 13:45:44 -04:00
Jon Fu 8ac3b21459 changed occurrences of getOrThrow to get 2020-09-02 13:45:15 -04:00
Xin Dong c6477bbc61
Merge branch 'master' into feature/range-split-points-based-on-size 2020-09-02 09:41:17 -07:00
Markus Pilman 834426e848
Merge pull request #3725 from sfc-gh-clin/add-latency-for-local-operations-to-mako-workload
Log latency for set, clear, clearrange in MakoWorkload
2020-09-01 18:29:10 -06:00
Chaoguang Lin 5078bc5728 clang-format MakoWorkload 2020-09-01 17:28:32 -07:00
Chaoguang Lin 83b09784c7 Change now() to timer() 2020-09-01 17:20:53 -07:00
Chaoguang Lin 34813f9e63 Change ms to us 2020-09-01 16:33:06 -07:00
Chaoguang Lin ce7a35a4be Log latency for set, clear, clearrange 2020-09-01 16:14:43 -07:00
Evan Tschannen 0443ea7a9b fix: prioritize marking a region as fully replicated over removing machine teams 2020-09-01 15:55:33 -07:00
A.J. Beamon de3dde5f51
Merge pull request #3716 from ajbeamon/fix-memory-error
Remove a line that logs whether confChange was non-null
2020-09-01 13:20:49 -07:00
Jon Fu b9baa996e3 Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-09-01 14:38:02 -04:00
Evan Tschannen cf19c666f0
Merge pull request #3718 from etschannen/master
Merge release 6.3 into master
2020-09-01 09:00:34 -07:00
Trevor Clinkenbeard 972b776a4d
Merge pull request #3719 from sfc-gh-xwang/master
fix rate calculation typo
2020-09-01 00:34:51 -07:00
Xiaoxi Wang 47893c847b fix rate calculation typo 2020-09-01 07:19:46 +00:00
Evan Tschannen 4001f9052e remove vcxproj files 2020-08-31 23:27:56 -07:00
Evan Tschannen 4e0dcd8a50 removed orphaned request tracking in ProxyCommitData 2020-08-31 21:25:54 -07:00
Evan Tschannen 12edadd059 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	fdbclient/Knobs.cpp
#	fdbclient/MasterProxyInterface.h
#	fdbrpc/simulator.h
#	fdbserver/MasterProxyServer.actor.cpp
#	tests/fast/CycleAndLock.txt
#	tests/fast/TxnStateStoreCycleTest.txt
#	tests/fast/VersionStamp.txt
#	tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
#	tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
#	versions.target
2020-08-31 19:33:34 -07:00
A.J. Beamon fb477399f8 Remove a line that logs whether confChange was non-null. This was previously a pointer, and the logging indicated whether the caller expected a result. It is now a reference type, and we can't determine what the caller expects. 2020-08-31 16:25:39 -07:00
Xin Dong 974c9248f8 The workload sometimes would wrongly report a passed tests as failed due to not return after the hotkey is found. 2020-08-31 14:09:01 -07:00
Xiaoxi Wang 944c78ac54 update WriteTagThrottling workload 2020-08-31 18:36:10 +00:00
Evan Tschannen 4c6fac058b
Merge pull request #3709 from etschannen/release-6.3
Disabled two features using knobs for performance reasons
2020-08-31 10:39:59 -07:00
Evan Tschannen 5c97461d18
Merge pull request #3653 from etschannen/feature-proxy-busy-loadbalance
Changed proxy load balancing to balance on CPU usage
2020-08-31 10:39:08 -07:00
Evan Tschannen bbb178c430
Merge pull request #3701 from satherton/fix-redwood-correctness-timeout
Prevent Redwood unit test timeout from trying to do too much work
2020-08-31 10:38:20 -07:00
Evan Tschannen d42a6b6ea7 remove spammy trace event 2020-08-31 10:37:00 -07:00
Young Liu 8994719e46 Merge branch 'master' into grv-proxy 2020-08-31 10:21:32 -07:00
Evan Tschannen fa306e07fd temporarily disable hot read detection on the storage servers because it is causing a 3% to 5% performance regression in read heavy workloads 2020-08-31 09:30:51 -07:00
Evan Tschannen 53def68946 fix compiler warning 2020-08-31 01:23:47 -07:00
Evan Tschannen 29eec30183 Merge branch 'release-6.2' into release-6.3
# Conflicts:
#	CMakeLists.txt
#	build/Dockerfile
#	build/Dockerfile.devel
#	documentation/sphinx/source/downloads.rst
#	fdbserver/Knobs.cpp
#	fdbserver/LogSystem.h
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/TagPartitionedLogSystem.actor.cpp
#	fdbserver/WaitFailure.actor.cpp
#	fdbserver/fdbserver.vcxproj
#	fdbserver/fdbserver.vcxproj.filters
#	packaging/msi/FDBInstaller.wxs
2020-08-31 01:10:29 -07:00
Evan Tschannen d750ed303a fixed whitespace 2020-08-30 22:34:03 -07:00
Evan Tschannen 331a49a62a do not allow a proxy to reset a connection with the logs immediately upon starting up 2020-08-30 18:50:19 -07:00
Steve Atherton 89ae0aaf39 In Redwood unit test, if the verify actor falls far behind the latest commit version it will unnecessarily verify the latest version repeatedly which extends the test's run time needlessly. 2020-08-29 23:47:54 -07:00
Steve Atherton 167d3c5a8f To avoid running too long, the Redwood correctness unit test now limits the size of the KV verification map. This map only grows as the test runs, as it contains a record for every key set or clear committed during the test. 2020-08-29 16:44:38 -07:00
Young Liu e87327b33b Merge master branch and keep master proxy reporting txn cost estimation to ratekeeper 2020-08-29 12:47:35 -07:00
Jon Fu 00c77ba2b4 Added beginVersion cmd line option and addressed code review comments 2020-08-28 14:29:22 -04:00
Jon Fu 4831f11e2a Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only 2020-08-28 12:47:19 -04:00
Jon Fu 0270ee13c8 change targetversion from maxlogend to contiguouslogend. also adjusted test workload and specs 2020-08-28 12:46:34 -04:00
Evan Tschannen f6f9aea09e fix: runLoopBusyness was always zero 2020-08-28 09:29:54 -07:00
Evan Tschannen ce1139e588 added missing dumpToken trace events 2020-08-27 17:17:27 -07:00
Evan Tschannen 86cba16d7d Added logging whenever a wait failure client triggers 2020-08-27 17:07:05 -07:00
Meng Xu ca9b1f5b34 Merge branch 'release-6.3' into mengxu/fr-sched-PR
Resolve conflict at BackupContainer.actor.cpp
2020-08-27 16:54:00 -07:00
Xin Dong cd86ca6850
Merge branch 'master' into feature/range-split-points-based-on-size 2020-08-27 10:06:02 -07:00
sfc-gh-tclinkenbeard c3991262cf Add nullptr check to traceAllInfo 2020-08-27 09:40:42 -07:00
Meng Xu ca486fdcd4 FastRestoreOldBackupTest:Revert workload time to 30s 2020-08-27 09:31:57 -07:00
A.J. Beamon d36d61e0ba
Merge pull request #3571 from sfc-gh-xwang/master
Consider write/clear operation in ratekeeper
2020-08-26 18:21:27 -07:00
Meng Xu 7a29a3157f FastRestore:Fix various bugs discovered by enhanced simulation
1. sendMutation request can be dispatched when the version batch has finished and its data has been deleted;
2. Request scheduler on loader may get into infinite loop because FASTRESTORE_SCHED_LOAD_REQ_BATCHSIZE knob can be set to 0 in simulation
2020-08-26 15:53:18 -07:00
Markus Pilman 6d0610c6b4
Merge pull request #3683 from sfc-gh-ljoswiak/setup-trace-batch
Add call to set up tracing on transaction commit path
2020-08-26 13:24:37 -06:00
Evan Tschannen 74edf61745
Merge pull request #3674 from sfc-gh-anoyes/anoyes/read-conflict-dbLocked
Add read conflict on \xff/dbLocked
2020-08-26 12:18:42 -07:00
Evan Tschannen 507c67c930 Added additional information to trace events 2020-08-26 11:42:23 -07:00
Chaoguang Lin 919c78d7ed Add wait to onError 2020-08-26 11:02:22 -07:00
Evan Tschannen 28cb5f242c another fix 2020-08-26 11:01:40 -07:00
Evan Tschannen e81ccd2dc9 another compiler fix 2020-08-26 10:59:06 -07:00
Evan Tschannen e531046b53 fix compiler errors 2020-08-26 10:56:21 -07:00
Evan Tschannen fd1a4304fa fix: made ConnectionResetInfo reference counted 2020-08-26 10:53:17 -07:00
Meng Xu f1bd2a18ed Resovle review comments: No functional change 2020-08-26 10:30:31 -07:00
Meng Xu 3d2b18b663 FastRestore:AtomicRestore:Add resetDBTimeout option 2020-08-26 10:21:17 -07:00
Meng Xu 3ca1359e89 FastRestore:Simulate cpu busy and adjust simulation of overused memory usage
Increase the chance of cpu busy and memory overused
2020-08-26 10:07:49 -07:00
Meng Xu bc766ab7ef FastRestoreKnob:Adjust knobs 2020-08-26 09:54:33 -07:00
Meng Xu a8bd628216 FastRestoreAtomicRestoreTest:Increase timeout from 2100 to 36000 2020-08-26 09:48:52 -07:00
Chaoguang Lin 6c981096ec Add test for \xff\xff/configuration/process/class_source/ 2020-08-25 19:46:13 -07:00
Chaoguang Lin a07b9f234d Add readonly range \xff\xff/configuration/process/class_source, and change \xff\xff/configuration/class/ to \xff\xff/configuration/process/class_type/ 2020-08-25 18:18:32 -07:00
Chaoguang Lin 4e08aca32e Change SevInfo to SevError 2020-08-25 14:58:47 -07:00
Meng Xu d8e73fddb6 FastRestore:Cancel actors when restore request finishes 2020-08-25 14:46:26 -07:00
Xiaoxi Wang b1c206b62a change rate calculation 2020-08-25 18:47:13 +00:00
Meng Xu bd7c07436b FastRestore:Add batchIndex to RestoreAsset for better performance tracking 2020-08-25 09:34:18 -07:00
Meng Xu 23c8d0154b FastRestoreOldBackupTest:Increase test duragion
Add comment on Rollback test as well
2020-08-25 09:14:37 -07:00
Meng Xu 778daf20c0 FastRestore:Fix incorrect assert 2020-08-24 19:59:56 -07:00
Meng Xu 996ba2374c FastREstore:Fix:Incorrect condition in printing out FastRestoreLoaderSendMutationToApplierDoneTooLate 2020-08-24 17:27:40 -07:00