Commit Graph

9370 Commits

Author SHA1 Message Date
Jingyu Zhou abcb458a44 Fix contiguousLogEnd calculation
The contiguous version should scan from the max of scanBegin version and the
minLogBegin version. Once we found a version that's larger, set it as the
contiguousLogEnd version.
2020-03-28 17:38:02 -07:00
Jingyu Zhou efbd01894d Really fix the stuck of backup 2020-03-28 17:38:02 -07:00
Jingyu Zhou f0d0c863e6 Attempt to fix 2020-03-28 17:38:02 -07:00
Jingyu Zhou 78f74ed6fe Skip copying inside BackupLogsDispatchTask 2020-03-28 17:38:02 -07:00
Jingyu Zhou a2b8be5dc6 Switch off old mutation logging on proxies for new backups
Specifically, stop writing LogRange mutation for new backup jobs.
2020-03-28 17:38:02 -07:00
Jingyu Zhou cf7404a771
Merge pull request #2871 from xumengpanda/mengxu/parallel-range-log-file-loading-PR
Performant restore [24/xx]: Process range and log files in parallel for old backup format
2020-03-28 17:33:50 -07:00
Meng Xu 92998d3df9 Add restore with old backup tests into CMakeLists 2020-03-28 16:08:22 -07:00
Meng Xu 13f343ec96 Resolve minor review comment 2020-03-28 16:03:01 -07:00
Meng Xu 6e60c62d7b Add ParallelRestoreCorrectnessAtomicOp.txt 2020-03-28 16:00:51 -07:00
Meng Xu 8a30526336 FastRestore:Remove commented assertion 2020-03-28 13:11:32 -07:00
Meng Xu 404a3e2619 FastRestore:Loader:Remove sanity chech for the order of sending log and range mutations 2020-03-27 23:36:13 -07:00
Meng Xu 21a5c67f9a FastRestore:Remove assertion on mutation sending order 2020-03-27 17:09:31 -07:00
Meng Xu 75fc9af5c8 Apply clang format 2020-03-27 16:55:52 -07:00
Meng Xu 0222e8096c FastRestore:Send log mutations and range mutations in parallel
With the subversion extension, appliers can order log and range mutations
based on LogMessageVersion instead of sending order.
2020-03-27 16:54:19 -07:00
Alvin Moore 137ea71203
Merge pull request #2865 from mpilman/features/build-test-harness
Build all joshua packages with cmake
2020-03-27 13:11:39 -07:00
Meng Xu f7233bade7 Rename ParallelRestoreCorrectnessAtomicOpTinyData.txt by removing TinyData 2020-03-27 13:08:59 -07:00
Meng Xu 5c6752fb24 Add tests for new restore with old backup 2020-03-27 13:07:38 -07:00
Meng Xu 97f8e46388 Sanity check subversion for log mutations 2020-03-27 13:07:08 -07:00
Meng Xu 32b0ba1822 Merge branch 'master' into mengxu/parallel-range-log-file-loading-PR 2020-03-27 12:13:47 -07:00
Meng Xu 113d0fb48b Remove incorrect assertion 2020-03-27 12:13:30 -07:00
Markus Pilman 644e0ff30b Use full version string in package names 2020-03-27 09:47:37 -07:00
Markus Pilman 0f224ead7a fixed formatting 2020-03-27 09:38:20 -07:00
Markus Pilman c5191f5c51
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:57 -07:00
Markus Pilman ce318c18fa
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:46 -07:00
Markus Pilman 893a38008c
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:38 -07:00
Markus Pilman 55e5d5b4d9
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:30 -07:00
Markus Pilman 94e653b6d8
Update contrib/Joshua/scripts/bindingTestScript.sh
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:21 -07:00
Markus Pilman def7f39328
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:08 -07:00
Meng Xu 01d58341aa
Merge pull request #2867 from jzhou77/backup-fix
Add an option for fast restore workload to restore old backups
2020-03-26 17:01:04 -07:00
Markus Pilman ac9f79116e Build bindingtester 2020-03-26 16:01:58 -07:00
Jingyu Zhou 96a7b4b459 Disable ParallelRestoreApiCorrectnessAtomicRestore test 2020-03-26 14:59:50 -07:00
Markus Pilman 99e4bb92c0 Generate valgrind tar file for Joshua 2020-03-26 14:02:21 -07:00
Jingyu Zhou 9a9af7d8a8 Add more trace event details on partitioned log 2020-03-26 13:57:31 -07:00
Markus Pilman c095d26830 include joshua scripts 2020-03-26 13:52:18 -07:00
Markus Pilman e8c41e3f82 fixed strip_target dependencies 2020-03-26 13:45:22 -07:00
Markus Pilman 74f21c831c fixed correctness tar name 2020-03-26 13:45:12 -07:00
Meng Xu 6299ad3913 FastRestore:Load range and log files in parallel for new backup format 2020-03-26 13:17:44 -07:00
Markus Pilman 022a90dc3a Merge branch 'correctness_pkg' into features/build-test-harness 2020-03-26 13:13:38 -07:00
Jingyu Zhou cffcac92ad Disable partitioned log for atomic restore workload
This will be reenabled after fixing test failures related to it.
2020-03-26 13:07:40 -07:00
Jingyu Zhou 6be913a430 Add partitioned logs option to AtomicRestore workload 2020-03-26 13:04:00 -07:00
Jingyu Zhou aca458cd96 Set 50% chance to restore old backup files for fast restore 2020-03-26 13:04:00 -07:00
Jingyu Zhou 99f4ef6e0c Fix restore loader to handle mutation sub number
For old backup format, give them a sub sequence number starting from 0 for each
commit version.
2020-03-26 13:04:00 -07:00
Jingyu Zhou 40b17e1e9b Remove a no longer unused knob 2020-03-26 13:04:00 -07:00
Jingyu Zhou 772ab70aee Add an option for fast restore to restore old backups
If "usePartitionedLogs" is set to false, then the workload uses old backups for
restore.
2020-03-26 13:04:00 -07:00
Markus Pilman 7e2938b290 Merge branch 'master' of github.com:apple/foundationdb into features/build-test-harness 2020-03-26 12:47:55 -07:00
Meng Xu 1052b23ee1
Merge pull request #2370 from atn34/test-watch-outliving-transaction
Test watch outliving transaction
2020-03-26 12:40:38 -07:00
A.J. Beamon 66786790ce
Merge pull request #2862 from tclinken/set-log-group-with-open-trace-file
Allow trace log group to be set after database is created
2020-03-26 12:00:03 -07:00
Andrew Noyes f6af6ec777 Avoid passing self to watch actor 2020-03-26 10:29:23 -07:00
Andrew Noyes cdb6bbfc85 Test watch outliving transaction 2020-03-26 10:09:03 -07:00
tclinken baf0fe956c Take trace mutex in setLogGroup 2020-03-26 09:55:03 -07:00