Commit Graph

9368 Commits

Author SHA1 Message Date
Jingyu Zhou f0d0c863e6 Attempt to fix 2020-03-28 17:38:02 -07:00
Jingyu Zhou 78f74ed6fe Skip copying inside BackupLogsDispatchTask 2020-03-28 17:38:02 -07:00
Jingyu Zhou a2b8be5dc6 Switch off old mutation logging on proxies for new backups
Specifically, stop writing LogRange mutation for new backup jobs.
2020-03-28 17:38:02 -07:00
Jingyu Zhou cf7404a771
Merge pull request #2871 from xumengpanda/mengxu/parallel-range-log-file-loading-PR
Performant restore [24/xx]: Process range and log files in parallel for old backup format
2020-03-28 17:33:50 -07:00
Meng Xu 92998d3df9 Add restore with old backup tests into CMakeLists 2020-03-28 16:08:22 -07:00
Meng Xu 13f343ec96 Resolve minor review comment 2020-03-28 16:03:01 -07:00
Meng Xu 6e60c62d7b Add ParallelRestoreCorrectnessAtomicOp.txt 2020-03-28 16:00:51 -07:00
Meng Xu 8a30526336 FastRestore:Remove commented assertion 2020-03-28 13:11:32 -07:00
Meng Xu 404a3e2619 FastRestore:Loader:Remove sanity chech for the order of sending log and range mutations 2020-03-27 23:36:13 -07:00
Meng Xu 21a5c67f9a FastRestore:Remove assertion on mutation sending order 2020-03-27 17:09:31 -07:00
Meng Xu 75fc9af5c8 Apply clang format 2020-03-27 16:55:52 -07:00
Meng Xu 0222e8096c FastRestore:Send log mutations and range mutations in parallel
With the subversion extension, appliers can order log and range mutations
based on LogMessageVersion instead of sending order.
2020-03-27 16:54:19 -07:00
Alvin Moore 137ea71203
Merge pull request #2865 from mpilman/features/build-test-harness
Build all joshua packages with cmake
2020-03-27 13:11:39 -07:00
Meng Xu f7233bade7 Rename ParallelRestoreCorrectnessAtomicOpTinyData.txt by removing TinyData 2020-03-27 13:08:59 -07:00
Meng Xu 5c6752fb24 Add tests for new restore with old backup 2020-03-27 13:07:38 -07:00
Meng Xu 97f8e46388 Sanity check subversion for log mutations 2020-03-27 13:07:08 -07:00
Meng Xu 32b0ba1822 Merge branch 'master' into mengxu/parallel-range-log-file-loading-PR 2020-03-27 12:13:47 -07:00
Meng Xu 113d0fb48b Remove incorrect assertion 2020-03-27 12:13:30 -07:00
Markus Pilman 644e0ff30b Use full version string in package names 2020-03-27 09:47:37 -07:00
Markus Pilman 0f224ead7a fixed formatting 2020-03-27 09:38:20 -07:00
Markus Pilman c5191f5c51
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:57 -07:00
Markus Pilman ce318c18fa
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:46 -07:00
Markus Pilman 893a38008c
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:38 -07:00
Markus Pilman 55e5d5b4d9
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:30 -07:00
Markus Pilman 94e653b6d8
Update contrib/Joshua/scripts/bindingTestScript.sh
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:21 -07:00
Markus Pilman def7f39328
Update cmake/AddFdbTest.cmake
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
2020-03-27 09:36:08 -07:00
Meng Xu 01d58341aa
Merge pull request #2867 from jzhou77/backup-fix
Add an option for fast restore workload to restore old backups
2020-03-26 17:01:04 -07:00
Markus Pilman ac9f79116e Build bindingtester 2020-03-26 16:01:58 -07:00
Jingyu Zhou 96a7b4b459 Disable ParallelRestoreApiCorrectnessAtomicRestore test 2020-03-26 14:59:50 -07:00
Markus Pilman 99e4bb92c0 Generate valgrind tar file for Joshua 2020-03-26 14:02:21 -07:00
Jingyu Zhou 9a9af7d8a8 Add more trace event details on partitioned log 2020-03-26 13:57:31 -07:00
Markus Pilman c095d26830 include joshua scripts 2020-03-26 13:52:18 -07:00
Markus Pilman e8c41e3f82 fixed strip_target dependencies 2020-03-26 13:45:22 -07:00
Markus Pilman 74f21c831c fixed correctness tar name 2020-03-26 13:45:12 -07:00
Meng Xu 6299ad3913 FastRestore:Load range and log files in parallel for new backup format 2020-03-26 13:17:44 -07:00
Markus Pilman 022a90dc3a Merge branch 'correctness_pkg' into features/build-test-harness 2020-03-26 13:13:38 -07:00
Jingyu Zhou cffcac92ad Disable partitioned log for atomic restore workload
This will be reenabled after fixing test failures related to it.
2020-03-26 13:07:40 -07:00
Jingyu Zhou 6be913a430 Add partitioned logs option to AtomicRestore workload 2020-03-26 13:04:00 -07:00
Jingyu Zhou aca458cd96 Set 50% chance to restore old backup files for fast restore 2020-03-26 13:04:00 -07:00
Jingyu Zhou 99f4ef6e0c Fix restore loader to handle mutation sub number
For old backup format, give them a sub sequence number starting from 0 for each
commit version.
2020-03-26 13:04:00 -07:00
Jingyu Zhou 40b17e1e9b Remove a no longer unused knob 2020-03-26 13:04:00 -07:00
Jingyu Zhou 772ab70aee Add an option for fast restore to restore old backups
If "usePartitionedLogs" is set to false, then the workload uses old backups for
restore.
2020-03-26 13:04:00 -07:00
Markus Pilman 7e2938b290 Merge branch 'master' of github.com:apple/foundationdb into features/build-test-harness 2020-03-26 12:47:55 -07:00
Meng Xu 1052b23ee1
Merge pull request #2370 from atn34/test-watch-outliving-transaction
Test watch outliving transaction
2020-03-26 12:40:38 -07:00
A.J. Beamon 66786790ce
Merge pull request #2862 from tclinken/set-log-group-with-open-trace-file
Allow trace log group to be set after database is created
2020-03-26 12:00:03 -07:00
Andrew Noyes f6af6ec777 Avoid passing self to watch actor 2020-03-26 10:29:23 -07:00
Andrew Noyes cdb6bbfc85 Test watch outliving transaction 2020-03-26 10:09:03 -07:00
tclinken baf0fe956c Take trace mutex in setLogGroup 2020-03-26 09:55:03 -07:00
Alvin Moore d80014cbc3 Added Joshua Readme and scripts for describing and implementing correctness for FoundationDB 2020-03-26 02:33:45 -07:00
Jingyu Zhou feedab02a0
Merge pull request #2855 from xumengpanda/mengxu/fr-api-atomicrestore-PR
Add ApiCorrectnessAtomicRestore workload for the new performant restore
2020-03-25 18:05:26 -07:00