13f343ec96
Resolve minor review comment
2020-03-28 16:03:01 -07:00
6e60c62d7b
Add ParallelRestoreCorrectnessAtomicOp.txt
2020-03-28 16:00:51 -07:00
8a30526336
FastRestore:Remove commented assertion
2020-03-28 13:11:32 -07:00
404a3e2619
FastRestore:Loader:Remove sanity chech for the order of sending log and range mutations
2020-03-27 23:36:13 -07:00
21a5c67f9a
FastRestore:Remove assertion on mutation sending order
2020-03-27 17:09:31 -07:00
75fc9af5c8
Apply clang format
2020-03-27 16:55:52 -07:00
0222e8096c
FastRestore:Send log mutations and range mutations in parallel
...
With the subversion extension, appliers can order log and range mutations
based on LogMessageVersion instead of sending order.
2020-03-27 16:54:19 -07:00
f7233bade7
Rename ParallelRestoreCorrectnessAtomicOpTinyData.txt by removing TinyData
2020-03-27 13:08:59 -07:00
5c6752fb24
Add tests for new restore with old backup
2020-03-27 13:07:38 -07:00
97f8e46388
Sanity check subversion for log mutations
2020-03-27 13:07:08 -07:00
32b0ba1822
Merge branch 'master' into mengxu/parallel-range-log-file-loading-PR
2020-03-27 12:13:47 -07:00
113d0fb48b
Remove incorrect assertion
2020-03-27 12:13:30 -07:00
01d58341aa
Merge pull request #2867 from jzhou77/backup-fix
...
Add an option for fast restore workload to restore old backups
2020-03-26 17:01:04 -07:00
96a7b4b459
Disable ParallelRestoreApiCorrectnessAtomicRestore test
2020-03-26 14:59:50 -07:00
9a9af7d8a8
Add more trace event details on partitioned log
2020-03-26 13:57:31 -07:00
6299ad3913
FastRestore:Load range and log files in parallel for new backup format
2020-03-26 13:17:44 -07:00
cffcac92ad
Disable partitioned log for atomic restore workload
...
This will be reenabled after fixing test failures related to it.
2020-03-26 13:07:40 -07:00
6be913a430
Add partitioned logs option to AtomicRestore workload
2020-03-26 13:04:00 -07:00
aca458cd96
Set 50% chance to restore old backup files for fast restore
2020-03-26 13:04:00 -07:00
99f4ef6e0c
Fix restore loader to handle mutation sub number
...
For old backup format, give them a sub sequence number starting from 0 for each
commit version.
2020-03-26 13:04:00 -07:00
40b17e1e9b
Remove a no longer unused knob
2020-03-26 13:04:00 -07:00
772ab70aee
Add an option for fast restore to restore old backups
...
If "usePartitionedLogs" is set to false, then the workload uses old backups for
restore.
2020-03-26 13:04:00 -07:00
1052b23ee1
Merge pull request #2370 from atn34/test-watch-outliving-transaction
...
Test watch outliving transaction
2020-03-26 12:40:38 -07:00
66786790ce
Merge pull request #2862 from tclinken/set-log-group-with-open-trace-file
...
Allow trace log group to be set after database is created
2020-03-26 12:00:03 -07:00
f6af6ec777
Avoid passing self to watch actor
2020-03-26 10:29:23 -07:00
cdb6bbfc85
Test watch outliving transaction
2020-03-26 10:09:03 -07:00
baf0fe956c
Take trace mutex in setLogGroup
2020-03-26 09:55:03 -07:00
feedab02a0
Merge pull request #2855 from xumengpanda/mengxu/fr-api-atomicrestore-PR
...
Add ApiCorrectnessAtomicRestore workload for the new performant restore
2020-03-25 18:05:26 -07:00
bb5799bd20
Merge pull request #2642 from xumengpanda/mengxu/new-backup-format-PR
...
FastRestore:Integrate with new backup format
2020-03-25 15:47:55 -07:00
0f57bf9685
Remove a SevError event
...
The same mutation can be present in overlapping mutation logs. Thus we cannot
assert its absence. This can be caused for multiple reasons. One possibility
is that new TLogs can copy mutations from old generation TLogs; another one
is backup worker is recruited without knowning previously saved progress.
2020-03-25 15:23:21 -07:00
a93f13cfd7
Remove redundant restoreRequestDone break in backup.actor
2020-03-25 15:19:46 -07:00
495afe2e0b
Improve how to wati for restore to finish
...
Remove default parameter for atomicRestore as suggested in review.
2020-03-25 13:54:21 -07:00
7d5ed53215
Allow trace log group to be set after database is created
2020-03-25 13:40:43 -07:00
f240d393f2
Add ParallelRestoreApiCorrectnessAtomicRestore.txt into ctests
2020-03-25 11:23:19 -07:00
1ba11dc74b
Apply clang format
2020-03-25 11:20:17 -07:00
0547cf2b4c
Merge pull request #2854 from dongxinEric/doc/update-api-docs-for-getEstimatedRangeBytes
...
Doc/update api docs for get estimated range bytes
2020-03-25 11:08:25 -07:00
120272f025
Change unlockDB from RestoreMaster to Agent
2020-03-25 11:04:49 -07:00
472f7bdd32
Rename a trace event to avoid confusion
...
Change from BackupRange to BackupVersionRange.
2020-03-25 11:03:05 -07:00
e0fbd9ecbe
Merge pull request #2847 from atn34/atn34/assert-no-return
...
Assert recoverAndEndEpoch does not become ready
2020-03-25 10:23:38 -07:00
353fcb2096
Update Go binding documentations for the new API to be consistent with others
2020-03-25 10:02:38 -07:00
30f3753699
Replace the Version to Int64Future
2020-03-25 09:41:47 -07:00
1501a365fb
Address review comments
2020-03-25 09:33:19 -07:00
e2f317a0da
Fix a crash failure
2020-03-25 09:18:49 -07:00
130b91c820
Fix segmentation fault in submitParallelRestore
2020-03-24 23:52:34 -07:00
00fb4c1a35
Fix an off by one error
...
Backup worker's saved version should start from its startVersion - 1, i.e.,
the startVersion is not saved yet. Otherwise, if the version range is just
the startVersion itself and there is no data, then the range [startVersion,
startVersion + 1) will be missing. This causes non-continuous partitioned logs.
2020-03-24 23:40:36 -07:00
ca8966a28b
Move lockDB into submitRestore request from restore worker
...
AtomicRestore needs to lock DB before we start the restore worker.
So we cannot lock DB in restore worker with a different randomUID.
2020-03-24 23:39:35 -07:00
e59becdec8
Refactor atomicRestore for atomicParallelRestore
2020-03-24 22:37:12 -07:00
6a8d6ddb8e
Introduce ParallelRestoreApiCorrectnessAtomicRestore.txt test
...
This covers ApiCorrectnessTest as workload for parallel restore.
2020-03-24 22:30:51 -07:00
669916467e
Add missing transaction reset call
2020-03-24 20:14:37 -07:00
5e729a5bcf
Merge branch 'master' of https://github.com/apple/foundationdb into backup-worker-bak
2020-03-24 19:54:36 -07:00
Meng Xu