6be913a430
Add partitioned logs option to AtomicRestore workload
2020-03-26 13:04:00 -07:00
aca458cd96
Set 50% chance to restore old backup files for fast restore
2020-03-26 13:04:00 -07:00
99f4ef6e0c
Fix restore loader to handle mutation sub number
...
For old backup format, give them a sub sequence number starting from 0 for each
commit version.
2020-03-26 13:04:00 -07:00
40b17e1e9b
Remove a no longer unused knob
2020-03-26 13:04:00 -07:00
772ab70aee
Add an option for fast restore to restore old backups
...
If "usePartitionedLogs" is set to false, then the workload uses old backups for
restore.
2020-03-26 13:04:00 -07:00
1052b23ee1
Merge pull request #2370 from atn34/test-watch-outliving-transaction
...
Test watch outliving transaction
2020-03-26 12:40:38 -07:00
66786790ce
Merge pull request #2862 from tclinken/set-log-group-with-open-trace-file
...
Allow trace log group to be set after database is created
2020-03-26 12:00:03 -07:00
f6af6ec777
Avoid passing self to watch actor
2020-03-26 10:29:23 -07:00
cdb6bbfc85
Test watch outliving transaction
2020-03-26 10:09:03 -07:00
baf0fe956c
Take trace mutex in setLogGroup
2020-03-26 09:55:03 -07:00
feedab02a0
Merge pull request #2855 from xumengpanda/mengxu/fr-api-atomicrestore-PR
...
Add ApiCorrectnessAtomicRestore workload for the new performant restore
2020-03-25 18:05:26 -07:00
bb5799bd20
Merge pull request #2642 from xumengpanda/mengxu/new-backup-format-PR
...
FastRestore:Integrate with new backup format
2020-03-25 15:47:55 -07:00
0f57bf9685
Remove a SevError event
...
The same mutation can be present in overlapping mutation logs. Thus we cannot
assert its absence. This can be caused for multiple reasons. One possibility
is that new TLogs can copy mutations from old generation TLogs; another one
is backup worker is recruited without knowning previously saved progress.
2020-03-25 15:23:21 -07:00
a93f13cfd7
Remove redundant restoreRequestDone break in backup.actor
2020-03-25 15:19:46 -07:00
495afe2e0b
Improve how to wati for restore to finish
...
Remove default parameter for atomicRestore as suggested in review.
2020-03-25 13:54:21 -07:00
7d5ed53215
Allow trace log group to be set after database is created
2020-03-25 13:40:43 -07:00
f240d393f2
Add ParallelRestoreApiCorrectnessAtomicRestore.txt into ctests
2020-03-25 11:23:19 -07:00
1ba11dc74b
Apply clang format
2020-03-25 11:20:17 -07:00
0547cf2b4c
Merge pull request #2854 from dongxinEric/doc/update-api-docs-for-getEstimatedRangeBytes
...
Doc/update api docs for get estimated range bytes
2020-03-25 11:08:25 -07:00
120272f025
Change unlockDB from RestoreMaster to Agent
2020-03-25 11:04:49 -07:00
472f7bdd32
Rename a trace event to avoid confusion
...
Change from BackupRange to BackupVersionRange.
2020-03-25 11:03:05 -07:00
e0fbd9ecbe
Merge pull request #2847 from atn34/atn34/assert-no-return
...
Assert recoverAndEndEpoch does not become ready
2020-03-25 10:23:38 -07:00
353fcb2096
Update Go binding documentations for the new API to be consistent with others
2020-03-25 10:02:38 -07:00
30f3753699
Replace the Version to Int64Future
2020-03-25 09:41:47 -07:00
1501a365fb
Address review comments
2020-03-25 09:33:19 -07:00
e2f317a0da
Fix a crash failure
2020-03-25 09:18:49 -07:00
130b91c820
Fix segmentation fault in submitParallelRestore
2020-03-24 23:52:34 -07:00
00fb4c1a35
Fix an off by one error
...
Backup worker's saved version should start from its startVersion - 1, i.e.,
the startVersion is not saved yet. Otherwise, if the version range is just
the startVersion itself and there is no data, then the range [startVersion,
startVersion + 1) will be missing. This causes non-continuous partitioned logs.
2020-03-24 23:40:36 -07:00
ca8966a28b
Move lockDB into submitRestore request from restore worker
...
AtomicRestore needs to lock DB before we start the restore worker.
So we cannot lock DB in restore worker with a different randomUID.
2020-03-24 23:39:35 -07:00
e59becdec8
Refactor atomicRestore for atomicParallelRestore
2020-03-24 22:37:12 -07:00
6a8d6ddb8e
Introduce ParallelRestoreApiCorrectnessAtomicRestore.txt test
...
This covers ApiCorrectnessTest as workload for parallel restore.
2020-03-24 22:30:51 -07:00
669916467e
Add missing transaction reset call
2020-03-24 20:14:37 -07:00
5e729a5bcf
Merge branch 'master' of https://github.com/apple/foundationdb into backup-worker-bak
2020-03-24 19:54:36 -07:00
edcbeb8992
Address review comments
...
Move transaction object outside of the loop and rename trace events.
2020-03-24 18:22:20 -07:00
d185359422
ApiCorrectnessAtomicRestore test: Increase timeout value
2020-03-24 17:36:17 -07:00
241c2703c8
Fix atomicParallelRestore interface
2020-03-24 17:00:55 -07:00
233a6bb2a7
Address review comments
2020-03-24 16:31:46 -07:00
80d62f3cb8
Fix:Add atomicParallelRestore to header
2020-03-24 16:28:08 -07:00
ffb0a439ea
atomicParallelRestore should not return a version
2020-03-24 16:05:09 -07:00
b173929316
Add atomicParallelRestore to AtomicRestore workload
2020-03-24 15:58:49 -07:00
01921c3953
Move FileBackupAgent parallel restore impl to FileBackupAgentImpl
...
For consistency with existing code, and
state variable can only occur in ACTOR.
2020-03-24 15:11:34 -07:00
81f7181c9e
Refactor submitParallelRestore function into FileBackupAgent
2020-03-24 14:44:55 -07:00
5584884c12
Refactor parallelRestoreFinish function into FileBackupAgent
2020-03-24 14:15:15 -07:00
4e477f5489
Copy atomicRestore to atomicParallelRestore
2020-03-24 13:59:35 -07:00
91037114e5
Fix documentation build
2020-03-24 13:35:07 -07:00
0fb728d019
Updated API documentations for the new getEstimatedRangeBytes api
2020-03-24 13:27:20 -07:00
5876d694cb
Use snake case to be consistent with conventions
2020-03-24 13:26:59 -07:00
7831bec2b0
Address review comments on trace events
2020-03-24 10:54:12 -07:00
241b24b358
Merge pull request #2852 from dongxinEric/feature/binding-tests-for-storage-server-metrics-api
...
Added a test in API tests for the new API(getEstimagedRangeSize).
2020-03-24 10:41:00 -07:00
69b572cc86
Added specs for the new tests.
2020-03-24 09:59:40 -07:00
Jingyu Zhou