f9f1ac6594
FastRestore:Revise TraceEvent for better diagnosis
2020-05-01 16:31:55 -07:00
134dbca0ee
FastRestore:Use cannonical way to trace error
2020-05-01 13:35:13 -07:00
41c0a1768f
FastRestore:Make FastRestore event type more descriptive
2020-05-01 10:27:08 -07:00
038f3834fc
Merge branch 'master' into mengxu/fr-code-improvement-PR
2020-05-01 09:26:29 -07:00
6bd71560f0
FastRestore:Reduce trace events in real cluster environment
2020-04-30 19:12:31 -07:00
f073049865
FastRestore:Revise trace events to be descriptive
...
Revert changes that send mutations to appliers out of order
2020-04-24 10:31:08 -07:00
d21da5065a
FastRestore:Loader:Merge MutationsVec and LogMessageVersionVec into VersionedMutationsVec
...
Remove the actor that sends one mutation message batch in the previous commit,
because that actor no longer reduces the code complexity.
2020-04-21 22:05:34 -07:00
061bcd2fb4
FastRestore:Replace typeString with safe getTypeString func
...
Also fix compilation error in previous commit
2020-04-13 15:15:54 -07:00
dbc9c23193
FastRestore:Loader:Send mutations at different versions in the same message to appliers
...
This increases the bandwidth sent from loaders to appliers.
2020-04-12 10:46:58 -07:00
55ee034e7f
Merge pull request #2916 from jzhou77/backup-fix
...
Remove version stamp ops from RestoreApplier
2020-04-11 14:04:11 -07:00
2325ab209f
FastRestore:Applier:Avoid extra copy in getAndComputeStagingKeys
2020-04-08 12:22:08 -07:00
5ebafdb94c
FastRestore:Apply clang-format to changes
2020-04-07 15:57:03 -07:00
e5b2cd81d5
FastRestore:Cleanup debug code
2020-04-07 15:56:44 -07:00
cd8215ecf2
Remove version stamp ops from RestoreApplier
...
Version stamp ops are converted into SET at the proxy, so the backup files
will never have them.
2020-04-06 22:27:47 -07:00
a51ff7aaae
FastRestore:Fix:buildVersionBatches may lose the last log file
...
If the last log file's endversion decides the last version batch's endversoin,
the buildVersionBatches function may quit early before include the last log file.
This causes some mutations missing and lead to incorrect DB.
This commit also addes an ASSERT(maxVBVersion >= targetVersion) to
alert such error as early as possible to simplify debug.
2020-04-06 12:24:26 -07:00
536e65cd76
FastRestore:Introduce debugFRMutation for debug keys
2020-04-05 15:00:36 -07:00
432c99afd0
FastRestore:Applier:Keep incompleteStagingKeys content before values are applied to DB
...
To avoid the incompleteStagingKeys is cleared before getAndComputeStagingKeys() finish using it.
2020-04-04 22:38:04 -07:00
a81ec332a9
FastRestore:Fix:Master cannot throttle on in progress version batches when it release batches out of order in simulation
2020-04-04 17:34:26 -07:00
6bce67ca75
FastRestore:Apply clang-format
2020-04-01 21:27:54 -07:00
c69c959428
FastRestore:Fix:It is legal for a backup key not exist in DB
2020-03-31 22:02:17 -07:00
25e96a13d3
FastRestore:Fix clearrange on a key mistakenly clear other keys
2020-03-31 17:45:19 -07:00
212dadc2a1
Fix bug in add mutation on applier
...
For clear range mutation, we may clear the right boundary key which should not be cleared.
2020-03-31 16:51:08 -07:00
33c4be9c42
Improve debug message for debug mutations
2020-03-31 16:00:51 -07:00
65e3b9192e
Add an assert for probably dead code
2020-03-28 21:19:47 -07:00
13f343ec96
Resolve minor review comment
2020-03-28 16:03:01 -07:00
196127fb92
Address review comments
2020-03-23 14:15:36 -07:00
fea6155714
StagingKey uses mutation instead of a vector of mutations for each log version
...
Because each log version contains commit version and subsequence number, each
key can only have one mutation for its log version. This simplifies
StagingKey::add() a lot.
2020-03-20 20:15:09 -07:00
799f0b4b0e
Small code refactor
2020-03-20 20:15:09 -07:00
ab0b59b0c3
Add subsequence number to restore loader & applier
...
The subsequence number is needed so that mutations of the same commit version
number, but from different partitioned logs can be correctly reassembled in
order.
For old backup files, the sub number is always 0. For partitioned mutation
logs, the actual sub number is used. For range files, the sub number is always
0.
2020-03-20 20:13:38 -07:00
c3b67c0c63
Fix OPEN_FOR_IDE build
2020-03-03 11:32:43 -08:00
e6457ba0d5
FastRestore:Correct type for imcompleteStagingKeys
2020-03-02 11:33:07 -08:00
2520e8d44c
FastRestore:Use more concise code as suggested in review
2020-03-01 22:32:36 -08:00
01c1a15caf
FastRestore:Applier:Limit fetch keys number in a txn in getAndComputeStagingKeys
2020-02-28 16:53:36 -08:00
fe8b8bbbff
FastRestore:Change vb state to class from enum
2020-02-27 20:15:25 -08:00
d77177367c
FastRestore:Track each ongoing version batch progress state for applier and loader roles
2020-02-27 19:47:22 -08:00
fbb6e8f39d
FastRestore:Create low memory situation in simulation on purpose
2020-02-26 14:54:38 -08:00
06495b90ae
FastRestore:Loader:Use isSchedulable to guard OOM
...
And trigger delayed actors that are blocked on memory to recheck memory.
2020-02-26 14:35:05 -08:00
a354f6ffa2
FastRestore:Applier:Use isSchedulable to guard OOM
2020-02-26 14:12:56 -08:00
ca726fc68e
FastRestore:Introduce OOM protection
...
An actor is schedulable to run if the current worker has enough resourc, i.e.,
the worker's memory usage is below the threshold;
Exception: If the actor is working on the current version batch, we have to schedule
the actor to run to avoid dead-lock.
Future: When we release the actors that are blocked by memory usage, we should release them
in increasing order of their version batch.
2020-02-26 14:09:18 -08:00
fbf5020af9
FastRestore:Applier:Add fetchKeys counter
2020-02-26 11:37:40 -08:00
6bd4703a9f
FastRestore:Resolve review comments
2020-02-20 14:27:34 -08:00
d5d26f589f
FastRestore:Cosmetic change to improve code readability
2020-02-19 15:43:51 -08:00
7897b1658f
FastRestore:Applier:Rename new apply actor names
2020-02-19 15:29:32 -08:00
e4258d73f5
FastRestore:Applier:Remove applying actors that do not have good perf
2020-02-19 15:27:59 -08:00
fe75a4cafb
FastRestore:Apply clang-format
2020-02-19 15:22:52 -08:00
03f699f2f9
Merge branch 'master' into mengxu/fast-restore-applier-multi-applying-PR
2020-02-19 15:22:33 -08:00
94d799552e
FastRestore:Apply clang-format against master
2020-02-18 16:41:59 -08:00
132f5aa9ba
FastRestore:Improve trace name and cosmetic change
2020-02-18 16:41:19 -08:00
31a6ec34b7
Merge branch 'master' into mengxu/fast-restore-agent-PR
2020-02-18 16:17:59 -08:00
c603b20e7e
FastRestore:Resolve review comments
2020-02-18 14:08:27 -08:00
Meng Xu