a93c23d239
Resovle review comments
2020-05-07 15:06:59 -07:00
e4bf6d570f
FastRestore:Add assertion and trace events for diagnosis
2020-05-05 19:12:15 -07:00
4d90384c58
Correct suppression event
2020-05-05 12:36:32 -07:00
c49b6756fe
FastRestoreApplier:Trace clear range op when it has too many for debug
2020-05-05 09:28:50 -07:00
759820cc61
FastRestoreApplier:Add warning when too many clears in a txn
2020-05-05 09:00:02 -07:00
62de02fb2c
FastRestoreApplier:Add delay to avoid overwelming DB
2020-05-05 08:47:26 -07:00
67b9e0b29a
FastRestoreApplier:Add sanity check and trace for debugging stall
2020-05-04 22:32:57 -07:00
d22af629cd
FastRestoreApplier:Add applierID and batchIndex for precompute stage
2020-05-04 16:32:09 -07:00
abda13e9df
FastRestoreApplier:Free memory at each VB and refactor handleApplyToDBRequest
2020-05-04 15:29:27 -07:00
135f6443da
FastRestoreApplier:Add trace to track applying status
2020-05-04 15:02:53 -07:00
0ba1551116
FastRestore:Trace memory usage periodically
2020-05-04 11:20:53 -07:00
7b5d43da9c
FastRestore:Remove unused field in RestoreRequest
2020-05-03 20:59:47 -07:00
ae86b5bb68
FastRestoreApplier:Continue when a key not exists in DB
...
Although we thought all keys cached in appliers should have
a base value in DB.
2020-05-03 20:47:21 -07:00
528466e0e6
FastRestore:Fix Valgrind error InvalidSuppression
...
Trace.error() must explicitly include error_code_actor_cancelled
to handle the error.
2020-05-02 19:52:05 -07:00
f9f1ac6594
FastRestore:Revise TraceEvent for better diagnosis
2020-05-01 16:31:55 -07:00
134dbca0ee
FastRestore:Use cannonical way to trace error
2020-05-01 13:35:13 -07:00
41c0a1768f
FastRestore:Make FastRestore event type more descriptive
2020-05-01 10:27:08 -07:00
038f3834fc
Merge branch 'master' into mengxu/fr-code-improvement-PR
2020-05-01 09:26:29 -07:00
6bd71560f0
FastRestore:Reduce trace events in real cluster environment
2020-04-30 19:12:31 -07:00
f073049865
FastRestore:Revise trace events to be descriptive
...
Revert changes that send mutations to appliers out of order
2020-04-24 10:31:08 -07:00
d21da5065a
FastRestore:Loader:Merge MutationsVec and LogMessageVersionVec into VersionedMutationsVec
...
Remove the actor that sends one mutation message batch in the previous commit,
because that actor no longer reduces the code complexity.
2020-04-21 22:05:34 -07:00
061bcd2fb4
FastRestore:Replace typeString with safe getTypeString func
...
Also fix compilation error in previous commit
2020-04-13 15:15:54 -07:00
dbc9c23193
FastRestore:Loader:Send mutations at different versions in the same message to appliers
...
This increases the bandwidth sent from loaders to appliers.
2020-04-12 10:46:58 -07:00
55ee034e7f
Merge pull request #2916 from jzhou77/backup-fix
...
Remove version stamp ops from RestoreApplier
2020-04-11 14:04:11 -07:00
2325ab209f
FastRestore:Applier:Avoid extra copy in getAndComputeStagingKeys
2020-04-08 12:22:08 -07:00
5ebafdb94c
FastRestore:Apply clang-format to changes
2020-04-07 15:57:03 -07:00
e5b2cd81d5
FastRestore:Cleanup debug code
2020-04-07 15:56:44 -07:00
cd8215ecf2
Remove version stamp ops from RestoreApplier
...
Version stamp ops are converted into SET at the proxy, so the backup files
will never have them.
2020-04-06 22:27:47 -07:00
a51ff7aaae
FastRestore:Fix:buildVersionBatches may lose the last log file
...
If the last log file's endversion decides the last version batch's endversoin,
the buildVersionBatches function may quit early before include the last log file.
This causes some mutations missing and lead to incorrect DB.
This commit also addes an ASSERT(maxVBVersion >= targetVersion) to
alert such error as early as possible to simplify debug.
2020-04-06 12:24:26 -07:00
536e65cd76
FastRestore:Introduce debugFRMutation for debug keys
2020-04-05 15:00:36 -07:00
432c99afd0
FastRestore:Applier:Keep incompleteStagingKeys content before values are applied to DB
...
To avoid the incompleteStagingKeys is cleared before getAndComputeStagingKeys() finish using it.
2020-04-04 22:38:04 -07:00
a81ec332a9
FastRestore:Fix:Master cannot throttle on in progress version batches when it release batches out of order in simulation
2020-04-04 17:34:26 -07:00
6bce67ca75
FastRestore:Apply clang-format
2020-04-01 21:27:54 -07:00
c69c959428
FastRestore:Fix:It is legal for a backup key not exist in DB
2020-03-31 22:02:17 -07:00
25e96a13d3
FastRestore:Fix clearrange on a key mistakenly clear other keys
2020-03-31 17:45:19 -07:00
212dadc2a1
Fix bug in add mutation on applier
...
For clear range mutation, we may clear the right boundary key which should not be cleared.
2020-03-31 16:51:08 -07:00
33c4be9c42
Improve debug message for debug mutations
2020-03-31 16:00:51 -07:00
65e3b9192e
Add an assert for probably dead code
2020-03-28 21:19:47 -07:00
13f343ec96
Resolve minor review comment
2020-03-28 16:03:01 -07:00
196127fb92
Address review comments
2020-03-23 14:15:36 -07:00
fea6155714
StagingKey uses mutation instead of a vector of mutations for each log version
...
Because each log version contains commit version and subsequence number, each
key can only have one mutation for its log version. This simplifies
StagingKey::add() a lot.
2020-03-20 20:15:09 -07:00
799f0b4b0e
Small code refactor
2020-03-20 20:15:09 -07:00
ab0b59b0c3
Add subsequence number to restore loader & applier
...
The subsequence number is needed so that mutations of the same commit version
number, but from different partitioned logs can be correctly reassembled in
order.
For old backup files, the sub number is always 0. For partitioned mutation
logs, the actual sub number is used. For range files, the sub number is always
0.
2020-03-20 20:13:38 -07:00
c3b67c0c63
Fix OPEN_FOR_IDE build
2020-03-03 11:32:43 -08:00
e6457ba0d5
FastRestore:Correct type for imcompleteStagingKeys
2020-03-02 11:33:07 -08:00
2520e8d44c
FastRestore:Use more concise code as suggested in review
2020-03-01 22:32:36 -08:00
01c1a15caf
FastRestore:Applier:Limit fetch keys number in a txn in getAndComputeStagingKeys
2020-02-28 16:53:36 -08:00
fe8b8bbbff
FastRestore:Change vb state to class from enum
2020-02-27 20:15:25 -08:00
d77177367c
FastRestore:Track each ongoing version batch progress state for applier and loader roles
2020-02-27 19:47:22 -08:00
fbb6e8f39d
FastRestore:Create low memory situation in simulation on purpose
2020-02-26 14:54:38 -08:00
Meng Xu