046260b9d7
FastRestore:Applier:Assert to ensure batchData will not be used after deleted
2020-08-17 22:42:41 -07:00
4a0315483b
FastRestore:Safeguard when request of earlier vb may be sent after the vb has finished
2020-08-17 22:20:54 -07:00
7b7490efe7
FastRestore:Debug trace for seg fault
2020-08-17 20:34:33 -07:00
97e49f2f70
Resolve throttling events
2020-08-10 22:01:12 -07:00
f071d81ad0
Report warning on FastRestoreApplierClearRangeMutationsStart if delayTime is too large
2020-08-03 14:08:31 -07:00
f36d5aa180
FR:Applier received bytes per batch
2020-07-31 17:48:55 -07:00
47c35a7a69
FastRestore:Add stats to ApplierBatchData
2020-07-31 14:59:45 -07:00
37c3bd8615
FastRestore:Ensure FASTRESTORE_NOT_WRITE_DB only work in non simulation mode
2020-07-30 20:27:54 -07:00
8cace30bb2
FastRestore:change TXN_BATCH_MAX_BYTES default to 1KB from 1MB
2020-07-30 16:43:35 -07:00
d16db8e733
FastRestore:Fix segmentation fault
2020-07-30 12:10:32 -07:00
d71361245b
FastRestore:Short cut DB for get and clearange
2020-07-30 11:17:05 -07:00
ad915e462e
Add knob FASTRESTORE_NOT_WRITE_DB to skip writting to DB
2020-07-30 10:17:17 -07:00
efb61bcac0
Rename knob to FASTRESTORE_TXN_EXTRA_DELAY
2020-06-29 21:16:30 -07:00
97e26d8eb0
FastRestore:Count appliedBytes
2020-06-29 10:18:18 -07:00
82dfb5ce3f
FastRestore:Update process metrics for restore master
2020-06-28 12:37:04 -07:00
bc98c84346
RestoreLoader release data early and revert Lower priority for RestoreApplierReceiveMutations actor
...
A quick evalution shows lowering priority for receive mutation actor does not help restore speed but hurt it.
2020-06-28 11:12:06 -07:00
ca7beb5a26
Fix compilation
2020-06-27 15:21:06 -07:00
e57dba00bd
FastRestore:Lower priority for RestoreApplierReceiveMutations actor
2020-06-27 15:16:38 -07:00
78c45c1200
Knob for txn delay and add back FlowLock to control txn concurrency
2020-06-27 10:13:34 -07:00
ecd2d8b239
FastRestore:Add counters for applier and disable FlowLock on applyStagingKeysBatch
2020-06-27 00:20:54 -07:00
5860a5b4db
FastRestore:Suppress or mute spammy trace events
2020-06-24 22:10:54 -07:00
3d6f69c8e2
FastRestore:addPrefix:Transform must clear both orignal and transformed range
...
Otherwise, anything left in the range can interfer with the result.
2020-06-21 22:18:12 -07:00
df064ac922
Merge pull request #3321 from xumengpanda/mengxu/fr-restore-ranges-PR
...
Fast Restore: Support restoring sub ranges in the framework
2020-06-09 21:01:39 -07:00
d85dc5a4d3
FastRestore:Only clear ranges that will be restored
...
Instead of clearning the entire normal key space.
This commit also removes some unnecessary tr->reset() which can invalid the txn backoff time.
2020-06-08 22:41:49 -07:00
28212d397d
RestoreApplier:Remove getValue actor
2020-06-08 20:32:52 -07:00
1edcee4e9d
RestoreApplier:Rewrite getKeys because key_not_exists error is handled by txn internally
2020-06-08 20:27:25 -07:00
5022566b35
Validate if key_not_found error ever happens
2020-06-08 16:59:00 -07:00
f00deefd5a
RestoreApplier:Remove unnecessary txn reset
2020-06-08 10:10:32 -07:00
8c81fedf11
RestoreApplier:Better handling of key not exist
2020-06-07 21:49:35 -07:00
94be3afcf8
RestoreApplier:Costmic change based on review
2020-06-06 21:17:57 -07:00
f51fca0bf3
FastRestore:Sanity check actors do not throw error silently
2020-06-05 17:44:24 -07:00
ffe949b04d
Applier:getAndComputeStagingKeys:reset txn at first error
...
When tr->onError() is ready, the txn state has been reset.
We cannot wait on the get() future from the txn because its state has been deleted.
If we do that, it will throw txn_cancelled error, which will be throw all the way
up to the RestoreApplier main loop.
The batchData->dbApplier, which is assigned by writeMutationsToDB(self->id(), req.batchIndex, batchData, cx),
will become ready but isError(). This will make all handleApplyToDBRequest throw error silently.
2020-06-05 16:40:19 -07:00
e9af22085b
Debug: getAndComputeStagingKeys may be stuck
...
Maybe wait(success(fValues[i])); never return
2020-06-04 21:26:14 -07:00
633587a95a
RestoreApplier:getAndComputeStagingKeys:retry for keys that exist in DB
...
Test shows that we cannot just skip the key that exist in DB but has
future_version error.
2020-06-03 21:17:35 -07:00
87a557dcb4
FastRestore:Applier:Treat future_version as key not exist
2020-06-03 18:30:59 -07:00
d5025a1779
getAndComputeStagingKeys: Improved handling of not exist keys
2020-06-03 15:32:36 -07:00
f5aef706f6
FastRestore:Delay leader election until restore requests are set
2020-05-12 19:11:08 -07:00
a93c23d239
Resovle review comments
2020-05-07 15:06:59 -07:00
e4bf6d570f
FastRestore:Add assertion and trace events for diagnosis
2020-05-05 19:12:15 -07:00
4d90384c58
Correct suppression event
2020-05-05 12:36:32 -07:00
c49b6756fe
FastRestoreApplier:Trace clear range op when it has too many for debug
2020-05-05 09:28:50 -07:00
759820cc61
FastRestoreApplier:Add warning when too many clears in a txn
2020-05-05 09:00:02 -07:00
62de02fb2c
FastRestoreApplier:Add delay to avoid overwelming DB
2020-05-05 08:47:26 -07:00
67b9e0b29a
FastRestoreApplier:Add sanity check and trace for debugging stall
2020-05-04 22:32:57 -07:00
d22af629cd
FastRestoreApplier:Add applierID and batchIndex for precompute stage
2020-05-04 16:32:09 -07:00
abda13e9df
FastRestoreApplier:Free memory at each VB and refactor handleApplyToDBRequest
2020-05-04 15:29:27 -07:00
135f6443da
FastRestoreApplier:Add trace to track applying status
2020-05-04 15:02:53 -07:00
0ba1551116
FastRestore:Trace memory usage periodically
2020-05-04 11:20:53 -07:00
7b5d43da9c
FastRestore:Remove unused field in RestoreRequest
2020-05-03 20:59:47 -07:00
ae86b5bb68
FastRestoreApplier:Continue when a key not exists in DB
...
Although we thought all keys cached in appliers should have
a base value in DB.
2020-05-03 20:47:21 -07:00
Meng Xu