d69bd2f661
FastRestore:Loader buffer data for multiple batches
2020-01-17 17:01:06 -08:00
bfbf2164c4
FastRestore:Applier buffer data for multiple batches
2020-01-17 17:01:01 -08:00
f436ea806e
FastRestore:Resolve review comment
...
1) Sort logfiles by endVersion
2) Exit program early when restore will not succeed
3) Do not increase nextVersion unncessarily when
calculate version batches.
4) Change assert condition that ensures progress in
calculating version batches.
2020-01-13 14:08:27 -08:00
c29e380076
FastRestore:Remove prevVersion from LoadingParam
2020-01-07 14:59:17 -08:00
9df02512ab
FastRestore:Apply clang-format
2020-01-07 11:50:32 -08:00
67e913c3d5
Change LoadingParam struct and endVersion definition
...
1) Remove endVersion field because it has been included in RestoreAsset;
2) Ensure endVersion in VersionBatch and RestoreAsset is always exclusive;
3) Revise ASSERT in laoder and applier in situations when the dummy commit version
is endVersion, to avoid false positive ASSERT failure.
2020-01-07 11:48:03 -08:00
c3f8f3b445
FastRestore:Build VersionBatch less than threshold size
2020-01-07 11:46:56 -08:00
c10035ba54
FastRestore:Use isInVersionRange based on code review
2019-12-23 15:01:27 -08:00
8d6f511816
FastRestore:Resolve review comment
...
Filter out range mutations that do not overlap with the restore range.
Small changes on format.
2019-12-22 20:09:10 -08:00
61b29de3ce
FastRestore:Self code review
...
Clean up commented code;
Add sanity check.
2019-12-20 22:24:34 -08:00
ddcf3fdd80
FastRestore:Apply clang format
2019-12-20 22:00:36 -08:00
2cd1f0780a
FastRestore:Split asset to subasset for async parsing files
2019-12-20 21:44:40 -08:00
e98b2a0d1c
FastRestore:Introduce RestoreAsset
2019-12-20 18:00:10 -08:00
ffc8f76710
FastRestore:Rename StringRefReaderMX to BackupStringRefReader
2019-12-19 11:49:37 -08:00
b5d7890ce0
FastRestore:Resolve review comments
2019-12-12 07:45:30 -08:00
9670d64fbd
FastRestore:Remove commented code
2019-12-11 16:48:40 -08:00
1371db4cdc
FastRestore:Self code review and cleanup
...
1. Review memory use cases and improve:
Ensure state varialble is initialized and
change unnecessary state variable to variable.
2. Remove debug code that is no longer useful;
3. Mute verbose debug.
2019-12-11 16:37:33 -08:00
9a6dabe47e
Merge branch 'mengxu/fastrestore-code-cleanup-PR' into mengxu/fast-restore-fix-valgrind-PR
2019-12-10 20:05:35 -08:00
feb2a8c70c
FastRestore Change RestoreSendMutationVectorVersionedRequest name
...
Change RestoreSendMutationVectorVersionedRequest to
RestoreSendVersionedMutationsRequest for better naming
2019-12-10 17:23:40 -08:00
20a19978f9
FastRestore:LoadingParam cleanup
2019-12-10 17:20:44 -08:00
e8dfc1c187
Replace pop_front(size) with new empty standalone obj
2019-12-06 23:16:49 -08:00
4a66366a05
Use MutationsVec instead of VectorRef
2019-12-06 22:00:40 -08:00
39a4f2372f
Change FASTRESTORE_SAMPLING_PERCENT to 0 to 100
2019-12-04 21:26:27 -08:00
c6b36dbffb
FastRestore:Sampling:Resolve review comments
2019-12-04 17:35:11 -08:00
dd91d26dfa
FastRestore:Sampling:Add FASTRESTORE_SAMPLING_RATE knob
2019-12-04 11:46:29 -08:00
2b987d1945
FastRestore:typedef Standalone<VectorRef<MutationRef>> MutationsVec
2019-12-04 11:39:55 -08:00
9383c3f0a6
FastRestore:Sampling:Apply clang format
2019-12-03 21:27:06 -08:00
3310f67e9e
Merge branch 'mengxu/fast-restore-fix-valgrind-PR' into mengxu/fast-restore-sampling-PR
2019-12-03 16:24:40 -08:00
153b713b53
FastRestore:Add sampling on parsed mutations
2019-12-03 12:52:17 -08:00
474f0067c4
Remove unneeded state
2019-11-25 23:10:14 -08:00
a04f314b1b
Merge pull request #2383 from jzhou77/restore
...
Use sizeof() to replace constant numbers
2019-11-22 16:14:44 -08:00
037e808253
Address review comments by changing variable names
2019-11-22 13:12:04 -08:00
9927a9013f
Use sizeof() to replace constant numbers
2019-11-22 11:47:25 -08:00
78f10f15b3
FastRestore:replace insert with emplace for map and vector
...
This resolves the review suggestions.
2019-11-21 22:47:04 -08:00
343bcd104a
FastRestore:Apply Clang format
2019-11-20 21:04:18 -08:00
3f5491318d
FastRestore:Fix bug that cause nondeterminism
...
1) Use map iterator instead of pointer to maintain stability when map is inserted or deleted
2) dummySampleWorkload: clear rangeToApplier data in each sampling phase. otherwise, we can
have an increasing number of keys assigned to the applier.
2019-11-15 11:30:09 -08:00
9e36b897e6
FastRestore:Loaders must send to appliers log files data before range files
2019-11-12 21:43:12 -08:00
592f4c0fc4
FastRestore:Remove RestoreSetApplierKeyRangeVectorRequest
2019-11-12 17:59:11 -08:00
7e4c4ea98e
FastRestore:Load mutations before assign ranges to appliers
2019-11-12 17:14:17 -08:00
ae7e42face
Merge pull request #2313 from xumengpanda/mengxu/fastrestore-applyToDB-bugfix-PR
...
Performant restore [8/XX]: Fix bugs in applyToDB logic and add more tests
2019-11-12 08:50:23 -08:00
630c29d160
FastRestore:resolve review comments
...
1) wait on whenAtLeast;
2) Put BigEndian64 into the function call and the decoder to prevent
future people from making the same mistake.
2019-11-11 17:00:16 -08:00
cf2ec3418c
Merge pull request #2317 from xumengpanda/mengxu/fastrestore-extend-atomicOpTest-PR
...
AtomicOps Test: Add more detailed debug information when test fails with opType = AddValue
2019-11-11 15:03:10 -08:00
0ccded1929
AtomicOps:Resolve review comments
2019-11-05 19:27:49 -08:00
c4d1e6e1a9
Trace:Severity:Include SevNoInfo to mute trace
...
Define SevFRMutationInfo to trace mutations in restore.
2019-11-04 16:18:40 -08:00
e345c9061f
FastRestore:Refine debug messages
2019-11-04 11:47:38 -08:00
7903b47b82
FastRestore:Remove unnecessary return
2019-10-24 13:09:24 -07:00
c53f817c5e
FastRestore:Convert handleInitVersionBatchRequest to plain func
2019-10-24 13:06:50 -07:00
60d26ff5d7
FastRestore:Resolve review comments
2019-10-24 12:52:12 -07:00
bae0c907a6
FastRestore:Convert unnecessary actor function to plain function
2019-10-23 15:10:34 -07:00
ab4a375b95
FastRestore:RestoreLoader:Define SerializedMutationPartMap type
2019-10-17 10:12:38 -07:00
Meng Xu