364644673f
Support TLog encryption in commit proxy ( #6942 )
...
This PR add support for TLog encryption through commit proxy. The encryption is done on per-mutation basis. As CP writes mutations to TLog, it inserts encryption header alongside encrypted mutations. Storage server (and other consumers of TLog such as storage cache and backup worker) decrypts the mutations as they peek TLog.
2022-06-29 14:21:05 -07:00
1e17f32469
Move StorageMetrics unit tests into cpp file
2022-05-10 13:21:23 -07:00
0a332ee1c1
Add proxy option to backup and restore params.
2022-03-28 17:10:49 -07:00
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
8dcac2f76d
Fix typos
2022-03-13 10:02:11 -03:00
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
59503a397e
Fix duplicate trace field.
2022-01-05 11:30:21 -08:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
3922fa8cdd
Fix duplicate TraceEvent detail name.
2021-08-10 17:41:31 -07:00
eb5c8eb78d
Added Label field for keys and ranges in MutationTracking.
2021-08-10 17:32:32 -07:00
faa4154a56
MutationTracking now uses a vector of keys to track. Removed "Mutation" detail from DEBUG_MUTATION() events because they are duplicates of the fields already logged in the returned MutationTracking event, which are now renamed and combined into "Mutation". Removed more toString() calls in TraceEvent detail values.
2021-08-09 23:30:45 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
79cec09255
Apply clang-tidy's performance-inefficient-vector-operation fix
...
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.
$ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
0ef44739ea
Fix OPEN_FOR_IDE build in preparation for using clang-tidy
2021-01-26 02:04:11 +00:00
c77d9e4abe
merge conflicts
2020-12-02 21:53:19 +00:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
41843f07e6
add simulator support for different process versions and ProtocolVersion test
2020-10-12 18:19:31 +00:00
a3c861b59d
Merge pull request #3777 from apple/release-6.3
...
Merge Release 6.3 into master
2020-09-14 10:32:50 -07:00
2febbe74ce
FastRestoreApplier:Fix conflict range inverted due to invalid memory access
...
Rerpot error on loader and applier if not error_code_operation_cancelled error ever happens
2020-09-09 14:40:21 -07:00
e69ee87f33
Refactor blob failure to simulateBlobFailure function
2020-09-09 10:54:21 -07:00
5880bd9fa7
BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error
2020-09-09 08:47:43 -07:00
83d1350d8d
FastRestore:Handle retriable blob error
2020-09-09 07:29:17 -07:00
6d58064964
Buggify _parsePartitionedLogFileOnLoader with blob http error
2020-09-09 06:46:08 -07:00
f3986d201d
Buggify backup decodeRangeFileBlock with blob connection error
2020-09-09 06:40:41 -07:00
12edadd059
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# fdbclient/Knobs.cpp
# fdbclient/MasterProxyInterface.h
# fdbrpc/simulator.h
# fdbserver/MasterProxyServer.actor.cpp
# tests/fast/CycleAndLock.txt
# tests/fast/TxnStateStoreCycleTest.txt
# tests/fast/VersionStamp.txt
# tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
# tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
# versions.target
2020-08-31 19:33:34 -07:00
7a29a3157f
FastRestore:Fix various bugs discovered by enhanced simulation
...
1. sendMutation request can be dispatched when the version batch has finished and its data has been deleted;
2. Request scheduler on loader may get into infinite loop because FASTRESTORE_SCHED_LOAD_REQ_BATCHSIZE knob can be set to 0 in simulation
2020-08-26 15:53:18 -07:00
f1bd2a18ed
Resovle review comments: No functional change
2020-08-26 10:30:31 -07:00
bd7c07436b
FastRestore:Add batchIndex to RestoreAsset for better performance tracking
2020-08-25 09:34:18 -07:00
778daf20c0
FastRestore:Fix incorrect assert
2020-08-24 19:59:56 -07:00
996ba2374c
FastREstore:Fix:Incorrect condition in printing out FastRestoreLoaderSendMutationToApplierDoneTooLate
2020-08-24 17:27:40 -07:00
e94261efe5
FastRestore:LoaderScheduler:Add validation on too old requests
2020-08-24 16:32:16 -07:00
6e3e36c8fc
FastRestore:RequestScheduler:Minor code style improvement
2020-08-24 10:45:46 -07:00
d9ea14ea6c
FastRestore:fix:loader can receive reply from vb that has been processed and deleted
2020-08-19 10:39:49 -07:00
88b94a3e59
FastRestore:Fix:Delayed request can be skipped from processing
...
When a request of sending mutations to appliers is delayed and the request for future version batches have been processed,
loader will increase its counter. The previous implementation will use the counter to decide if a request is too old and
have been processed. If the counter is larger than the batch index of the request, the request will be ignored although
it may not have been processed.
This causes correctness test failure for fast restore.
2020-08-18 23:48:06 -07:00
e1d150662f
FastRestore:small fix on compile error
2020-08-18 16:15:11 -07:00
a32cf7d293
FastRestore:Loader:Add sanity check before remove batch data
2020-08-18 15:44:22 -07:00
9b2f667bbe
FastRestore:Fix uninitialized variable
2020-08-18 11:58:57 -07:00
4a0315483b
FastRestore:Safeguard when request of earlier vb may be sent after the vb has finished
2020-08-17 22:20:54 -07:00
7b7490efe7
FastRestore:Debug trace for seg fault
2020-08-17 20:34:33 -07:00
01255b7ead
FastRestoreLoader:Sched:Add trace field and misc code style improvement
2020-08-16 09:43:04 -07:00
be75fd4894
Workaround flow limitation:state variable must be initialized at the beginning of actors
2020-08-16 08:45:13 -07:00
c2dfc5a00a
Fix compile error
2020-08-16 08:13:28 -07:00
cb0e44ff42
FastRestoreLoader:Sched send mutations per loadingParam requests
2020-08-16 07:38:51 -07:00
98167a6d20
FastRestoreLoader:Fix:sched may stuck due to race condition
2020-08-16 06:29:09 -07:00
015397be74
FastRestore:LoaderScheduler:Control number of load request to release at once
2020-08-15 22:18:48 -07:00
1e571a5a1a
FastRestore:Loader:Kick off scheduler when loader starts to have new requests
2020-08-15 21:57:00 -07:00
a035e7a872
FastRestore:LoaderScheduler:Add knobs
2020-08-15 21:44:51 -07:00
7e302b5910
FastRestore:Loader:Add scheduler to dispatch requests based on resource utilization and request priority
2020-08-15 21:21:51 -07:00
2f52c5f79b
Merge branch 'release-6.3'
...
# Conflicts:
# fdbserver/RestoreLoader.actor.cpp
2020-08-11 22:45:43 -07:00
c27542d4e9
FR:SampleBackup:Send reply when receive duplicate msg
2020-08-07 16:50:36 -07:00
Yi Wu