862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
a4aad591fd
FR:Remove applyStagingKeysBatchLock flow lock
2020-09-23 08:47:37 -07:00
6da7e56e03
FastRestore:Add more trace events
2020-09-22 19:57:19 -07:00
17ece3d477
FastRestore:Fix FastRestoreApplierTransactionRateControl events
2020-09-22 13:25:43 -07:00
b4254473d7
FastRestore:Add transaction rate info tracer
2020-09-22 09:10:11 -07:00
002b1bec4c
FastRestore:Control write traffic at each applier
...
Controller assigns each applier a write rate.
Applier keeps the write-rate worth of transactions outstanding to DB.
This is to avoid heavily overloading DB while still keep enough
traffic to DB to get a good write throughput.
2020-09-22 08:14:28 -07:00
a3c861b59d
Merge pull request #3777 from apple/release-6.3
...
Merge Release 6.3 into master
2020-09-14 10:32:50 -07:00
ce92f1a224
FR:Init StagingKey when created
...
The key field was used in various places, such as figuring out the conflict key range.
We should not leave it empty
2020-09-09 16:12:32 -07:00
c9171e15b8
Change FastRestoreApplierPrecomputeResultSkipUnexpectedBackupMutation to SevError
2020-09-09 10:42:51 -07:00
5880bd9fa7
BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error
2020-09-09 08:47:43 -07:00
9eeed8144e
FastRestore:Fix bug in precomputeResult
2020-09-08 16:37:12 -07:00
12edadd059
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# fdbclient/Knobs.cpp
# fdbclient/MasterProxyInterface.h
# fdbrpc/simulator.h
# fdbserver/MasterProxyServer.actor.cpp
# tests/fast/CycleAndLock.txt
# tests/fast/TxnStateStoreCycleTest.txt
# tests/fast/VersionStamp.txt
# tests/slow/ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
# tests/slow/ParallelRestoreOldBackupCorrectnessCycle.txt
# versions.target
2020-08-31 19:33:34 -07:00
a8bd628216
FastRestoreAtomicRestoreTest:Increase timeout from 2100 to 36000
2020-08-26 09:48:52 -07:00
2f52c5f79b
Merge branch 'release-6.3'
...
# Conflicts:
# fdbserver/RestoreLoader.actor.cpp
2020-08-11 22:45:43 -07:00
fe5902994c
Merge pull request #3605 from apple/release-6.3
...
Merge Release 6.3 to master
2020-08-05 23:37:44 -07:00
27a3d37594
RestoreApplier:Change receivedBytes from double to long
2020-07-31 22:10:18 -07:00
e22baa552b
FastRestoreApplier:Init stats variables
2020-07-31 21:56:03 -07:00
47c35a7a69
FastRestore:Add stats to ApplierBatchData
2020-07-31 14:59:45 -07:00
d16db8e733
FastRestore:Fix segmentation fault
2020-07-30 12:10:32 -07:00
ad915e462e
Add knob FASTRESTORE_NOT_WRITE_DB to skip writting to DB
2020-07-30 10:17:17 -07:00
ef8c1060a2
Merge branch 'master' into mengxu/tmp-merge-6.3
2020-07-13 10:15:56 -07:00
b09dddc07e
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/ConfigureCompiler.cmake
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/fdbrpc.vcxproj
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogSystemPeekCursor.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/Status.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/flow.vcxproj
2020-07-10 15:06:34 -07:00
f3302833ce
Merge pull request #3435 from apple/release-6.3
...
Merge Release 6.3 to master
2020-06-30 10:08:28 -07:00
a5febfbc75
Remove spurious comments
2020-06-29 10:26:01 -07:00
97e26d8eb0
FastRestore:Count appliedBytes
2020-06-29 10:18:18 -07:00
4b35685e99
FastRestore:Adjust knob based on circus test
2020-06-29 10:13:18 -07:00
e57dba00bd
FastRestore:Lower priority for RestoreApplierReceiveMutations actor
2020-06-27 15:16:38 -07:00
ecd2d8b239
FastRestore:Add counters for applier and disable FlowLock on applyStagingKeysBatch
2020-06-27 00:20:54 -07:00
00b93dd562
Merge pull request #3311 from apple/release-6.3
...
Merge Release 6.3 into master
2020-06-08 21:31:10 -07:00
d5025a1779
getAndComputeStagingKeys: Improved handling of not exist keys
2020-06-03 15:32:36 -07:00
ccaac162e2
Resolve performance concerns of nearly-no-op debugMutation being frequently called
...
This introduces unhygenic macro variants that inline a `ENABLED &&`
before the TraceEvent. This way, they get entirely compiled out unless
enabled.
Then rewrite all debugMutation uses via sed.
2020-05-13 18:44:15 -07:00
27da91ab9e
Merge remote-tracking branch 'upstream/master' into mutation-debugging
2020-05-13 12:51:44 -07:00
d22af629cd
FastRestoreApplier:Add applierID and batchIndex for precompute stage
2020-05-04 16:32:09 -07:00
135f6443da
FastRestoreApplier:Add trace to track applying status
2020-05-04 15:02:53 -07:00
41c0a1768f
FastRestore:Make FastRestore event type more descriptive
2020-05-01 10:27:08 -07:00
2c17fff6bb
FastRestore:Add batchIndex into metric event name
2020-05-01 10:05:33 -07:00
061bcd2fb4
FastRestore:Replace typeString with safe getTypeString func
...
Also fix compilation error in previous commit
2020-04-13 15:15:54 -07:00
dbc9c23193
FastRestore:Loader:Send mutations at different versions in the same message to appliers
...
This increases the bandwidth sent from loaders to appliers.
2020-04-12 10:46:58 -07:00
cd8215ecf2
Remove version stamp ops from RestoreApplier
...
Version stamp ops are converted into SET at the proxy, so the backup files
will never have them.
2020-04-06 22:27:47 -07:00
6bce67ca75
FastRestore:Apply clang-format
2020-04-01 21:27:54 -07:00
ff3fd5c380
FastRestore:Fix:Type can be invalid for typeString array
2020-04-01 10:47:06 -07:00
c69c959428
FastRestore:Fix:It is legal for a backup key not exist in DB
2020-03-31 22:02:17 -07:00
25e96a13d3
FastRestore:Fix clearrange on a key mistakenly clear other keys
2020-03-31 17:45:19 -07:00
212dadc2a1
Fix bug in add mutation on applier
...
For clear range mutation, we may clear the right boundary key which should not be cleared.
2020-03-31 16:51:08 -07:00
33c4be9c42
Improve debug message for debug mutations
2020-03-31 16:00:51 -07:00
40d10aa990
Fix debugMutation uses that were concurrently added in new backup code
2020-03-27 04:01:18 -07:00
0f57bf9685
Remove a SevError event
...
The same mutation can be present in overlapping mutation logs. Thus we cannot
assert its absence. This can be caused for multiple reasons. One possibility
is that new TLogs can copy mutations from old generation TLogs; another one
is backup worker is recruited without knowning previously saved progress.
2020-03-25 15:23:21 -07:00
196127fb92
Address review comments
2020-03-23 14:15:36 -07:00
3f31ebf659
New backup:Revise event name and explain code
2020-03-23 10:55:44 -07:00
3513bbefe6
StagingKey uses mutation instead of a vector of mutations for each log version
...
Because each log version contains commit version and subsequence number, each
key can only have one mutation for its log version. This simplifies
StagingKey::add() a lot.
2020-03-18 16:44:17 -07:00
Meng Xu