4c93a23af9
Partitioned logs need to compute continuous begin Version
...
Because different tags may start at different versions, tag 0 can start at a
higher version. In this case, another tag's high version should be used as
the start version for continuous logs.
2020-03-18 16:41:35 -07:00
f697ccd1b9
Add describePartitionedBackup() for parallel restore
...
For partitioned logs, computing continuous log end version from min logs begin
version. Old backup test keeps using describeBackup() to be correctness clean.
Rename partitioned log file so that the last number is block size.
2020-03-18 16:41:35 -07:00
af967210ee
StagingKey can add out-of-order mutations
...
For partitioned logs, mutations of the same version may be sent to applier
out-of-order. If one loader advances to the next version, an applier may
receive later version mutations for different loaders. So, dropping of early
mutations is wrong.
2020-03-18 16:41:35 -07:00
eb6a889c78
Describe backup uses partitioned logs to find continuous end version
...
For partitioned logs, the continuous end version has to be done range by range,
where each range must contain continuous version for all tags.
2020-03-18 16:41:34 -07:00
84d79ce6f7
Check partitioned log files are continuous for RestoreSet
...
The idea of checking is to use Tag 0 to find out ranges and their number of
tags. Then for each tag 1 and above, check versions are continuous.
2020-03-18 16:41:34 -07:00
ace409b49a
Add subsequence number to restore loader & applier
...
The subsequence number is needed so that mutations of the same commit version
number, but from different partitioned logs can be correctly reassembled in
order.
For old backup files, the sub number is always 0. For partitioned mutation
logs, the actual sub number is used. For range files, the sub number is always
0.
2020-03-18 16:41:34 -07:00
d8c6bf585d
Include a total number of tags in partition log file names
...
This is needed for BackupContainer to check partitioned mutation logs are
continuous, i.e., restorable to a version.
2020-03-18 16:39:40 -07:00
c2623b5c20
Return partitioned logs for RestorableFileSet
2020-03-18 16:39:40 -07:00
55005952f2
Add a knob to switch mutation logs for parallel restore
...
Knob FASTRESTORE_USE_PARTITIONED_LOGS, default is true to enable partitioned
mutation logs. Otherwise, old mutation logs are used.
2020-03-18 16:39:40 -07:00
f6c27ca0d0
Check block padding is \0xff for new mutation logs
2020-03-18 16:37:02 -07:00
3664c6948b
Consolidate StringRefReader classes
...
Fix a compiler error of unused variable too.
2020-03-18 16:37:02 -07:00
3c088b2352
Integrate parallel restore with partitioned logs
...
In parallel restore, use new getPartitionedRestoreSet() to get a set containing
partitioned mutation logs. The loader uses a new parser to extract mutations
from partitioned logs.
TODO: fix unable to restore errors.
2020-03-18 16:33:58 -07:00
21feb78f8a
Add mutation log version names
...
I.e., BACKUP_AGENT_MLOG_VERSION for 2001 and PARTITIONED_MLOG_VERSION for 4110.
2020-03-18 16:33:58 -07:00
fe26037488
Add partitioned logs to BackupContainer
2020-03-18 16:33:58 -07:00
b4ab78764c
FastRestore:Add comment for integrating with new backup format
2020-03-18 16:33:58 -07:00
7bcc0e15f2
Backup worker: enable 50% of time in simulation
...
Make this randomization a separate one.
2020-03-18 16:33:58 -07:00
58a9bfa78b
Merge pull request #2820 from dongxinEric/fix/1977/add-back-trace-event-flush-failure-report
...
Fix/1977/add back trace event flush failure report
2020-03-18 16:11:44 -07:00
a476127f5f
Merge pull request #2802 from xumengpanda/mengxu/debug-master-PR
...
Fix correctness failure on master branch
2020-03-18 16:07:36 -07:00
648dc4a933
Merge pull request #2257 from zjuLcg/report-conflicting-key
...
Report conflicting keys
2020-03-18 13:39:42 -07:00
487d131b38
Merge pull request #2828 from Daniel-B-Smith/chrome-tracing-fixes
...
A couple of small fixes to the chrome tracing helper
2020-03-18 10:18:53 -07:00
291b6c3a53
Merge pull request #2829 from bnamasivayam/master
...
Increate QuietDatabase time to 90 seconds for real world cases.
2020-03-17 17:08:47 -07:00
d8a9034085
Merge pull request #2741 from ajbeamon/roughness-cleanup
...
Clean up and add comments to Roughness calculation
2020-03-17 15:45:35 -07:00
747434a13d
Increate QuietDatabase time to 90 seconds for real world cases.
2020-03-17 14:36:07 -07:00
5385d94fbd
Merge pull request #2827 from zjuLcg/temp-branch
...
Delete unnecessary parameters in MakoWorkload
2020-03-17 14:27:14 -07:00
5ad9807c21
Small fixes
2020-03-17 17:21:52 -04:00
a234e4693b
Merge pull request #2826 from etschannen/master
...
Merge 6.2 into Master
2020-03-17 13:58:47 -07:00
16cd68d3d9
Delete unnecessary parameters
2020-03-17 13:41:14 -07:00
e08f0201f1
merge release 6.2 into master
2020-03-17 12:51:47 -07:00
29f16630f5
update installer WIX GUID following release
2020-03-17 11:47:51 -07:00
330e78b06a
update version to 6.2.20
2020-03-17 11:47:51 -07:00
b44747fe12
Merge pull request #2726 from alexmiller-apple/fdbcli-hints
...
Update LineNoise, and use hints to provide inline argument descriptions.
2020-03-17 11:24:44 -07:00
377b7a3c86
Merge remote-tracking branch 'upstream/master'
2020-03-17 14:06:21 -04:00
31a9f0a26c
Fix the segfault
2020-03-17 11:03:46 -07:00
0a46c6276e
Merge pull request #2821 from etschannen/release-6.2
...
reverted broken change and updated release notes for 6.2.19
2020-03-17 09:50:27 -07:00
f92c7dbd64
added another release note
2020-03-17 09:46:39 -07:00
9dd7df564e
Update documentation/sphinx/source/release-notes.rst
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-03-17 09:42:15 -07:00
04052226df
reverting a change which causes data inconsistency between the primary and secondary
2020-03-17 09:41:44 -07:00
8e96e5a525
Expand comment about ignoring malformed escape sequences
2020-03-16 18:25:56 -07:00
7edeaa3da5
updated release notes for 6.2.19
2020-03-16 17:22:06 -07:00
787a5caaca
Merge pull request #2805 from ajbeamon/localized-allocation-trace-depth
...
Don't disallow allocation tracking when a trace event is open
2020-03-16 16:21:09 -07:00
c7a9d184f0
Merge pull request #2790 from tclinken/ignore-create-directory-errors
...
Ignore createDirectory error if directory already exists
2020-03-16 16:13:47 -07:00
c197520fa7
Merge pull request #2810 from alexmiller-apple/fdbcli-tlsinfo
...
Add a `tlsinfo` command to fdbcli that prints the certificate chain.
2020-03-16 15:47:32 -07:00
f1523bd472
Setting the network thread more than once is a no-op
2020-03-16 15:37:06 -07:00
96187618a0
Fix condition to check whether allocation tracing is enabled
2020-03-16 15:12:50 -07:00
7f559bc712
Cleanup code and apply clang-format
...
Self code review
2020-03-16 15:08:32 -07:00
ed4d02a3e4
Merge pull request #2812 from etschannen/feature-proxy-mem-limit
...
Limit the amount of requests the proxy can queue up in memory
2020-03-16 14:56:56 -07:00
1513df22f3
AutoQuorumChange:Exclude unreliable node from coordinator in simulation
2020-03-16 14:39:25 -07:00
7769218303
Move an increment after an ASSERT.
2020-03-16 14:11:07 -07:00
d8cfabe73b
Extend the allocation tracing disabling flag to cover more parts of trace logging as a precaution. Make it possible to disable via knob.
2020-03-16 13:59:31 -07:00
2038a56ff4
Merge pull request #2819 from etschannen/feature-first-proxy
...
A "proxy" class process would not be preferred as the "first proxy" for restore and DR purposes
2020-03-16 13:53:28 -07:00
Jingyu Zhou