1d081401c6
Fix Tag::load bug
2020-10-22 14:13:14 -07:00
95654ce7e7
Change dbLocked to snake_case naming db_locked
2020-10-22 11:35:39 -07:00
2b4ec39a2f
Avoid passing references from an iterator into an insertion that invalidates the iterator.
2020-10-22 11:30:08 -07:00
1d64b2c335
clang-format
2020-10-22 11:28:03 -07:00
b728fe473e
Add consistencycheck command to special keys
2020-10-22 11:08:54 -07:00
dec643987f
Add wait time after dd not found error
2020-10-21 16:58:23 -07:00
8d937a56f2
Throw operation_failed() if dd not found when calling ddMetricsRequestServer
...
Remove time_out error in DDMetricsList rpc
Add retry logic for dd special key when dd not found
Update FuzzApiCorrectness test
2020-10-21 15:28:03 -07:00
e183a10c8c
Add "waitForDestUID" flag for abortBackup
2020-10-20 22:55:31 -07:00
8cc3e4d3c6
Merge release-6.3 into master
2020-10-19 22:51:56 -07:00
c84a5f4014
change IChecksum interface and remove coordinator protocol version optional
2020-10-19 21:39:07 +00:00
a4ccf8c5b6
add checksum class to change checksum in the future. expose get server protocol api
2020-10-19 16:57:11 +00:00
944f30484a
Merge pull request #3759 from dongxinEric/misc/3739/expose-time-since-last-recovery
...
This resolves issue #3739 by exposing time since last full recovery.
2020-10-19 09:03:31 -07:00
47a9bc9d3d
Fix commit_unknown_result issue
2020-10-18 23:11:19 -07:00
bfd3328448
Fix a race between submit and abort backup
...
After submit a backup, immediately abort the backup may cause a rare race
condition, which results in BackupCorrectnessLeftoverVersionKey error.
Specifically, in the StartFullBackupTaskFunc:
1st Txn sets the destUid at the source database and the 2nd Txn writes the dest
DB.
An abort can come after the 1st Txn succeeds, and clears the config rage so
that the 2nd Txn above would fail. Because 2nd Txn didn't write destUid, the
3rd Txn of abort can't read the correct source DB for latestVersionKey, which
contains the destUid value.
The fix is to let the 1st Txn of abort to wait until destUid becomes valid.
2020-10-18 23:11:15 -07:00
545ee4269d
master conflicts
2020-10-19 01:03:54 +00:00
1c533e7363
fix flowtransport conflicts
2020-10-19 01:00:03 +00:00
c1d88b39ab
Merge remote-tracking branch 'origin/master' into add-upgrade-and-backup-restore-test
2020-10-17 11:35:12 -07:00
c1b4e62bde
Change the trace from SevInfo to SevDebug
2020-10-15 13:55:31 -07:00
0718329705
Make getDesiredClusterKeyName const
...
This fixes a virtual function overload in NameQuorumChange
2020-10-15 10:39:38 -07:00
24ea35e56f
Merge pull request #3748 from sfc-gh-ljoswiak/visibility-2
...
Add TLogVersion::V6
2020-10-14 17:35:32 -07:00
bf00369576
getRange only enters special key space codepath when both begin key and end key are in (\xff\xff, \xff\xff\xff)
2020-10-14 16:57:38 -07:00
e47e0108a7
Add new TLogVersion
2020-10-14 15:41:10 -07:00
a1e868a569
Merge pull request #3862 from sfc-gh-tclinkenbeard/use-override-more
...
Add uses of override keyword, remove unnecessary uses of virtual
2020-10-14 15:06:45 -07:00
bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
...
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
4eb20a1283
merge anoyes/stable-interface and add back in isCompatible
2020-10-12 20:39:37 +00:00
2f5b0bef08
switch to test newer incompatible version. Fix PR comments. Modify schema
2020-10-12 18:29:16 +00:00
a4e1efbc0e
revert MultiVersionTransaction
2020-10-12 18:25:14 +00:00
5488ff1d81
draft diff protocol
2020-10-12 18:24:03 +00:00
b233f44d2d
remove some print statements and spin lock that was used for debugging
2020-10-12 18:20:37 +00:00
41843f07e6
add simulator support for different process versions and ProtocolVersion test
2020-10-12 18:19:31 +00:00
76d0027fa2
merge anoyes/stable-interface and add back in isCompatible
2020-10-12 18:18:30 +00:00
d3b57d9873
Merge pull request #3823 from sfc-gh-jfu/jfu-snapshot-record-version
...
Have snapshot record to system keys and write lastEpochEnd on recovery
2020-10-09 07:27:19 -07:00
dc4612712d
Merge remote-tracking branch 'origin/master' into use-override-more
2020-10-09 07:12:41 -07:00
ccf2ca73f3
Fix Windows CMake build
2020-10-08 23:58:58 -07:00
76440018fb
Remove unnecessary error thrown when calling clear range on the lock special key range
2020-10-08 21:39:13 -07:00
5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
...
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
c66a775d64
Merge remote-tracking branch 'upstream/master' into add-lock-unlock-to-special-keys
2020-10-08 14:23:16 -07:00
53d3759e73
Add lock command to special key \xff\xff/dbLocked
2020-10-08 14:23:02 -07:00
2def07a18d
Apply suggestions from code review
2020-10-08 13:31:47 -07:00
a9607bdcec
Explicitly seal classes that inherit but aren't inherited from
2020-10-07 21:58:24 -07:00
a3de8c333b
Use override where applicable in fdbclient
2020-10-07 16:57:38 -07:00
f03a355c4e
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/downloads.rst
# fdbrpc/FlowTransport.actor.cpp
# fdbrpc/FlowTransport.h
# flow/Arena.h
# flow/Knobs.cpp
# flow/Knobs.h
# packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
dea7000970
Merge remote-tracking branch 'upstream/master' into visibility-1
2020-10-06 18:38:15 -07:00
7218586f2f
adjust test spec and backup describe
2020-10-06 17:03:09 -04:00
76da0a7cbc
Backport #3426 to release-6.2
2020-10-05 13:53:51 -07:00
c622a11c70
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-10-05 13:15:56 -04:00
43ac7f21a4
Mark toString methods const
2020-10-04 23:27:10 -07:00
d944e13e5b
Make TaskBucket::getAvailableSpace const
2020-10-04 23:27:09 -07:00
8ae0679112
Remove unnecessary separation of declaration and definition of version field
2020-10-04 23:27:05 -07:00
b294e148c5
another compile fix
2020-10-04 16:54:43 -07:00
614c8bc895
Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy
2020-10-04 16:20:24 -07:00
6326de052b
Remove unused LogFileWriter::FFs
2020-10-02 17:03:08 -07:00
cacf4be449
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-10-02 12:57:17 -04:00
268ba0bddc
Merge remote-tracking branch 'origin/release-6.3' into merge-6.3
2020-10-01 14:14:06 -06:00
13d1978c19
Merge branch 'release-6.3' into feature-fix-fault-tolerance
2020-09-30 19:00:31 -07:00
8c729ca8e6
only add additional fault tolerance for availability if automatic failover is enabled
2020-09-30 18:04:23 -07:00
c7a55fe15f
Merge pull request #3829 from yliucode/process-class
...
Support specifying proxy as process class and converting it to commit proxy
2020-09-30 16:46:24 -07:00
fe5c30e778
fault tolerance was not being properly increased when usable regions was 2 and satellites are configured.
2020-09-30 16:41:00 -07:00
0c579674e6
Add more help messages for configuring 'proxies'
2020-09-30 13:29:01 -07:00
e2705668be
Lower WATCH_TIMEOUT default value to 30.0
2020-09-30 11:52:51 -07:00
8da8c6ff0d
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-30 14:25:29 -04:00
44cd3b0999
add stopBackup to incrementalBackup workload
2020-09-30 14:24:52 -04:00
3705f5a6fe
Merge pull request #3820 from sfc-gh-tclinkenbeard/improve-type-safety
...
Improve ConfigurationResult and CoordinatorsResult type safety
2020-09-30 10:46:59 -07:00
c34385ef7b
Support proxy process class
2020-09-30 01:23:10 -07:00
21071b6214
Merge pull request #3813 from xumengpanda/mengxu/merge-to-master-PR
...
Merge 6.3 to master and resolve conflicts
2020-09-28 15:24:18 -07:00
cfaa207f88
fix spacing and newlines at ends of files
2020-09-28 16:32:20 -04:00
3aa4070adb
remove trace events used for debugging
2020-09-28 16:27:30 -04:00
e02caac2d9
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-28 16:20:33 -04:00
6c726ba8dd
Improve ConfigurationResult and CoordinatorsResult type safety
2020-09-27 15:29:15 -07:00
1fb11858e6
Fix failure caused by merge
2020-09-27 11:03:16 -07:00
8d9e7adac7
Merge pull request #3817 from yliucode/proxy-doc
...
Mention GRV & Commit proxy in docs and add backward compatibility for configuring proxies
2020-09-25 22:58:55 -07:00
480fc82779
Resolve review comments
2020-09-25 16:58:54 -07:00
bbc7ce581e
Resolve conflicts merging from 6.3 to master
2020-09-25 16:08:31 -07:00
3278e21b0b
Change description of GRV proxy and Commit proxy in docs and add backward compatibility support for configuring proxies in fdbcli
2020-09-25 10:50:48 -07:00
b6b87cd8d4
fileconfigure fails with Assertion false failed
2020-09-25 20:20:00 +03:00
862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
a96d6f85c5
Removed redundant field number_of_old_generations_of_tlogs from status json
2020-09-24 09:44:51 -07:00
6ed330572e
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-24 12:09:17 -04:00
feb3bda79e
Merge pull request #3797 from xumengpanda/mengxu/fr-write-traffic-control-PR
...
Fast Restore: Add write rate control
2020-09-23 15:50:08 -07:00
3a03c9bd74
Remove deprecated enumActionResult
2020-09-23 15:14:09 -07:00
d13a9d3c91
Make EBackupState an enum class
2020-09-23 15:14:09 -07:00
69580593dd
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-23 15:35:05 -04:00
50f681cd32
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-09-23 10:54:49 -07:00
b37daf88d4
add jitter to the delay
2020-09-22 22:24:40 -07:00
09ff05c629
backed out buggify of apiVersion because there are too many tests that depend on it for a patch release
2020-09-22 21:15:49 -07:00
e0fbadfeb5
Added a buggify so older API versions are run in simulation
2020-09-22 18:58:01 -07:00
c99627a7ab
Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-apiverison-batch-error
2020-09-22 17:06:17 -07:00
7dec47938e
protect api versions older than 630 from seeing batch_transaction_throttled errors
2020-09-22 17:05:09 -07:00
002b1bec4c
FastRestore:Control write traffic at each applier
...
Controller assigns each applier a write rate.
Applier keeps the write-rate worth of transactions outstanding to DB.
This is to avoid heavily overloading DB while still keep enough
traffic to DB to get a good write throughput.
2020-09-22 08:14:28 -07:00
9b7d75b6dd
WIP
2020-09-21 13:41:35 -04:00
7d606ebff4
Fix some accidental s/NULL/nullptr replacements
2020-09-20 11:33:18 -07:00
b4e209bd4e
Replace NULL with nullptr in fdbclient
2020-09-20 11:33:09 -07:00
a52ef0acb8
Merge pull request #3677 from sfc-gh-clin/add-setclass-to-special-key-space
...
Add setclass to special keys
2020-09-19 14:55:03 -07:00
d4e3e98957
Use new getRestoreSet in RestoreController
2020-09-17 17:56:36 -07:00
71309a79f9
Merge branch 'release-6.3' into range
2020-09-17 17:54:33 -07:00
cf69f455a9
Merge pull request #3785 from apple/release-6.3
...
Merge Release 6.3 to master
2020-09-17 14:43:56 -07:00
4df0f60729
Instead of using fully_recovered, use accepting_commits as a singal of DB turned available. Also add the number of old generations into status
2020-09-17 09:55:25 -07:00
ea27e4bd9c
Merge pull request #3772 from yliucode/rename
...
Rename master proxy as commit proxy
2020-09-16 20:06:50 -07:00
1fe31aa9e8
Merge pull request #3706 from yliucode/tlogs
...
Report missing old tlogs in recovery before storage servers are fully recovered
2020-09-16 14:07:06 -07:00
cc5bc16bd8
Rename more places from proxy to commit proxy
2020-09-15 22:29:49 -07:00
8224e17a08
Integrate extended getRestoreSet API into fast restore
...
The extended getRestoreSet provides a much smaller set of backup files for small keyrange restore;
This commit integrate it into fast restore so that fast restore does not have to filter out unneeded backup files.
2020-09-15 09:38:43 -07:00
f9a5b727a6
Add comments questions and TODOs
2020-09-14 16:28:01 -07:00
a3c861b59d
Merge pull request #3777 from apple/release-6.3
...
Merge Release 6.3 into master
2020-09-14 10:32:50 -07:00
0282a1745a
Address more review comments
2020-09-11 16:44:09 -07:00
0a7d2d31f1
Review some review comments from AJ
2020-09-11 16:17:23 -07:00
6dbcd42ebd
No need for a new protocol version since the serialization won't change. Added code for deserializing from old binary.
2020-09-11 16:16:56 -07:00
1b923477f7
Apply suggestions from code review
...
Co-authored-by: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-09-11 14:28:56 -07:00
260c8d9568
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-11 15:05:58 -04:00
22996284c7
added changes to allow writing of last epoch end version to special keys when performing recovery due to snapshot
2020-09-11 15:00:11 -04:00
537d6ffad2
change count
2020-09-11 17:46:14 +00:00
9947b416ee
Merge branch 'master' of https://github.com/apple/foundationdb into fdbtest
2020-09-11 15:27:35 +00:00
da468df628
fix unthrottle signal bug
2020-09-11 07:22:54 +00:00
084c69b2ad
fix reason serialization bug
2020-09-11 07:21:57 +00:00
35bef73a1c
Rename proxy to commit proxy
2020-09-10 17:44:15 -07:00
d270b15c89
Merge remote-tracking branch 'upstream/master' into add-setclass-to-special-key-space
2020-09-09 16:21:19 -07:00
2803e6be52
Remove enabling ACCESS_SYSTEM_KEYS
2020-09-09 16:17:43 -07:00
f10e9ea679
FastRestoreApplier:Add write conflict range
2020-09-09 12:12:14 -07:00
1155d015c9
fetch current log generation as well
2020-09-09 11:54:58 -07:00
e69ee87f33
Refactor blob failure to simulateBlobFailure function
2020-09-09 10:54:21 -07:00
989299dc74
Merge pull request #3751 from sfc-gh-tclinkenbeard/fix-tb-traces
...
Use consistent prefix for task bucket traces
2020-09-09 10:26:54 -07:00
5880bd9fa7
BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error
2020-09-09 08:47:43 -07:00
20733ae1eb
Simulate throwing blob http error immediately after read blob file
2020-09-09 06:47:19 -07:00
cffb54ba0b
Buggify decodeLogFileBlock with blob http error
2020-09-09 06:44:02 -07:00
f3986d201d
Buggify backup decodeRangeFileBlock with blob connection error
2020-09-09 06:40:41 -07:00
62dd1f7234
Merge pull request #3696 from sfc-gh-xwang/tag-report
...
report busiest write tag of each storage server
2020-09-08 15:21:14 -07:00
2935d3d4f6
change workload; solve some comments
2020-09-08 21:47:49 +00:00
4363dd0f25
This resolves issue #3739 by exposing time since last full recovery.
2020-09-08 14:26:01 -07:00
23e1ff694c
Report missing old tlogs in recovery between accepting commits and storage recovered
2020-09-08 13:35:42 -07:00
223c419f35
Merge pull request #3676 from sfc-gh-jfu/jfu-incremental-backup-only
...
incremental backup and restore
2020-09-08 12:06:25 -07:00
6c1b09829a
Replace TB_ trace prefix with TaskBucket
2020-09-08 11:38:53 -07:00
43e3e320e3
Refactor getrange for read-write module and add a test to make sure we have consistent results
2020-09-08 11:08:48 -07:00
a188f91fe6
Merge pull request #3738 from dongxinEric/misc/remove-unused-client-knob
...
The actual BYTE_LIMIT_UNLIMITED and ROW_LIMIT_UNLIMITED are defined i…
2020-09-08 10:46:27 -07:00
63a5f26a23
Remove implicitly setting read_system_keys when set SPECIAL_KEY_SPACE_ENABLE_WRITES, we can achieve read/set system keys through transaction object
2020-09-08 10:46:19 -07:00
0eb7092faa
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-incremental-backup-only
2020-09-08 12:53:29 -04:00
fda6e0c275
Use consistent prefix for task bucket traces
2020-09-05 21:42:51 -07:00
3728ed03dd
Resolve comments
2020-09-05 18:55:09 -07:00
1ad5e17458
add support for comparing original and current impls
2020-09-05 11:14:59 -07:00
53b7721d6c
Add additional trace information
2020-09-04 15:36:47 -07:00
1ca7fe1a05
Add span metadata message
2020-09-04 15:36:47 -07:00
443e9a251d
Igore value in set in excluding(fail) implementation
2020-09-04 15:21:47 -07:00
ee2ce6e758
Refactor ryw function which updating read result with writes in the transaction
2020-09-04 14:54:32 -07:00
ecf2c0109c
more concise status json
2020-09-04 18:40:45 +00:00
fb758bf937
update Schemas.cpp
2020-09-04 16:34:05 +00:00
ae7bf24353
Merge pull request #3549 from yliucode/grv-proxy
...
Separate out a new role GrvProxy to serve GRVs.
2020-09-03 19:03:45 -07:00
b9bdd6c0ff
The actual BYTE_LIMIT_UNLIMITED and ROW_LIMIT_UNLIMITED are defined inside the struct GetRangeLimits and these two unused declarations can be confusing.
2020-09-03 14:55:16 -07:00
f9624b5a1a
Avoid unnecessary copy
2020-09-03 13:56:23 -07:00
2fa8697796
Merge pull request #3726 from satherton/taskbucket-comments
...
Added some overview comments to TaskBucket
2020-09-02 13:58:44 -07:00
cac1b8a6e5
Update fdbclient/TaskBucket.h
2020-09-02 13:55:18 -07:00
87693cae81
merge master branch and resolve conflicts
2020-09-02 13:44:33 -07:00
430921f25f
Update local kv's contruction using result.arena()
2020-09-02 13:38:24 -07:00
sfc-gh-tclinkenbeard