c7694e7b50
Merge remote-tracking branch 'origin/master' into refactor-workloads
2020-10-14 09:21:47 -07:00
7ef8ceffcc
Merge remote-tracking branch 'origin/master' into add-incremental-backup-retry
2020-10-14 09:02:34 -07:00
e096297bc1
add traces
2020-10-13 16:07:09 -04:00
2bac64edc7
Add more comments and refactor for SkipList
...
There are no functional changes.
2020-10-13 12:55:08 -07:00
bd9ff04af2
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-fix-snapshot-backup-agent
2020-10-13 13:14:09 -04:00
bbc6d2aa03
Merge pull request #3872 from sfc-gh-clin:add-lock-unlock-to-special-keys
...
Add lock/unlock to special keys
2020-10-13 09:09:49 -07:00
dc35f2b4f5
Bug fix: In page remap cleanup, if a page update's next update was at exactly the oldest retained version then the earlier update would still choose to copy the updated page over top of the original (but it shouldn't) which would race with the later update's copy if the same remap cleanup cycle pops both updates from the queue.
2020-10-13 02:26:46 -07:00
69c6b2aa3b
Merge pull request #3888 from dongxinEric/bugfix/3885/fix-silient-se-commit-failure
...
Fix a silent commit failure in SS commit pipeline
2020-10-13 02:11:15 -07:00
48b9846a6a
adjust test spec and manual g_simulator setting
2020-10-12 17:12:20 -04:00
4f7f16068a
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-fix-snapshot-backup-agent
2020-10-12 17:10:28 -04:00
53d7d8ba63
add try-catch block that ignores backup_unneeded
2020-10-12 15:31:22 -04:00
89469921bb
Merge pull request #3891 from etschannen/feature-reset-proxy-connections
...
Reset a proxy's network connection with the master or resolvers if it is too far behind
2020-10-12 11:21:24 -07:00
2a2d513c55
Move 'debug_advanceMaxCommittedVersion' to immediately after the new 'durableVersion' got written to SE's commit buffer because:
...
- Before the interleaving change, SE's commit buffer is empty after each commit. And thus when SS restores the durable state after a reboot, the durable version on disk won't change;
- After the interleaving change, however, the commit buffer is pre-populated with new data for next commit when the current one is done. As a result of that, SE could change the durable version on disk without advancing max committed version.
Note this is not fixing a real bug in storage server, just a fix to make sure the durability debug contract(always increase max committed version, then commit new durable version, then increase min committed version) is followed correctly.
2020-10-12 11:14:02 -07:00
40281372e5
remove g_simulator manual setting
2020-10-12 13:09:14 -04:00
cd5e3914bf
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-fix-too-many-files
2020-10-12 13:07:25 -04:00
736f436712
Expose all errors from all futures joint.
2020-10-12 09:51:18 -07:00
e11b518ab1
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-fix-snapshot-backup-agent
2020-10-12 12:42:08 -04:00
d0a4e32754
Merge pull request #3790 from sfc-gh-tclinkenbeard/fix-ismetadatamutation
...
Fix isMetadataMutation condition
2020-10-12 08:28:57 -07:00
1378ecba4d
If a proxy is sufficiently far behind, reset network connections to attempt to fix the problem
2020-10-11 23:06:26 -07:00
dc1700d728
Prevent error from delay actor.
2020-10-11 14:29:44 -07:00
cf5337daa0
Improve performance of metadata check in applyMetadataMutations
2020-10-10 14:45:04 -07:00
a94821e5e3
first attempt at fixing issue with workload
2020-10-10 02:18:18 -04:00
28707de343
Moved debug_advanceMaxCommittedVersion to right before the commit future is waited. Throw the SE commit failure that was previously siliently swallowed.
2020-10-09 19:21:22 -07:00
e8e9e8064c
Add handling logic when buggify throws commit_unknown_result when commit lock
2020-10-09 14:26:56 -07:00
8bed7ba9a2
Add a loop to make sure we unlock the database in the end
2020-10-09 12:32:37 -07:00
3b66a1f2d4
Fix a couple places where we were creating vectors with default elements rather than reserving space.
2020-10-09 10:51:06 -07:00
be1ca196ee
turn off consistency check until after restore (where we remove the backup agent)
2020-10-09 13:30:01 -04:00
eefaa30fa4
Added retry loop for getReadVersion in IncrementalBackupWorkload
2020-10-09 07:43:21 -07:00
d3b57d9873
Merge pull request #3823 from sfc-gh-jfu/jfu-snapshot-record-version
...
Have snapshot record to system keys and write lastEpochEnd on recovery
2020-10-09 07:27:19 -07:00
dc4612712d
Merge remote-tracking branch 'origin/master' into use-override-more
2020-10-09 07:12:41 -07:00
ccf2ca73f3
Fix Windows CMake build
2020-10-08 23:58:58 -07:00
0b09833f21
add wait statement
2020-10-09 01:57:44 -04:00
9dd67bedaa
Add test for lock and unlock special keys
2020-10-08 18:28:47 -07:00
202f4b2735
Merge pull request #3870 from dongxinEric/misc/bug-fixes
...
Misc/bug fixes
2020-10-08 15:26:40 -07:00
5d902f9177
Merge pull request #3394 from dongxinEric/feature/range-split-points-based-on-size
...
Feature/range split points based on size
2020-10-08 14:25:11 -07:00
175d52312a
Prevent segmentation fault.
2020-10-08 13:36:15 -07:00
6062520197
Declare _canPipelineCommits to be const.
2020-10-08 13:35:27 -07:00
a9607bdcec
Explicitly seal classes that inherit but aren't inherited from
2020-10-07 21:58:24 -07:00
e258dffc65
Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining
...
Pipeline the commit process in SS.
2020-10-07 20:34:57 -07:00
8571dcfe28
Use override where applicable in fdbserver
2020-10-07 18:41:19 -07:00
4005abdeb3
Replace virtual with override in flow
2020-10-07 15:55:11 -07:00
012886a8d8
Fix Windows CMake build
2020-10-07 12:30:19 -07:00
e696c4a8eb
remove redundant line
2020-10-07 15:23:17 -04:00
31efdf1cc2
change race condition check logic for incremental backup container
2020-10-07 14:53:16 -04:00
675d9fa6ac
Adopt more suggestions from Steve as per offline discussions.
2020-10-07 09:57:28 -07:00
dea7000970
Merge remote-tracking branch 'upstream/master' into visibility-1
2020-10-06 18:38:15 -07:00
e94c372815
Fix serialization order
2020-10-06 18:33:29 -07:00
6594cdd9bb
Rename 'durableDelay' to 'durableMinDelay' for better readability.
2020-10-06 14:48:52 -07:00
b6a39123ec
Apply suggestions from code review
2020-10-06 14:46:45 -07:00
348ea10f3c
add retry loop to SimpleAtomicAdd check
2020-10-06 15:39:25 -04:00
c0f857cd64
attempt to have logAntiQuorum as an option preset prior to starting simulation and running tests
2020-10-06 14:47:13 -04:00
7e6c0c9d94
explicitly throw error when snapshot has failed rather than hitting hard-to-trace assertion
2020-10-06 14:31:41 -04:00
b610f01c77
Fine-grained interleaving
2020-10-06 09:32:22 -07:00
efe50b68e6
fix compile error
2020-10-05 14:16:52 -07:00
b4ad989252
use stack transaction instead of heap
2020-10-05 16:51:01 -04:00
a4cba5a162
Remove verifyOnly from Cycle workload
2020-10-05 13:19:12 -07:00
1ab1e12dcd
Use discontinueBackup in RestoreBackup.actor.cpp
2020-10-05 12:56:25 -07:00
7ba06a4434
fix: min and max compute estimate logging on the proxy was always zero
...
added comments and fixed formatting
2020-10-05 12:35:10 -07:00
c622a11c70
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-10-05 13:15:56 -04:00
a646df520d
add option to remove snaptest retry limit
2020-10-05 13:15:43 -04:00
b4ffc04e4e
Add UpgradeAndBackupRestore test
2020-10-04 23:27:10 -07:00
7f0d14c8e4
Modernize/refactor workloads directory
2020-10-04 22:29:07 -07:00
5807b1ec3d
changed the recent requests to be the per second amount; increased precision of cpu estimate
2020-10-04 19:31:40 -07:00
f546034366
do not prevent computePerOperation from being updated for small computeDurations. Added logging for the compute per operation. Protect against erroneously large compute estimates
2020-10-04 19:19:05 -07:00
da26b0411c
increased the proxy commit memory limit
2020-10-04 19:16:51 -07:00
52a6496a54
fix compiler errors
2020-10-04 16:50:54 -07:00
614c8bc895
Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy
2020-10-04 16:20:24 -07:00
0e96233015
Merge pull request #3842 from sfc-gh-tclinkenbeard/remove-unflushed-data-ratio-knob
...
Remove deprecated UNFLUSHED_DATA_RATIO knob
2020-10-02 12:32:39 -07:00
f757beaac1
Remove deprecated UNFLUSHED_DATA_RATIO knob
2020-10-02 10:36:21 -07:00
268ba0bddc
Merge remote-tracking branch 'origin/release-6.3' into merge-6.3
2020-10-01 14:14:06 -06:00
91a8367acb
Avoid slow task in ~DataDistributionTracker
2020-10-01 11:44:55 -07:00
b1180f8eb4
fixed naming and comments
2020-09-30 20:35:09 -07:00
b1570c740f
extraTlogEligileZones should consider the database available both during a failover and also if the cluster cannot recruit tlogs in the remote region
2020-09-30 18:10:04 -07:00
8c729ca8e6
only add additional fault tolerance for availability if automatic failover is enabled
2020-09-30 18:04:23 -07:00
9f61039858
more fixes
2020-09-30 16:52:58 -07:00
d7454ac7da
fixed compile error
2020-09-30 16:49:36 -07:00
2a279f64af
Merge branch 'release-6.3' into feature-fix-fault-tolerance
2020-09-30 16:42:18 -07:00
fe5c30e778
fault tolerance was not being properly increased when usable regions was 2 and satellites are configured.
2020-09-30 16:41:00 -07:00
8da8c6ff0d
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-30 14:25:29 -04:00
44cd3b0999
add stopBackup to incrementalBackup workload
2020-09-30 14:24:52 -04:00
3705f5a6fe
Merge pull request #3820 from sfc-gh-tclinkenbeard/improve-type-safety
...
Improve ConfigurationResult and CoordinatorsResult type safety
2020-09-30 10:46:59 -07:00
3aa92286aa
FastRestore:Fix segmentation fault
2020-09-29 22:28:52 -07:00
21071b6214
Merge pull request #3813 from xumengpanda/mengxu/merge-to-master-PR
...
Merge 6.3 to master and resolve conflicts
2020-09-28 15:24:18 -07:00
3ceb44f4df
add TEST macros in code paths
2020-09-28 16:40:38 -04:00
cfaa207f88
fix spacing and newlines at ends of files
2020-09-28 16:32:20 -04:00
3aa4070adb
remove trace events used for debugging
2020-09-28 16:27:30 -04:00
e02caac2d9
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-28 16:20:33 -04:00
57a80cfef2
added new SimpleAtomicAdd workload and adjusted SnapIncrementalRestore restarting test
2020-09-28 16:19:48 -04:00
6c726ba8dd
Improve ConfigurationResult and CoordinatorsResult type safety
2020-09-27 15:29:15 -07:00
c613fc6dee
Merge pull request #3761 from sfc-gh-tclinkenbeard/document-watchbytes-overhead
...
Add comments for WATCH_OVERHEAD_BYTES
2020-09-26 20:39:27 -07:00
480fc82779
Resolve review comments
2020-09-25 16:58:54 -07:00
3278e21b0b
Change description of GRV proxy and Commit proxy in docs and add backward compatibility support for configuring proxies in fdbcli
2020-09-25 10:50:48 -07:00
862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
58e043c7a5
Enable run loop profiler for test and multitest roles.
2020-09-24 14:14:55 -07:00
a78da8b3ce
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version
2020-09-24 14:40:01 -04:00
d1be303821
Merge pull request #3793 from sfc-gh-jfu/jfu-ibackup-test-timeout
...
Attempt to use version instead of test duration to have backup capture all changes
2020-09-24 11:17:49 -07:00
a96d6f85c5
Removed redundant field number_of_old_generations_of_tlogs from status json
2020-09-24 09:44:51 -07:00
ebad1cd9e6
change waitBackup result from int to EBackupState
2020-09-24 12:31:21 -04:00
5f7e784742
Merge pull request #3796 from xis19/master
...
Refactor fdbserver.actor.cpp for better readability
2020-09-24 09:12:47 -07:00
d0b8f7b1e3
Merge branch 'master' of https://github.com/apple/foundationdb into jfu-ibackup-test-timeout
2020-09-24 12:10:15 -04:00
sfc-gh-tclinkenbeard