63359bfc8b
FastRestore:handleInitVersionBatchRequest:Ensure exact once execution
...
Also increase the test workload for BackupAndParallelRestoreWithAtomicOp test
2019-11-03 17:26:13 -08:00
ef14f7a718
Merge pull request #2292 from etschannen/master
...
Merge 6.2 into master
2019-10-25 09:18:20 -07:00
a30e6ec147
Merge pull request #2277 from xumengpanda/mengxu/fastrestore-atomicOpTest-increaseLoadAndBugFix-PR
...
Performant restore [7/XX]: Add tests for transactionBatchSizeThreshold when apply mutations
2019-10-24 21:21:14 -07:00
3325980c03
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/DataDistribution.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/WorkerInterface.actor.h
# fdbserver/worker.actor.cpp
# versions.target
2019-10-24 17:38:15 -07:00
2f34ee684f
fixed indentation issues
2019-10-24 13:21:28 -07:00
b74e5b1543
added sample file for attrition test outside of simulation
2019-10-24 13:10:59 -07:00
7af3239ee7
FastRestore:AtomicOpTest:Debug:1 key per group for ops keyspace
2019-10-23 14:36:34 -07:00
ba7e499efe
FastRestore:AtomicOpTest:Limit 1 actor per client
2019-10-23 14:04:14 -07:00
e676348710
Merge pull request #1955 from fzhjon/mark-ss-failed
...
Add fdbcli and API command to mark storage servers as permanently failed
2019-10-22 23:36:30 -07:00
96d463bab6
FastRestore:Fix bug in applying mutations and increase atomicOp test worload
...
When Applier applies mutations to the destination cluster, it advances the
mutation cursor twice when it should only advance it once.
This makes restore miss some mutations when the applying txn includes
more than 1 mutations.
2019-10-22 23:24:23 -07:00
01b4fb5e5e
CMake:Add performant restore test to CMake
2019-10-21 14:52:03 -07:00
970327b554
FastRestore:Add ParallelRestoreCorrectnessAtomicOpTinyData.txt
2019-10-21 14:42:11 -07:00
450a09e117
Code Review Changes
2019-09-24 15:48:50 -07:00
471e283128
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-09-18 11:49:07 -07:00
680f0e3bd8
updated CMakeLists
2019-09-13 09:30:40 -07:00
05eac989da
unpair the cycle test with configure test for now, because tests were timing out
2019-09-13 09:20:06 -07:00
5bf8d61d81
pair configureDatabase tests with cycle tests to ensure that changing the configuration does not corrupt data
2019-09-04 18:41:22 -07:00
807b02551e
updated help message and changed existing workload to use mark as failed feature
2019-08-27 14:39:43 -07:00
10be4b8127
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-08-07 04:20:14 -07:00
da9c4e97d3
Added new pager interface, IPager2, whose write interface enables forcing the user to handle a page update causing a copy to a new Page ID. Implemented FIFOQueue<T> which uses pages of T stored in a Pager2 instance to implement a FIFO queue. Implemented COWPager, a copy-on-write Pager2 in which all page writes cause a change of Page ID. VersionedBTree, still only operating in single-version mode, now uses Pager2.
2019-08-07 02:36:33 -07:00
f453c12f0f
remove object serialization flag from TestRunner
2019-08-06 10:20:01 -07:00
438bc636d5
Rename max_machine_failures_without_losing_X to max_zone_failures_without_losing_X in status.
2019-07-30 14:02:31 -07:00
8149b5b352
Merge pull request #1413 from atn34/change-connection-file
...
Switch cluster file feature
2019-07-26 13:27:37 -07:00
9afd162e2f
remove snap v1 related code
2019-07-25 17:29:31 -07:00
a2f2ad96ff
code review comments and merge to master changes
2019-07-24 15:36:28 -07:00
c18558cf55
enable DD mode in restore based on test spec
2019-07-24 15:36:28 -07:00
24cbe313a1
Snap*Restart in ver 2 and buggify off in restore
2019-07-24 15:36:28 -07:00
9ad9bd4c1f
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-24 15:22:26 -07:00
2f9f3c184f
Merge pull request #1870 from alexmiller-apple/txnStateStore-workload
...
Add ability to bulk load data into TxnStateStore
2019-07-22 13:20:39 -07:00
4ac1a0f557
Add ability to bulk load data into TxnStateStore
...
* Changes BulkLoad workload to support a specific volume of data to load
* Changes BulkLoad and Cycle to correctly handle \xff in keyPrefix
* Adds BulkLoad to TxnStateStoreCycleTest
2019-07-19 18:01:24 -07:00
3045826e3c
Merge pull request #1819 from mpilman/flatbuffers-fixes2
...
Flatbuffers fixes2
2019-07-19 16:33:50 -07:00
c3a8ae4752
Merge pull request #1791 from fzhjon/fetch-keys-requests-priority
...
Introduce priority to fetchKeys requests from data distribution
2019-07-19 14:54:51 -07:00
1ac2d01b03
Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2
2019-07-18 09:50:08 -07:00
d5caf0c1b4
Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2
2019-07-16 14:47:40 -07:00
f3d3f80ed2
Merge branch 'master' into bugFix
2019-07-15 14:49:19 -07:00
4b0fdabae5
mark test file as IGNORE and comment out dead placeholder code
2019-07-15 09:45:16 -07:00
1a18c859c7
knobified the durability lag rate controls
2019-07-12 18:50:56 -07:00
969957e619
Merge branch 'master' into change-connection-file
2019-07-12 11:39:19 -07:00
1cf036fc9f
Remove SnapTestAttrition from ctest
...
SnapTestAttrition the test file was removed in a53bf928
2019-07-12 10:43:42 -07:00
bc29b1e78a
Use flatbuffers for testing
2019-07-11 23:03:31 -07:00
274cb40eb8
Run tests under valgrind when USE_VALGRIND
2019-07-11 23:03:31 -07:00
35be45d4ce
Remove code accidentally left for testing ctest fix
2019-07-11 10:01:26 -07:00
695ca5e8e9
moved location of DDMetricsExclude test txt file
2019-07-11 09:56:59 -07:00
f12a3909f3
renamed workloads and made code style adjustments
2019-07-11 09:56:58 -07:00
1e9d31597c
removed extra parameter from getRange, added knob to guard new changes, and adjusted style/formatting in several places
2019-07-11 09:56:58 -07:00
f707d186fe
added new priority for fetchkeys requests and adjusted ddmetrics workload to run parallel with mako
2019-07-11 09:56:58 -07:00
3b8998b836
added workload to measure data distribution speeds
2019-07-11 09:56:58 -07:00
62d7ee9424
Fix ctests so each restart test has a seed unique to the first test
...
and any other test
2019-07-11 09:56:51 -07:00
a53bf9289a
remove SnapTestAttrition because it is causing correctness errors
2019-07-09 17:13:24 -07:00
1582a2a24d
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-09 13:41:54 -07:00
1b939d5208
Merge pull request #1749 from satherton/feature-redwood
...
Update redwood storage engine to latest correctness-passing version
2019-06-28 16:22:06 -07:00
83fae6cc15
Fix ExternalWorkload not being a part of the old build/test system.
2019-06-25 21:42:35 -07:00
8f06ccb065
Increased testDuration for DifferentClustersSameRV test
...
Unlocking the second database can take a while because the local
ratekkeeper throttles reads until the updateStorage actor catches up
after advanceVersion
2019-06-25 16:56:49 -07:00
0fe6edc254
Merge pull request #1678 from mpilman/features/external-workload
...
Features/external workload
2019-06-25 13:53:19 -07:00
f1f1081202
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-06-24 20:17:49 -07:00
afb0dbcd1c
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-06-20 19:11:29 -07:00
148f922813
Revert "Add CycleTest (currently times out)"
...
This reverts commit f367fcaf25
2019-06-20 18:34:41 -07:00
1611f0385a
Remove symbolication from ctest, as a large number of backtraces times out tests.
...
HugeArenaAllocation in particular is a bit spammy of backtraces,
intentionally, and is causing testing in the CI to time out.
2019-06-19 13:37:59 -07:00
2eff2b7e21
First simple test is working (but very buggy)
2019-06-19 13:03:41 -07:00
d4b7f9b606
Fixed some cmake, compile, and IDE warnings.
2019-06-17 18:55:49 -07:00
f367fcaf25
Add CycleTest (currently times out)
2019-06-11 14:10:34 -07:00
02e173b601
Add changeConnectionFile method to Transaction
...
Also add tests
2019-06-11 13:58:22 -07:00
8144882d7b
Merge branch 'apple-master' into features/local-rk
2019-06-10 19:40:25 -07:00
46b77819aa
Fixed LocalRatekeeper test
2019-06-10 18:25:58 -07:00
100789b354
More bug fixes in handling upperBound changes in modified pages and worst-case delta size calculation. Normalized some formatting in debug statements. Fixed compile error on linux. Updated test specs.
2019-06-05 20:58:47 -07:00
66811b7bd2
update to latest version
2019-06-03 16:49:19 -07:00
d3a179b6f9
Multiple bug fixes
...
- wait for snapTLogFailKeys in a loop, otherwise in some race
condition it can cause a false assert
- in single region, there does not seem to be a guarantee of
tagLocalityListKey for a given DC ID, avoiding that assert for now
- to find the workers that are coordinators, looking up by primary
address is not sufficient in some cases, hence looking by both
primary and secondary address
- test make files to reflect the location of the new test cases
2019-05-28 22:07:46 -07:00
16fc7b6aaa
move SnapTests into restarting/from_6.2.0
2019-05-28 22:07:46 -07:00
c53c4fa898
reduce the snap test durations
2019-05-28 22:07:46 -07:00
4083af0b01
Avoid using trackLatest for TLog pop test cases
2019-05-28 22:07:46 -07:00
61e93a9304
Address review comments and minor fixes
2019-05-28 22:07:46 -07:00
8838ba3d3b
Split SnapTestSimpleRestart into two test cases
2019-05-28 22:07:46 -07:00
8370871e4c
stale RESTORE option related code removed
2019-05-28 22:07:46 -07:00
898bed66c1
Allow only whitelisted binary path for exec op
2019-05-28 22:07:46 -07:00
f129d996fe
Remove dumpAfterTest=true in snap tests
2019-05-28 22:07:46 -07:00
281c785f94
'--restoring' cmd line arg removed for fdbserver
...
'--restoring' command line option was introduced to indicate
simulated fdbserver to restore from snapshot and restart the cluster.
As part of this change that option is removed and restore
information is stored in the restartInfo.ini.
2019-05-28 22:07:46 -07:00
3d5998e9dd
tlog: when pops are disabled, store them & replay
...
In Tlogs, disable pop is done whlie taking snapshots. Earlier, tlogs
were ignoring the pops if it got pop requests when pops were
disabled. In this change, instead of ignoring the pop - it remembers
the list of pops in-memory and plays them once the popping is
enabled.
2019-05-28 22:07:46 -07:00
4bc4c615da
exec op to all tlog, restore change in test &other
...
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
2019-05-28 22:07:46 -07:00
a7920ef311
Merge branch 'master' of https://github.com/apple/foundationdb into MakoWorkload
2019-05-28 18:21:02 -07:00
ca034846b4
update comments, parameter names and descriptions
2019-05-28 17:29:45 -07:00
0f46b6c926
Add sql_create_report call to generate sql file
...
After all testing iterations are complete, run sql_create_report to generate the sql
file and (for now) archive it with the job.
SNOW-78373
2019-05-28 10:22:24 -07:00
31939fd044
Merge pull request #1597 from atn34/cmake-flatbuffers-off
...
Hardcode flatbuffers to off for ctest
2019-05-23 11:13:54 -07:00
b451c2cd56
Merge pull request #1497 from alexmiller-apple/fastrecovery
...
Add an \xff keyrange that is backed by the txnStateStore.
2019-05-23 10:52:35 -07:00
57968d9df7
Merge branch 'master' of https://github.com/apple/foundationdb into MakoWorkload
2019-05-21 16:24:11 -07:00
12a51b2d39
fix bugs, update naming and comments, refine functions
2019-05-20 18:26:30 -07:00
f673be456b
Added jenkins file for Snowflake
...
We (snowflake) want to run build and simulation tests against PRs.
For this we need a jenkins file in the repository. This should not
impact anything else
2019-05-17 11:17:40 -10:00
8082362830
Hardcode flatbuffers to off for ctest
2019-05-16 13:02:06 -07:00
106bb7677d
update
2019-05-15 12:58:12 -07:00
6afce01744
Implementation complete (not yet working)
2019-05-13 14:15:22 -07:00
5678a7417e
Mako Workload
2019-05-09 15:55:05 -07:00
0a7c74eac9
Add TxnStateStoreCycleTest to CMake
2019-04-25 17:09:35 -07:00
797d431934
Add an \xff keyrange that is backed by the txnStateStore.
2019-04-25 17:04:20 -07:00
0dd57dd454
Fix a few typos
2019-04-13 17:29:30 -07:00
d2e74cb2c0
Fix stupid rounding error
2019-04-08 11:05:29 -07:00
bdba8e22eb
Added test and bugfixes
2019-04-08 11:05:29 -07:00
2627bcd35e
Merge branch 'master' into feature-metadata-version
2019-03-10 21:13:28 -07:00
121ed4acf9
Trace the cmake SEED
...
This will be included in the repro instructions, so we need to trace it
to keep track of it.
2019-03-08 15:03:56 -08:00
21f55e1878
Merge pull request #1190 from bnamasivayam/restore-multiple-ranges
...
Add support for restoring multiple ranges.
2019-03-05 10:15:55 -08:00
075fdef31a
Merge branch 'master' into feature-metadata-version
...
# Conflicts:
# fdbclient/DatabaseContext.h
2019-03-03 22:58:45 -08:00
057ebe56e4
fix: unknownCommit handling relied on soleOwnership of the version stamp keys, so we need to use a second key to track the commit version for the metadataVersionKey
...
renamed a confusing option
2019-03-03 21:31:40 -08:00
Meng Xu