627075e0ef
1. add test parameters; 2. change ContinuousSample initialization 3. add&change metrics
2020-07-06 18:18:15 +00:00
17570b5b10
Make the testspec more restrictive in terms of what can be set where.
...
Testspec is currently very permissive in very misleading ways. In particular,
the tester parser itself will swallow K=V settings and apply them at the test
level, which breaks how a person would expect the scoping to work. Other
settings apply to the entire simulation run globally, but appear to be workload
specific. Even further, others affect simulation cluster creation or test
harness behavior, but can again be set anywhere in a testspec.
This changes testspec parsing to error if a setting that applies globally is
anywhere but the top of the file, or if a setting that applies test-wide is
applied to a workload instead of a test.
2020-07-06 02:03:30 -07:00
121465b051
change metrics
...
fix latency sum bug
2020-07-03 21:49:44 +00:00
98b3ef69aa
1.change metrics; 2.change the way to generate key (avoid to much confict)
2020-07-03 00:20:39 +00:00
c1face27f1
fix dangling pointer bugs
2020-07-02 18:35:39 +00:00
c76e0cfe04
Add a ReadAfterWrite workload, to measure TLog->SS propagation delay.
2020-07-01 02:17:43 -07:00
6ebd3c8083
1. add range clear operation to workload
...
2. add tag logic to WriteTagThrottling
2020-07-01 06:35:14 +00:00
ad6904413c
Merge branch 'master' of https://github.com/apple/foundationdb
2020-07-01 04:58:02 +00:00
dc568cad21
add HealthMetricsWorkload and WriteTagThrottlingWorkload to prepare for new throttling feature
2020-07-01 04:56:21 +00:00
22f7f804b8
Merge branch 'release-6.3' into mengxu/merge-6.3-PR
2020-06-28 11:19:39 -07:00
90861ead6a
Reenable all tests in BackupAndParallelRestoreWithAtomicOp
2020-06-23 08:29:31 -07:00
b122575dda
BackupAndParallelRestoreWithAtomicOp:Turn on debug options
2020-06-22 16:53:43 -07:00
6e18b6454a
Remove obsolete file
2020-06-16 10:13:57 -07:00
96206a8032
Merge pull request #3368 from apple/release-6.3
...
Merge Release 6.3 to master
2020-06-15 20:15:22 -07:00
39c50798b8
Merge pull request #3341 from sfc-gh-clin/fix-rpc-calls-in-special-key-space
...
Fix fetchShardMetricsList_impl and add read cache in special key space
2020-06-15 16:12:38 -07:00
acbfe2e4c9
Revert "Revert "Initial RocksDB""
2020-06-15 12:45:36 -04:00
980bee1d13
Fix fetchShardMetricsList_impl and add read cache in special key space
2020-06-12 10:12:44 -07:00
9cd1614c82
Revert "Initial RocksDB"
2020-06-11 15:29:46 -07:00
a4dbb5dd01
Merge branch 'trace-batch-thread-hostile' into rocksdb-6.3
2020-06-11 15:53:57 +00:00
693d9e8b89
Merge branch 'master' into fdb_cache_wo_allocator
2020-06-09 15:09:58 -07:00
cf1ae52751
Merge pull request #3281 from apple/release-6.3
...
Merge Release 6.3 into master
2020-06-03 10:55:37 -07:00
cf13e00a8f
Merge remote-tracking branch 'origin/release-6.3' into fdb_cache_wo_allocator
2020-06-01 17:38:31 -07:00
d128252e90
Merge release-6.3 into master
2020-05-22 09:25:32 -07:00
5d361fe532
Copy/paste rebase onto 6.3
2020-05-22 15:02:51 +00:00
56e14a25fe
Normal correctness package can be built when USE_VALGRIND is on. Correctness and valgrind now stage their files in separate directories. Add CMakeCache.txt to testing packages.
2020-05-21 13:35:54 -07:00
6ce574f5ad
Merge remote-tracking branch 'upstream/release-6.3' into add-data-distribution-metrics
2020-05-17 23:36:52 -07:00
5632af3b4d
Removed inaccurate warning
2020-05-16 20:20:11 -07:00
25390f0968
Remove SimpleExternalTest.txt from fdb_test_files even if UBSAN is enabled
2020-05-13 22:46:07 -07:00
02307ba7b6
Merge branch 'master' into transaction-tagging
...
# Conflicts:
# fdbclient/DatabaseContext.h
2020-05-09 07:50:29 -07:00
ef724bf939
Merge remote-tracking branch 'upstream/master' into add-data-distribution-metrics
2020-05-08 18:39:28 -07:00
749fcd13b0
Merge branch 'master' into fdb_cache_wo_allocator
2020-05-08 16:23:29 -07:00
13ff1b3b20
Revert "Exclude Snap related tests from correctness test temporily"
...
This reverts commit f95bbc0ffa
2020-05-07 14:39:13 -07:00
b1055a8501
Merge branch 'master' into transaction-tagging
2020-05-05 16:03:39 -07:00
f329164fb4
Merge pull request #2532 from dongxinEric/feature/hot-read-key-detection-part-2
...
Feature/hot read key detection part 2
2020-05-05 14:33:34 -07:00
378951360d
Merge pull request #3086 from alexmiller-apple/ctest-use-crash
...
Make ctest use --crash when running simulation tests.
2020-05-05 12:49:21 -07:00
f040694bf0
Make ctest use --crash when running simulation tests.
...
So that tests with ASSERTion failures don't cause CI to run slow and
produce massive logs. There's also rarely a reason to continue through
assertion failures.
2020-05-05 03:11:09 -07:00
dd033736ed
Merge branch 'master' into fdb_cache_subfeature2
2020-05-04 17:29:43 -07:00
80a235aa80
Add some correctness tests
2020-05-04 10:15:18 -07:00
1927f15898
Now the ReadHotDetection tests can be run reliably. Added it to nightly bundle.
2020-04-28 13:26:02 -07:00
50ad4a33fe
Added possibility to include and exclude tests
...
Description
Testing
2020-04-28 16:19:04 +00:00
7dd7406c59
Merge branch 'master' into feature/hot-read-key-detection-part-2
2020-04-16 14:54:05 -07:00
b85dc16c6d
Merge branch 'master' into fdb_cache_subfeature2
2020-04-14 17:07:41 -07:00
65620b00f3
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-04-13 14:21:07 -07:00
f95bbc0ffa
Exclude Snap related tests from correctness test temporily
...
Those tests fail with high chance in nightly test.
2020-04-10 10:01:07 -07:00
2a856b23d3
Added test files and updated vsproj files
2020-04-06 09:53:18 -07:00
a0c8946f31
Merge branch 'master' into fdb_cache_subfeature2
2020-04-02 12:27:04 -07:00
38d7f16de0
FastRestore:Add new tests to ctests
2020-04-01 21:34:26 -07:00
16898408ae
FastRestore:Rename tests for new backup by adding NewBackup keyword
2020-04-01 21:31:54 -07:00
1eea388278
Add unit test for splitMutation test file
...
Comment out debug trace as well.
2020-04-01 21:27:18 -07:00
321b1b6ee7
Add RunRestoreWorkerWorkload into Parallel Restore WriteDuringReadAtomicRestore tests
2020-03-31 18:48:29 -07:00
a1d2779f04
Add ParallelRestoreWriteDuringReadAtomicRestore.txt
...
Test for fast restore with new backup.
2020-03-31 18:25:17 -07:00
0fa3bfa6f3
Increase timeout threshold for ParallelRestoreOldBackupWriteDuringReadAtomicRestore.txt
2020-03-31 18:21:36 -07:00
8471cb5351
Disable ParallelRestoreApiCorrectnessAtomicRestore.txt temporarily.
2020-03-31 15:52:07 -07:00
ccbbdc4ba4
Unit test:Verify splitMutation by comparing with intersectingRanges result
2020-03-31 12:13:02 -07:00
a63f8b839c
Merge remote-tracking branch 'upstream/master' into private-keyspace-framework
2020-03-31 12:05:42 -07:00
37ca0bf7e2
add code to handle the three flags in RangeResultRef
2020-03-30 23:27:09 -07:00
faab8a0d19
Add ParallelRestoreOldBackupApiCorrectnessAtomicRestore.txt
...
Add tests/ParallelRestoreApiCorrectnessAtomicRestore.txt back which
was mistakenly deleted in the previous commit.
2020-03-30 21:59:47 -07:00
73345a7ebd
Add new tests for parallel restore
...
Add two tests for parallel restore with old backup data
ParallelRestoreApiCorrectnessAtomicRestore.txt
ParallelRestoreOldBackupWriteDuringReadAtomicRestore.txt
2020-03-30 21:53:26 -07:00
60f6edc3b5
Merge pull request #2860 from zjuLcg/report-conflicting-key-roll-forward
...
Report conflicting key roll forward
2020-03-30 17:33:56 -07:00
cf7404a771
Merge pull request #2871 from xumengpanda/mengxu/parallel-range-log-file-loading-PR
...
Performant restore [24/xx]: Process range and log files in parallel for old backup format
2020-03-28 17:33:50 -07:00
92998d3df9
Add restore with old backup tests into CMakeLists
2020-03-28 16:08:22 -07:00
6e60c62d7b
Add ParallelRestoreCorrectnessAtomicOp.txt
2020-03-28 16:00:51 -07:00
137ea71203
Merge pull request #2865 from mpilman/features/build-test-harness
...
Build all joshua packages with cmake
2020-03-27 13:11:39 -07:00
f7233bade7
Rename ParallelRestoreCorrectnessAtomicOpTinyData.txt by removing TinyData
2020-03-27 13:08:59 -07:00
5c6752fb24
Add tests for new restore with old backup
2020-03-27 13:07:38 -07:00
97f8e46388
Sanity check subversion for log mutations
2020-03-27 13:07:08 -07:00
96a7b4b459
Disable ParallelRestoreApiCorrectnessAtomicRestore test
2020-03-26 14:59:50 -07:00
acaf91ac47
Merge branch 'master' into fdb_cache_subfeature2
2020-03-26 13:33:08 -07:00
cffcac92ad
Disable partitioned log for atomic restore workload
...
This will be reenabled after fixing test failures related to it.
2020-03-26 13:07:40 -07:00
7e2938b290
Merge branch 'master' of github.com:apple/foundationdb into features/build-test-harness
2020-03-26 12:47:55 -07:00
ce6c4cb8d8
Build and package TestHarness
2020-03-25 17:36:47 -07:00
f240d393f2
Add ParallelRestoreApiCorrectnessAtomicRestore.txt into ctests
2020-03-25 11:23:19 -07:00
6a8d6ddb8e
Introduce ParallelRestoreApiCorrectnessAtomicRestore.txt test
...
This covers ApiCorrectnessTest as workload for parallel restore.
2020-03-24 22:30:51 -07:00
289487559d
Revert "Revert "Merge pull request #2257 from zjuLcg/report-conflicting-key""
...
This reverts commit 804fe1b22e
2020-03-24 18:11:15 -07:00
d185359422
ApiCorrectnessAtomicRestore test: Increase timeout value
2020-03-24 17:36:17 -07:00
6df92a8b8d
A work version of strict workload test
2020-03-23 23:55:56 -07:00
247ab84323
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-03-23 17:01:17 -07:00
d7cf562ec4
Enable rollback workload for multi cycle restore test
2020-03-23 16:33:11 -07:00
85a9f6ab96
Merge branch 'master' into mengxu/fr-enable-rollback-and-abort-test-PR
2020-03-23 16:32:00 -07:00
14c641ce2b
Enable rollback workload
2020-03-23 16:29:01 -07:00
4756447b74
Enable abortAndRestartAfter option for restore cycle test
2020-03-23 15:15:14 -07:00
f936adedb0
Change tab to space for test workload file
2020-03-23 15:10:35 -07:00
d0bce1a105
Add ParallelRestoreCorrectnessCycle test into CMakeList
2020-03-23 13:56:33 -07:00
9c2c3b26d3
Merge branch 'master' into mengxu/fr-multi-cycle-test-PR
2020-03-23 13:54:20 -07:00
00ada7e086
Enable attrition workload for restore multi cycle tests
2020-03-23 13:50:05 -07:00
ace19eefe4
Introduce multi cycle tests for fast restore
2020-03-23 12:59:43 -07:00
5b2b2a8767
Enable assassination for restore cycle test
2020-03-23 12:28:04 -07:00
48db54424f
Add assassination workload to restore test workload
...
Add assert to ensure restore worker is reliable and not killed.
2020-03-23 11:11:13 -07:00
4f61973ede
Remove the ReportConflictingKeys.txt test from CMake
2020-03-19 10:49:54 -07:00
308d82245c
Remove ReportConflictingKeys.txt workload
...
The PR 2257 still has many correctness failures.
Remove the workload to avoid noise while we are working on the fixes.
2020-03-19 10:34:14 -07:00
aef9b515de
Change the workload to a more controlled test like ConflictRange test
2020-03-13 12:42:28 -07:00
7118759dfa
Delete code to test resolvers' performance, simplify the workload to only test correctness
2020-03-13 01:48:55 -07:00
357ca88b4c
test spec file
2020-03-10 18:51:45 -07:00
7a76e9556d
Merge remote-tracking branch 'upstream/master' into report-conflicting-key
2020-03-04 11:24:39 -08:00
087c6fa33d
Merge branch 'master' into feature-redwood
2020-02-26 12:25:04 -08:00
cd949eca71
Merge branch 'master' into fdb_cache_subfeature2
2020-02-26 11:22:08 -08:00
735e7c71f2
Add the cachetest back in.
2020-02-24 17:26:20 -08:00
be70b18c22
Remove Cachetest for merge.
2020-02-24 16:46:36 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
f1ec780b31
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-redwood
2020-02-21 17:43:11 -08:00
e9fa7803a8
FastRestore:Test:Let simulation tests turn buggify on and off
2020-02-20 07:06:04 -08:00
0507e9703c
FastRestore:Disable abort fast restore test
2020-02-20 06:54:14 -08:00
e794003b16
FastRestore:AtomicOpTest:Enable buggify
2020-02-19 20:25:07 -08:00
28768715f2
Custom bookkeeping allocator for PTree. Simple Cache workload.
2020-02-14 11:42:47 -08:00
0c7c815396
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# tests/CMakeLists.txt
2020-02-12 16:12:57 -08:00
8fb9722277
Fix newline at end of file
2020-02-11 13:52:56 -08:00
86089fdc1b
Merge branch 'release-6.2' into atn34/configure-locked
2020-02-11 13:51:41 -08:00
17660fb18d
Fix simulation test
2020-02-11 13:49:19 -08:00
1e1e75123f
Add simulation testing
2020-02-11 11:10:22 -08:00
5a9d420cb7
Merge remote-tracking branch 'upstream/release-6.2' into release-merges/20200210
2020-02-10 10:02:05 -08:00
5c294e2646
Several build improvements - enables parallel builds on Windows (again)
2020-02-09 08:34:14 -08:00
85cc35e81e
Merge branch 'master' into HEAD
2020-02-05 14:59:55 -08:00
058de86b88
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-02-05 10:07:36 -08:00
09f3690f09
Fix OPEN_FOR_IDE build
2020-02-03 10:42:05 -08:00
b0a1af1288
Added the actual read hot detection algorithm and logging machanism.
...
- When a shard has a read bandwidth larger than a threshold value(configurable via knob), and it's read-bandwidth/byte-size ratio is also larger than a threshold(configurable via knob), the corresponding shard tracker will run the algorithm
- The algorithm will divide the shard into 10MB(configurable via knob) chunks and try to find the chunk(s) that has large aforementioned ratio
- Then those ranges will be logged into TraceEvents. This will later do more like actually cache them.
2020-01-21 11:19:52 -08:00
7708034cc9
Added the function used to determin sub-ranges within a read hot shard that has a readSize-to-byteSize ratio higher than the knob value. Alos added unit tests for that function.
2020-01-21 11:15:52 -08:00
4de8f2fdfa
Move the test to basic correctness test
2020-01-16 14:42:14 -08:00
1d6ac716a1
Merge remote-tracking branch 'origin' into add-data-distribution-metrics
2020-01-15 13:20:04 -08:00
4fa619e782
Merge branch 'fdb_cache_subfeature2' into features/cache-initialization
2020-01-07 19:59:05 -08:00
d3d6016c90
Merge remote-tracking branch 'negoyal/fdb_cache_subfeature2' into features/cache-initialization
2020-01-07 19:53:09 -08:00
e8e5f2d118
Bug fixes in the cache role.
2020-01-07 16:51:40 -08:00
29b77863f0
Cache warmup and Consistency check workload changes.
2020-01-07 13:06:58 -08:00
10719200c3
A hack way to call API through getRange("\xff\xff/conflicting_keys\<start_key>", "\xff\xff/conflicting_keys\<end_key>").
2020-01-06 11:22:11 -08:00
414be7a0e4
Merge pull request #2479 from AlvinMooreSr/release_6.2_merge
...
Release 6.2 merge
2019-12-19 09:36:05 -08:00
21390c493a
Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
...
Resolved merge by keeping new test file from master branch: SampleNoSimAttrition.txt adding new constraint from Release branch about existing test file: SimpleExternalTest.txt
# Conflicts:
# tests/CMakeLists.txt
2019-12-18 09:05:08 -08:00
4d027428df
FastRestore:Test:Increase txn load to 2500txn per second
...
More transactions per second can create more data in a backup version,
this can help test if fast restore handles large memory allocation correctly.
2019-12-16 12:13:29 -08:00
650be617f1
FastRestore:Add tests to CMakefile
2019-12-12 10:32:13 -08:00
c9994422a9
Ignore instead of omit
2019-12-12 09:12:34 -08:00
960e87ee8c
Disable SimpleExternalTest for UBSAN temporarily
2019-12-12 09:12:34 -08:00
64abaaf02d
FastRestore:Reenable tests for nightly test
2019-12-12 07:57:52 -08:00
a6cfad6c3f
FastRestore: Add ParallelRestoreCorrectnessAtomicOpTinyData test back
2019-12-11 16:39:17 -08:00
04230b59bf
Increase load a bit
2019-12-07 21:14:56 -08:00
d95180f853
Reduce parallel restore atomic op test workload
2019-12-06 21:23:08 -08:00
eacfdf6da3
Add a simple workload, ReportConflictingKeysWorkload, to test correctness of the API and performance overhead added to the resovler.
2019-12-06 16:21:03 -08:00
cf2563f1c7
Mix of various things, a lot of which will change.
2019-12-05 17:10:32 -08:00
edf52e8c97
First version for reporting conflicting keys
2019-12-02 13:45:39 -08:00
545a12533a
Added redwood sequential insert unit test.
2019-12-01 23:40:59 -08:00
923a22db66
Change CMakeLists for the parallel tests
2019-11-25 10:54:56 -08:00
78b8961891
Move parallel restore tests to tests folder
...
Valgrind found errors on these two parallel restore tests,
although correctness test confirms these two tests have no correctness error.
To prevent these two parallel restore tests from spamming valgrind test results,
we exclude these two tests from our nightly tests for now.
2019-11-25 10:53:10 -08:00
dbfa3dc217
Merge pull request #2200 from negoyal/storage-cache-subfeature1
...
Storage cache subfeature1
2019-11-20 13:59:06 -08:00
4e404e34e5
Added prefix size comparison test which generates records with a configurable prefix pattern and compares storage size between Redwood and the SQLite storage engine.
2019-11-17 17:53:42 -08:00
d46c7ded59
Merge remote-tracking branch 'origin/master' into storage-cache-subfeature1
2019-11-14 17:52:22 -08:00
5144e57e11
Reenable restart tests in from_5.*
2019-11-14 14:49:51 -08:00
d147071bfd
Merge branch 'release-6.2'
...
# Conflicts:
# fdbserver/MemoryPager.actor.cpp
2019-11-12 16:31:47 -08:00
a4a0bf18f9
Merging with Master.
2019-11-12 13:01:29 -08:00
ae7e42face
Merge pull request #2313 from xumengpanda/mengxu/fastrestore-applyToDB-bugfix-PR
...
Performant restore [8/XX]: Fix bugs in applyToDB logic and add more tests
2019-11-12 08:50:23 -08:00
9227de5c20
Redwood correctness unit test was using wallclock based time limit which breaks determinism.
2019-11-11 15:13:58 -08:00
120444ab4e
Merge pull request #2334 from mpilman/features/package-tests
...
Add cmake command to package tests
2019-11-08 19:52:57 -08:00
d0d036b3a7
Add cmake command to package tests
2019-11-08 21:39:21 +00:00
fe79742881
Merge pull request #2333 from apple/release-6.2
...
Merge release 6.2 to master
2019-11-08 11:01:07 -08:00
b50d26c4da
KVStoreTest now runs sqlite and redwood tests.
2019-11-05 13:46:04 -08:00
27c7ef09a3
FastRestore:Revise code in self review
...
When we read the txnId from decodeRestoreApplierKey func,
we should convert the integer to little endian.
2019-11-03 20:43:17 -08:00
7cf87e9ae3
FastRestore:Add ParallelRestoreCorrectnessCycle.txt test
2019-11-03 17:31:54 -08:00
63359bfc8b
FastRestore:handleInitVersionBatchRequest:Ensure exact once execution
...
Also increase the test workload for BackupAndParallelRestoreWithAtomicOp test
2019-11-03 17:26:13 -08:00
ef14f7a718
Merge pull request #2292 from etschannen/master
...
Merge 6.2 into master
2019-10-25 09:18:20 -07:00
a30e6ec147
Merge pull request #2277 from xumengpanda/mengxu/fastrestore-atomicOpTest-increaseLoadAndBugFix-PR
...
Performant restore [7/XX]: Add tests for transactionBatchSizeThreshold when apply mutations
2019-10-24 21:21:14 -07:00
3325980c03
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/DataDistribution.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/WorkerInterface.actor.h
# fdbserver/worker.actor.cpp
# versions.target
2019-10-24 17:38:15 -07:00
2f34ee684f
fixed indentation issues
2019-10-24 13:21:28 -07:00
b74e5b1543
added sample file for attrition test outside of simulation
2019-10-24 13:10:59 -07:00
7af3239ee7
FastRestore:AtomicOpTest:Debug:1 key per group for ops keyspace
2019-10-23 14:36:34 -07:00
ba7e499efe
FastRestore:AtomicOpTest:Limit 1 actor per client
2019-10-23 14:04:14 -07:00
e676348710
Merge pull request #1955 from fzhjon/mark-ss-failed
...
Add fdbcli and API command to mark storage servers as permanently failed
2019-10-22 23:36:30 -07:00
96d463bab6
FastRestore:Fix bug in applying mutations and increase atomicOp test worload
...
When Applier applies mutations to the destination cluster, it advances the
mutation cursor twice when it should only advance it once.
This makes restore miss some mutations when the applying txn includes
more than 1 mutations.
2019-10-22 23:24:23 -07:00
01b4fb5e5e
CMake:Add performant restore test to CMake
2019-10-21 14:52:03 -07:00
970327b554
FastRestore:Add ParallelRestoreCorrectnessAtomicOpTinyData.txt
2019-10-21 14:42:11 -07:00
450a09e117
Code Review Changes
2019-09-24 15:48:50 -07:00
471e283128
Merge branch 'master' of https://github.com/apple/foundationdb into mark-ss-failed
2019-09-18 11:49:07 -07:00
680f0e3bd8
updated CMakeLists
2019-09-13 09:30:40 -07:00
05eac989da
unpair the cycle test with configure test for now, because tests were timing out
2019-09-13 09:20:06 -07:00
5bf8d61d81
pair configureDatabase tests with cycle tests to ensure that changing the configuration does not corrupt data
2019-09-04 18:41:22 -07:00
807b02551e
updated help message and changed existing workload to use mark as failed feature
2019-08-27 14:39:43 -07:00
10be4b8127
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-08-07 04:20:14 -07:00
da9c4e97d3
Added new pager interface, IPager2, whose write interface enables forcing the user to handle a page update causing a copy to a new Page ID. Implemented FIFOQueue<T> which uses pages of T stored in a Pager2 instance to implement a FIFO queue. Implemented COWPager, a copy-on-write Pager2 in which all page writes cause a change of Page ID. VersionedBTree, still only operating in single-version mode, now uses Pager2.
2019-08-07 02:36:33 -07:00
f453c12f0f
remove object serialization flag from TestRunner
2019-08-06 10:20:01 -07:00
438bc636d5
Rename max_machine_failures_without_losing_X to max_zone_failures_without_losing_X in status.
2019-07-30 14:02:31 -07:00
8149b5b352
Merge pull request #1413 from atn34/change-connection-file
...
Switch cluster file feature
2019-07-26 13:27:37 -07:00
9afd162e2f
remove snap v1 related code
2019-07-25 17:29:31 -07:00
a2f2ad96ff
code review comments and merge to master changes
2019-07-24 15:36:28 -07:00
c18558cf55
enable DD mode in restore based on test spec
2019-07-24 15:36:28 -07:00
24cbe313a1
Snap*Restart in ver 2 and buggify off in restore
2019-07-24 15:36:28 -07:00
9ad9bd4c1f
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-24 15:22:26 -07:00
2f9f3c184f
Merge pull request #1870 from alexmiller-apple/txnStateStore-workload
...
Add ability to bulk load data into TxnStateStore
2019-07-22 13:20:39 -07:00
4ac1a0f557
Add ability to bulk load data into TxnStateStore
...
* Changes BulkLoad workload to support a specific volume of data to load
* Changes BulkLoad and Cycle to correctly handle \xff in keyPrefix
* Adds BulkLoad to TxnStateStoreCycleTest
2019-07-19 18:01:24 -07:00
3045826e3c
Merge pull request #1819 from mpilman/flatbuffers-fixes2
...
Flatbuffers fixes2
2019-07-19 16:33:50 -07:00
c3a8ae4752
Merge pull request #1791 from fzhjon/fetch-keys-requests-priority
...
Introduce priority to fetchKeys requests from data distribution
2019-07-19 14:54:51 -07:00
1ac2d01b03
Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2
2019-07-18 09:50:08 -07:00
d5caf0c1b4
Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2
2019-07-16 14:47:40 -07:00
f3d3f80ed2
Merge branch 'master' into bugFix
2019-07-15 14:49:19 -07:00
4b0fdabae5
mark test file as IGNORE and comment out dead placeholder code
2019-07-15 09:45:16 -07:00
1a18c859c7
knobified the durability lag rate controls
2019-07-12 18:50:56 -07:00
969957e619
Merge branch 'master' into change-connection-file
2019-07-12 11:39:19 -07:00
1cf036fc9f
Remove SnapTestAttrition from ctest
...
SnapTestAttrition the test file was removed in a53bf928
2019-07-12 10:43:42 -07:00
bc29b1e78a
Use flatbuffers for testing
2019-07-11 23:03:31 -07:00
274cb40eb8
Run tests under valgrind when USE_VALGRIND
2019-07-11 23:03:31 -07:00
35be45d4ce
Remove code accidentally left for testing ctest fix
2019-07-11 10:01:26 -07:00
695ca5e8e9
moved location of DDMetricsExclude test txt file
2019-07-11 09:56:59 -07:00
f12a3909f3
renamed workloads and made code style adjustments
2019-07-11 09:56:58 -07:00
1e9d31597c
removed extra parameter from getRange, added knob to guard new changes, and adjusted style/formatting in several places
2019-07-11 09:56:58 -07:00
f707d186fe
added new priority for fetchkeys requests and adjusted ddmetrics workload to run parallel with mako
2019-07-11 09:56:58 -07:00
3b8998b836
added workload to measure data distribution speeds
2019-07-11 09:56:58 -07:00
62d7ee9424
Fix ctests so each restart test has a seed unique to the first test
...
and any other test
2019-07-11 09:56:51 -07:00
a53bf9289a
remove SnapTestAttrition because it is causing correctness errors
2019-07-09 17:13:24 -07:00
1582a2a24d
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-09 13:41:54 -07:00
1b939d5208
Merge pull request #1749 from satherton/feature-redwood
...
Update redwood storage engine to latest correctness-passing version
2019-06-28 16:22:06 -07:00
83fae6cc15
Fix ExternalWorkload not being a part of the old build/test system.
2019-06-25 21:42:35 -07:00
8f06ccb065
Increased testDuration for DifferentClustersSameRV test
...
Unlocking the second database can take a while because the local
ratekkeeper throttles reads until the updateStorage actor catches up
after advanceVersion
2019-06-25 16:56:49 -07:00
0fe6edc254
Merge pull request #1678 from mpilman/features/external-workload
...
Features/external workload
2019-06-25 13:53:19 -07:00
f1f1081202
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-06-24 20:17:49 -07:00
afb0dbcd1c
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-06-20 19:11:29 -07:00
148f922813
Revert "Add CycleTest (currently times out)"
...
This reverts commit f367fcaf25
2019-06-20 18:34:41 -07:00
1611f0385a
Remove symbolication from ctest, as a large number of backtraces times out tests.
...
HugeArenaAllocation in particular is a bit spammy of backtraces,
intentionally, and is causing testing in the CI to time out.
2019-06-19 13:37:59 -07:00
2eff2b7e21
First simple test is working (but very buggy)
2019-06-19 13:03:41 -07:00
d4b7f9b606
Fixed some cmake, compile, and IDE warnings.
2019-06-17 18:55:49 -07:00
e2b6acffce
Merge branch 'master' into add-data-distribution-metrics
2019-06-12 16:12:16 -07:00
b473a8a830
changed on-the-wire format to use serialized flatbuffers, added cycletest to workload, and fixed small bug in trace
2019-06-11 15:45:06 -07:00
f367fcaf25
Add CycleTest (currently times out)
2019-06-11 14:10:34 -07:00
02e173b601
Add changeConnectionFile method to Transaction
...
Also add tests
2019-06-11 13:58:22 -07:00
8144882d7b
Merge branch 'apple-master' into features/local-rk
2019-06-10 19:40:25 -07:00
46b77819aa
Fixed LocalRatekeeper test
2019-06-10 18:25:58 -07:00
100789b354
More bug fixes in handling upperBound changes in modified pages and worst-case delta size calculation. Normalized some formatting in debug statements. Fixed compile error on linux. Updated test specs.
2019-06-05 20:58:47 -07:00
66811b7bd2
update to latest version
2019-06-03 16:49:19 -07:00
a6bee65f11
Merge branch 'master' into add-data-distribution-metrics
2019-05-30 11:11:49 -07:00
d3a179b6f9
Multiple bug fixes
...
- wait for snapTLogFailKeys in a loop, otherwise in some race
condition it can cause a false assert
- in single region, there does not seem to be a guarantee of
tagLocalityListKey for a given DC ID, avoiding that assert for now
- to find the workers that are coordinators, looking up by primary
address is not sufficient in some cases, hence looking by both
primary and secondary address
- test make files to reflect the location of the new test cases
2019-05-28 22:07:46 -07:00
16fc7b6aaa
move SnapTests into restarting/from_6.2.0
2019-05-28 22:07:46 -07:00
c53c4fa898
reduce the snap test durations
2019-05-28 22:07:46 -07:00
4083af0b01
Avoid using trackLatest for TLog pop test cases
2019-05-28 22:07:46 -07:00
61e93a9304
Address review comments and minor fixes
2019-05-28 22:07:46 -07:00
8838ba3d3b
Split SnapTestSimpleRestart into two test cases
2019-05-28 22:07:46 -07:00
8370871e4c
stale RESTORE option related code removed
2019-05-28 22:07:46 -07:00
898bed66c1
Allow only whitelisted binary path for exec op
2019-05-28 22:07:46 -07:00
f129d996fe
Remove dumpAfterTest=true in snap tests
2019-05-28 22:07:46 -07:00
281c785f94
'--restoring' cmd line arg removed for fdbserver
...
'--restoring' command line option was introduced to indicate
simulated fdbserver to restore from snapshot and restart the cluster.
As part of this change that option is removed and restore
information is stored in the restartInfo.ini.
2019-05-28 22:07:46 -07:00
3d5998e9dd
tlog: when pops are disabled, store them & replay
...
In Tlogs, disable pop is done whlie taking snapshots. Earlier, tlogs
were ignoring the pops if it got pop requests when pops were
disabled. In this change, instead of ignoring the pop - it remembers
the list of pops in-memory and plays them once the popping is
enabled.
2019-05-28 22:07:46 -07:00
4bc4c615da
exec op to all tlog, restore change in test &other
...
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
2019-05-28 22:07:46 -07:00
a7920ef311
Merge branch 'master' of https://github.com/apple/foundationdb into MakoWorkload
2019-05-28 18:21:02 -07:00
ca034846b4
update comments, parameter names and descriptions
2019-05-28 17:29:45 -07:00
0f46b6c926
Add sql_create_report call to generate sql file
...
After all testing iterations are complete, run sql_create_report to generate the sql
file and (for now) archive it with the job.
SNOW-78373
2019-05-28 10:22:24 -07:00
31939fd044
Merge pull request #1597 from atn34/cmake-flatbuffers-off
...
Hardcode flatbuffers to off for ctest
2019-05-23 11:13:54 -07:00
b451c2cd56
Merge pull request #1497 from alexmiller-apple/fastrecovery
...
Add an \xff keyrange that is backed by the txnStateStore.
2019-05-23 10:52:35 -07:00
74a705645b
changed kv format of metrics list and added simulation test
2019-05-22 12:44:32 -07:00
57968d9df7
Merge branch 'master' of https://github.com/apple/foundationdb into MakoWorkload
2019-05-21 16:24:11 -07:00
12a51b2d39
fix bugs, update naming and comments, refine functions
2019-05-20 18:26:30 -07:00
f673be456b
Added jenkins file for Snowflake
...
We (snowflake) want to run build and simulation tests against PRs.
For this we need a jenkins file in the repository. This should not
impact anything else
2019-05-17 11:17:40 -10:00
8082362830
Hardcode flatbuffers to off for ctest
2019-05-16 13:02:06 -07:00
106bb7677d
update
2019-05-15 12:58:12 -07:00
6afce01744
Implementation complete (not yet working)
2019-05-13 14:15:22 -07:00
5678a7417e
Mako Workload
2019-05-09 15:55:05 -07:00
0a7c74eac9
Add TxnStateStoreCycleTest to CMake
2019-04-25 17:09:35 -07:00
797d431934
Add an \xff keyrange that is backed by the txnStateStore.
2019-04-25 17:04:20 -07:00
0dd57dd454
Fix a few typos
2019-04-13 17:29:30 -07:00
Xiaoxi Wang