09f3690f09
Fix OPEN_FOR_IDE build
2020-02-03 10:42:05 -08:00
b0a1af1288
Added the actual read hot detection algorithm and logging machanism.
...
- When a shard has a read bandwidth larger than a threshold value(configurable via knob), and it's read-bandwidth/byte-size ratio is also larger than a threshold(configurable via knob), the corresponding shard tracker will run the algorithm
- The algorithm will divide the shard into 10MB(configurable via knob) chunks and try to find the chunk(s) that has large aforementioned ratio
- Then those ranges will be logged into TraceEvents. This will later do more like actually cache them.
2020-01-21 11:19:52 -08:00
7708034cc9
Added the function used to determin sub-ranges within a read hot shard that has a readSize-to-byteSize ratio higher than the knob value. Alos added unit tests for that function.
2020-01-21 11:15:52 -08:00
4de8f2fdfa
Move the test to basic correctness test
2020-01-16 14:42:14 -08:00
1d6ac716a1
Merge remote-tracking branch 'origin' into add-data-distribution-metrics
2020-01-15 13:20:04 -08:00
10719200c3
A hack way to call API through getRange("\xff\xff/conflicting_keys\<start_key>", "\xff\xff/conflicting_keys\<end_key>").
2020-01-06 11:22:11 -08:00
21390c493a
Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
...
Resolved merge by keeping new test file from master branch: SampleNoSimAttrition.txt adding new constraint from Release branch about existing test file: SimpleExternalTest.txt
# Conflicts:
# tests/CMakeLists.txt
2019-12-18 09:05:08 -08:00
650be617f1
FastRestore:Add tests to CMakefile
2019-12-12 10:32:13 -08:00
c9994422a9
Ignore instead of omit
2019-12-12 09:12:34 -08:00
960e87ee8c
Disable SimpleExternalTest for UBSAN temporarily
2019-12-12 09:12:34 -08:00
a6cfad6c3f
FastRestore: Add ParallelRestoreCorrectnessAtomicOpTinyData test back
2019-12-11 16:39:17 -08:00
eacfdf6da3
Add a simple workload, ReportConflictingKeysWorkload, to test correctness of the API and performance overhead added to the resovler.
2019-12-06 16:21:03 -08:00
545a12533a
Added redwood sequential insert unit test.
2019-12-01 23:40:59 -08:00
923a22db66
Change CMakeLists for the parallel tests
2019-11-25 10:54:56 -08:00
4e404e34e5
Added prefix size comparison test which generates records with a configurable prefix pattern and compares storage size between Redwood and the SQLite storage engine.
2019-11-17 17:53:42 -08:00
5144e57e11
Reenable restart tests in from_5.*
2019-11-14 14:49:51 -08:00
d147071bfd
Merge branch 'release-6.2'
...
# Conflicts:
# fdbserver/MemoryPager.actor.cpp
2019-11-12 16:31:47 -08:00
ae7e42face
Merge pull request #2313 from xumengpanda/mengxu/fastrestore-applyToDB-bugfix-PR
...
Performant restore [8/XX]: Fix bugs in applyToDB logic and add more tests
2019-11-12 08:50:23 -08:00
9227de5c20
Redwood correctness unit test was using wallclock based time limit which breaks determinism.
2019-11-11 15:13:58 -08:00
d0d036b3a7
Add cmake command to package tests
2019-11-08 21:39:21 +00:00
27c7ef09a3
FastRestore:Revise code in self review
...
When we read the txnId from decodeRestoreApplierKey func,
we should convert the integer to little endian.
2019-11-03 20:43:17 -08:00
ef14f7a718
Merge pull request #2292 from etschannen/master
...
Merge 6.2 into master
2019-10-25 09:18:20 -07:00
3325980c03
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/DataDistribution.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/WorkerInterface.actor.h
# fdbserver/worker.actor.cpp
# versions.target
2019-10-24 17:38:15 -07:00
b74e5b1543
added sample file for attrition test outside of simulation
2019-10-24 13:10:59 -07:00
01b4fb5e5e
CMake:Add performant restore test to CMake
2019-10-21 14:52:03 -07:00
680f0e3bd8
updated CMakeLists
2019-09-13 09:30:40 -07:00
10be4b8127
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-08-07 04:20:14 -07:00
da9c4e97d3
Added new pager interface, IPager2, whose write interface enables forcing the user to handle a page update causing a copy to a new Page ID. Implemented FIFOQueue<T> which uses pages of T stored in a Pager2 instance to implement a FIFO queue. Implemented COWPager, a copy-on-write Pager2 in which all page writes cause a change of Page ID. VersionedBTree, still only operating in single-version mode, now uses Pager2.
2019-08-07 02:36:33 -07:00
8149b5b352
Merge pull request #1413 from atn34/change-connection-file
...
Switch cluster file feature
2019-07-26 13:27:37 -07:00
9afd162e2f
remove snap v1 related code
2019-07-25 17:29:31 -07:00
a2f2ad96ff
code review comments and merge to master changes
2019-07-24 15:36:28 -07:00
9ad9bd4c1f
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-24 15:22:26 -07:00
c3a8ae4752
Merge pull request #1791 from fzhjon/fetch-keys-requests-priority
...
Introduce priority to fetchKeys requests from data distribution
2019-07-19 14:54:51 -07:00
4b0fdabae5
mark test file as IGNORE and comment out dead placeholder code
2019-07-15 09:45:16 -07:00
1cf036fc9f
Remove SnapTestAttrition from ctest
...
SnapTestAttrition the test file was removed in a53bf928
2019-07-12 10:43:42 -07:00
695ca5e8e9
moved location of DDMetricsExclude test txt file
2019-07-11 09:56:59 -07:00
f12a3909f3
renamed workloads and made code style adjustments
2019-07-11 09:56:58 -07:00
3b8998b836
added workload to measure data distribution speeds
2019-07-11 09:56:58 -07:00
1582a2a24d
Merge branch 'master' of https://github.com/apple/foundationdb into change-connection-file
2019-07-09 13:41:54 -07:00
1b939d5208
Merge pull request #1749 from satherton/feature-redwood
...
Update redwood storage engine to latest correctness-passing version
2019-06-28 16:22:06 -07:00
83fae6cc15
Fix ExternalWorkload not being a part of the old build/test system.
2019-06-25 21:42:35 -07:00
f1f1081202
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-06-24 20:17:49 -07:00
2eff2b7e21
First simple test is working (but very buggy)
2019-06-19 13:03:41 -07:00
d4b7f9b606
Fixed some cmake, compile, and IDE warnings.
2019-06-17 18:55:49 -07:00
e2b6acffce
Merge branch 'master' into add-data-distribution-metrics
2019-06-12 16:12:16 -07:00
02e173b601
Add changeConnectionFile method to Transaction
...
Also add tests
2019-06-11 13:58:22 -07:00
8144882d7b
Merge branch 'apple-master' into features/local-rk
2019-06-10 19:40:25 -07:00
66811b7bd2
update to latest version
2019-06-03 16:49:19 -07:00
a6bee65f11
Merge branch 'master' into add-data-distribution-metrics
2019-05-30 11:11:49 -07:00
d3a179b6f9
Multiple bug fixes
...
- wait for snapTLogFailKeys in a loop, otherwise in some race
condition it can cause a false assert
- in single region, there does not seem to be a guarantee of
tagLocalityListKey for a given DC ID, avoiding that assert for now
- to find the workers that are coordinators, looking up by primary
address is not sufficient in some cases, hence looking by both
primary and secondary address
- test make files to reflect the location of the new test cases
2019-05-28 22:07:46 -07:00
8838ba3d3b
Split SnapTestSimpleRestart into two test cases
2019-05-28 22:07:46 -07:00
281c785f94
'--restoring' cmd line arg removed for fdbserver
...
'--restoring' command line option was introduced to indicate
simulated fdbserver to restore from snapshot and restart the cluster.
As part of this change that option is removed and restore
information is stored in the restartInfo.ini.
2019-05-28 22:07:46 -07:00
3d5998e9dd
tlog: when pops are disabled, store them & replay
...
In Tlogs, disable pop is done whlie taking snapshots. Earlier, tlogs
were ignoring the pops if it got pop requests when pops were
disabled. In this change, instead of ignoring the pop - it remembers
the list of pops in-memory and plays them once the popping is
enabled.
2019-05-28 22:07:46 -07:00
4bc4c615da
exec op to all tlog, restore change in test &other
...
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
2019-05-28 22:07:46 -07:00
a7920ef311
Merge branch 'master' of https://github.com/apple/foundationdb into MakoWorkload
2019-05-28 18:21:02 -07:00
74a705645b
changed kv format of metrics list and added simulation test
2019-05-22 12:44:32 -07:00
5678a7417e
Mako Workload
2019-05-09 15:55:05 -07:00
0a7c74eac9
Add TxnStateStoreCycleTest to CMake
2019-04-25 17:09:35 -07:00
0dd57dd454
Fix a few typos
2019-04-13 17:29:30 -07:00
bdba8e22eb
Added test and bugfixes
2019-04-08 11:05:29 -07:00
d2bde4e55b
Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics
2019-02-27 16:30:25 -08:00
8afb7fbb9d
Merge pull request #1160 from alexmiller-apple/tstlog-fork
...
Spill-By-Reference TLog Part 2: New and Old TLogServers co-exist harmoniously
2019-02-26 18:00:04 -08:00
abfe057805
Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics
2019-02-25 13:47:16 -08:00
b8910ba7cd
Merge branch 'master' into feature-fix-force-recovery
...
# Conflicts:
# fdbclient/ManagementAPI.actor.h
# fdbserver/DataDistribution.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KillRegion.actor.cpp
2019-02-22 14:38:13 -08:00
6d23eb2d1a
Implement log_version.
...
This mega-commit introduces a new configuration setting, `log_version`,
that controls the TLog implementations and features that are available
within FDB, so that users can opt in to new features if they're willing
to sacrifice backwards compatibility.
2019-02-22 12:15:23 -08:00
2a20ea6ac5
Merge branch 'master' of https://github.com/apple/foundationdb into add-health-metrics
2019-02-22 09:41:47 -08:00
1292f5c59a
Add TEST_LOG_FORMAT to cmake
2019-02-21 14:36:01 -08:00
8ed89fd711
fixed review comments
2019-02-19 11:26:53 -08:00
a09afe5906
Added Throttling workload to test native health metrics API
2019-02-04 13:04:25 -08:00
f5e5f02a7f
Update tests/CMakeLists.txt
...
Co-Authored-By: mpilman <markus@pilman.ch>
2019-01-24 13:16:06 -08:00
83a9d5790f
Update tests/CMakeLists.txt
...
Co-Authored-By: mpilman <markus@pilman.ch>
2019-01-24 13:15:59 -08:00
5b8c98fd22
Update tests/CMakeLists.txt
...
Co-Authored-By: mpilman <markus@pilman.ch>
2019-01-24 13:15:42 -08:00
a50f2d953a
Update tests/CMakeLists.txt
...
set buggify to on by default
Co-Authored-By: mpilman <markus@pilman.ch>
2019-01-24 13:15:14 -08:00
58964af7e1
ctest improvements - #1058
...
- A set of CMake variables controls whether to keep
the simfdb directory and the traces and whether we
want to aggregate the traces into a single file
- Test labels now contain the directory they are in
so that one can now run `ctest -R fast/`
- A different binary can be used for restart tests. CMake
will automatically look for an installed fdb and use that
by default. If none is found, it will use the built one
but it will also print a warning
- CMake will throw an error if there are any text files in
the tests directory that are not associated with a test.
- Moved testing from fdbserver/CMakeLists.txt to
tests/CMakeLists.txt
- Moved fdb testing functions to its own cmake module
2019-01-22 14:34:51 -08:00
Andrew Noyes