f14e08a991
addRocksDBPerfContextMetrics
2022-02-23 22:29:07 -05:00
96dd86ebf8
update RocskDB and Boost
...
add Finduring, and include into fdbserver
add BOOST asio/uring settings to fdbserver compile
move portable rocks, liburing up to be configurable at build time.
2022-02-09 10:48:18 -06:00
c605226a56
Add error handling in RocksDB KVS. ( #6277 )
2022-01-25 13:59:25 -08:00
162bce7a58
Rocksdb write rate limiter.
2022-01-18 13:23:00 -08:00
ef4038fe8d
Rocksdb read range iterator pool to reuse iterators.
2022-01-18 02:05:21 -08:00
1c2c2783ea
Implement single deletion in RocksDB.
...
If the target deletion range contains a single key, convert it into a
single deletion, to avoid extra cost in RocksDB due to range deletion.
2022-01-04 09:59:40 -08:00
1f30368e71
KeyValueStoreRocksDB histograms to track latencies
2021-12-21 23:09:46 -08:00
d23b8645f8
Enabling rocksdb metrics logger in simulation.
2021-12-07 15:18:29 -08:00
9dccb0131e
Clean up RocksDB error logging
2021-11-12 12:14:12 -05:00
394b9dc619
Code review changes
2021-11-10 11:53:27 -05:00
66520eb1c1
Utilize read types to do selective throttling
2021-11-10 11:51:04 -05:00
8822c589de
Move table factory options to CFOptions
2021-11-09 17:29:06 -05:00
421dee532c
Add const qualifiers in KeyValueStoreRocksDB.actor.cpp
2021-10-18 13:40:47 -07:00
9713a14ef1
Reverse order of read type and debug ID args
2021-10-18 12:23:09 -04:00
df53cc9580
Add an enum to IKeyValueStore to indicate the source/priority of the read
2021-10-15 14:35:59 -04:00
3c35bc69d7
Decoupling the rocksdb version check to multiple static asserts.
2021-09-17 08:53:23 -07:00
c0e6b09fe3
Enforcing 6.22.1 rocksdb version and enabling readRange deadline option.
2021-09-17 08:53:23 -07:00
627ef2e739
Enable rocksdb storage in simulation with some simulation fixes.
2021-09-10 11:11:29 -07:00
f7842ff3db
Log SubCode and Severity for RocksDB erros ( #5515 )
2021-08-31 21:03:02 -07:00
b75fb0d105
Improve error to status conversion
2021-08-23 11:35:26 -04:00
473e2bbf5f
Fix error handling of reads.
2021-08-23 11:35:26 -04:00
0fbfcc582e
Increasing the read cancellation timeouts to 5 minutes in simulation.
2021-08-19 14:47:12 -07:00
6c97782378
Timedout reads cancellation check removed in simulation.
2021-08-19 14:11:28 -07:00
02b3ed3ff1
Adding deadline option to rocksdb calls.
2021-08-19 14:11:28 -07:00
fbb393f998
Added readrange timeoout check and rocksdb read deadline option.
2021-08-19 14:11:28 -07:00
24ac173c95
Adding ReadRangeAction timeout, returning error and using timer_monotonic changes.
2021-08-19 14:11:28 -07:00
01e85610ab
Cancelling the timedout reads with rocksdb storage.
2021-08-19 14:11:28 -07:00
b8e2575f92
Update log directory to server log directory. ( #5396 )
2021-08-18 13:07:56 -07:00
adc466acb6
Fix key name
2021-07-30 13:27:43 -04:00
a26dbba66f
Add RocksDB metrics
2021-07-30 13:27:43 -04:00
0f99d449fc
Revert "Merge pull request #5194 from apple/revert-5144-rocksdb-in-simulation"
...
This reverts commit 8ecff8c730c8e18f99f0
2021-07-16 12:01:01 -04:00
b7b845dab8
Revert "RocksDB in simulation"
2021-07-15 16:41:33 -07:00
ed35df9a00
Add comment explaining coro threads
2021-07-13 14:56:23 -04:00
0613ff892f
Enable RocksDB in simulation
2021-07-13 12:25:11 -04:00
1feedb700b
Don't log key not found as an error
2021-07-09 13:07:06 -04:00
efdda3cd0e
Revert "Revert "Add support for running non-simulation tests on Joshua""
2021-06-15 14:15:02 -04:00
549cf0512b
Revert "Add support for running non-simulation tests on Joshua"
2021-06-14 19:23:56 -04:00
8af0affeff
Add support for running non-simulation tests on Joshua
2021-06-14 15:00:45 -04:00
25c3468a08
Estimate live SST file size and not live data size
2021-06-11 16:53:10 -04:00
5c2d7b6080
Create RangeResult type alias
2021-05-03 13:14:16 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
2bb4f2e59f
Merge branch 'release-6.3-pre-format' into master-format
...
This merges release-6.3 branch right before it was fully formatted.
There were quite a few conflicts that are resolved here. CoroFlow had
a check for OOM errors introduced in 6.3, but didn't seem applicable in
the new implmentation which seems to use boost.
2021-03-10 09:37:41 -08:00
d2d6fcccfa
Merge pull request #4009 from Daniel-B-Smith/rocksdb-named-threads
...
Name the RocksDB background threads
2021-03-03 20:34:32 -08:00
346a4e3ecd
Merge branch 'release-6.3'
...
# Conflicts:
# fdbcli/fdbcli.actor.cpp
# fdbrpc/LoadBalance.actor.h
# fdbrpc/MultiInterface.h
# fdbserver/ClusterController.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/masterserver.actor.cpp
2021-03-01 18:52:06 -08:00
179dea5a1b
Name the RocksDB background threads
2021-03-01 20:35:55 +00:00
0e9e53a163
Make the RocksDB init method idempotent
2021-03-01 17:20:41 +00:00
a8d08a8792
Merge branch 'release-6.3' into merge-release-6.3-into-master
2021-02-09 10:18:00 -08:00
aaf0a9aa7b
Merge branch 'release-6.3' into merge-release-6.3-into-master
...
# Conflicts:
# build/docker-compose.yaml
# cmake/ConfigureCompiler.cmake
# fdbclient/FileBackupAgent.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/IAsyncFile.h
# fdbrpc/IRateControl.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
4c5079e4f6
Add option to prevent synchronous file delets on reads.
2021-02-03 21:57:55 +00:00
de700b83f2
Review fixes
2021-02-01 21:58:39 +00:00
ecf5c2b591
Add knobs for prefix bloom filters and larger block cache
2021-01-11 21:38:34 +00:00
7fbc4d7391
Resolve conflicts
2020-12-04 23:58:42 +00:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
7c21df59fb
Attempt to suggest compactions after commit finishes
2020-11-02 21:24:01 +00:00
d7d9c31536
Revert "Add some thread yields"
...
This reverts commit 3485f1addd
2020-10-26 19:29:36 +00:00
c859d75bd5
Add some trace events for opening / closing database
2020-10-22 20:58:55 +00:00
06b0bb70d1
Don't close a null DB and report errors from destroying the DB
2020-10-22 16:58:11 +00:00
410d418c3e
Revert "Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining"
...
This reverts commit e258dffc650e96233015
2020-10-19 09:28:38 -07:00
a99b68a7e3
Add knob for tuning number of RocksDB read threads
2020-10-14 22:07:46 +00:00
2671157f8f
Merge branch 'rocksdb-data-estimate' into rocksdb-unsafe-fsync
2020-10-09 16:56:54 +00:00
6e287eb0d1
Merge remote-tracking branch 'upstream/release-6.3' into rocksdb-unsafe-fsync
2020-10-09 16:53:05 +00:00
dc4612712d
Merge remote-tracking branch 'origin/master' into use-override-more
2020-10-09 07:12:41 -07:00
8571dcfe28
Use override where applicable in fdbserver
2020-10-07 18:41:19 -07:00
34d1d04904
Commit pipelining attempt #2
2020-09-23 20:03:04 -07:00
62fb44e3b9
Use the live data estimate instead of total data
2020-09-09 19:30:13 +00:00
e1b7dd0c7d
Merge remote-tracking branch 'upstream/release-6.3' into dyoungworth/fixMerge1
2020-08-22 12:25:19 -07:00
366bb1bd33
Remove slow range read logging
2020-08-21 17:37:47 +00:00
94571786bf
Add RocksDB work
2020-08-14 17:16:20 +00:00
c2e49b2dfa
Revert "Revert "Add some thread yields""
...
This reverts commit 97165dd278
2020-08-13 15:12:54 +00:00
d3ad73906f
Add a compact on deletion collector to tables
2020-08-12 20:58:12 +00:00
16582f9d7b
Refactor to avoid relying on object slicing
2020-08-12 20:50:32 +00:00
97165dd278
Revert "Add some thread yields"
...
This reverts commit 3485f1addd
2020-08-11 18:49:51 +00:00
8c20124612
Short circuit reads
2020-08-11 18:49:40 +00:00
9065fb3127
Add knob for periodic compactions
2020-08-11 18:00:01 +00:00
1245143ec1
Enable per-level stats and enforce iterator upper/lower bounds
2020-08-10 19:47:26 +00:00
7c2e45d5a6
Add some RocksDB stats
2020-08-07 18:48:19 +00:00
fe5902994c
Merge pull request #3605 from apple/release-6.3
...
Merge Release 6.3 to master
2020-08-05 23:37:44 -07:00
3485f1addd
Add some thread yields
2020-08-05 21:45:48 +00:00
50b1649f8d
Add more data to the slow query log
2020-08-03 19:02:29 +00:00
8ac89fc37f
Log trace events for slow range reads
2020-07-31 16:33:42 +00:00
a94c4cce85
Add an unsafe option to disable manual fsyncing rocksdb
2020-07-30 22:31:18 +00:00
abd2e6b979
Add some knobs for tuning and lz4 compaction
2020-07-30 15:42:26 +00:00
6efd5996c2
Calculate data stored by Rocks
2020-07-29 22:08:32 +00:00
fd973e5055
RocksDB fixes
2020-07-29 21:26:36 +00:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
69139e3593
Remove unneeded option
2020-06-24 20:30:18 +00:00
dd42d11f1b
Revert "Revert "Re-use RocksDB iterators for scans""
...
This reverts commit 00102dc6f8
2020-06-24 16:01:00 +00:00
00102dc6f8
Revert "Re-use RocksDB iterators for scans"
...
This reverts commit 90060ac29f
2020-06-18 21:31:16 +00:00
dff1209672
Delete some dead code and clean up includes
2020-06-18 18:58:18 +00:00
9286122544
Increase read concurrency
2020-06-18 18:42:45 +00:00
90060ac29f
Re-use RocksDB iterators for scans
2020-06-18 18:01:21 +00:00
01e7b5356e
Use SeekForPrev to simplify reverse iteration logic
2020-06-18 17:54:19 +00:00
318f290365
Delete files on background thread.
2020-06-18 17:46:21 +00:00
d32e97e4c4
Added const to RocksDBKeyValueStore methods to support new IKeyValueStore interface
2020-06-17 18:20:43 -07:00
2a9fd611c5
Don't log ok status on DB close
2020-06-15 16:51:42 +00:00
acbfe2e4c9
Revert "Revert "Initial RocksDB""
2020-06-15 12:45:36 -04:00
9cd1614c82
Revert "Initial RocksDB"
2020-06-11 15:29:46 -07:00
346749d73c
Implement deleting the data
2020-06-11 18:21:09 +00:00
c3178d829d
No longer double delete the DB
2020-06-11 16:12:29 +00:00
35372cbe38
Don't use global TraceBatch object
2020-06-11 15:28:36 +00:00
d310a45cb4
Crash when trying to set up the rocks engine without building rocks
2020-06-10 20:19:06 +00:00
9c9eebe9ff
Improve cleanup
2020-06-05 18:30:19 +00:00
6893e7bfbe
Refresh RocksDB iterators instead of allocating new ones
2020-06-04 15:55:45 +00:00
98013e87b3
No longer use a tailing iterator
2020-06-03 21:11:57 +00:00
46b7803622
Fix reverse iteration seek behavior
2020-06-03 21:02:44 +00:00
455cb41283
Add a warning if attempting to use RocksDB without it built
2020-06-03 21:02:09 +00:00
4b402b213a
Enable FDB to be built without RocksDB
2020-06-03 20:32:39 +00:00
830ddb3b5f
Don't re-use cursors
2020-06-02 22:54:52 +00:00
62f2eb7875
Further reverse iteration work
2020-06-02 22:32:05 +00:00
8c5b40a92a
Implement reverse range reads
2020-06-02 20:32:02 +00:00
c2e8029837
Don't spam logs with missing keys
2020-05-28 21:19:12 +00:00
f2bf42e7ac
Cowardly refusing to write and empty batch
2020-05-28 21:12:16 +00:00
6386563cee
Convert to buffering writes in the storage server thread
2020-05-27 19:59:51 +00:00
b6dd425c1c
RocksDB is kind of working!
2020-05-22 21:25:40 +00:00
5d361fe532
Copy/paste rebase onto 6.3
2020-05-22 15:02:51 +00:00
Zhe Wang