Commit Graph

84 Commits

Author SHA1 Message Date
Steve Atherton 557efa1b53
Merge pull request #7406 from sfc-gh-fzhao/RedwoodDataMovementNonCacheRead
Redwood add non-cache reads options
2022-08-30 17:25:05 -07:00
Fuheng Zhao 7c6dbaf3cf update trState ReadOption to optional 2022-08-24 15:04:17 -07:00
Nim Wijetunga a857609478 refactor ekp interface 2022-08-23 23:04:12 -07:00
Fuheng Zhao 4e748d6bed use optional ReadOptions and RangeReadOptions in requests 2022-08-23 17:16:47 -07:00
Fuheng Zhao 78f4b4f739 add RangReadOptions which inherit from ReadOptions 2022-08-22 11:46:01 -07:00
Fuheng Zhao d24a52952c update readOption struct 2022-08-10 10:16:26 -07:00
Fuheng Zhao 1a5b8fbd30 resolve conflicts 2022-08-09 09:57:11 -07:00
Fuheng Zhao e4fb565057 use readOptions to pass type and cacheResult 2022-08-08 15:30:56 -07:00
Markus Pilman 1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
Yi Wu 7d7ce0909f
Restart tests carry forward encryption knobs value (#7497)
Previously to get around the issue that EKP is not present when restart test switching encryption from on to off and read encrypted data, EKP was made to start in simulation regardless of encryption knob. This PR revert that change, and instead force restart test not to change encryption knob, by passing previous encryption knob through restartInfo.ini file. Also since we don't allow downgrading an encrypted cluster to previous version, disable encryption in downgrade tests.

Also adding an assert to allow reading encrypted mutations only if encryption knob is on. We may reconsider allowing switching encryption on/off for existing cluster, but for now we don't allow it.
2022-07-14 14:45:17 -07:00
Andrew Noyes 8c1813e377 Don't pass nullptr to memcpy 2022-07-07 10:19:20 -07:00
Yi Wu 6246664006
Support encrypting TxnStateStore (#7253)
Adding encryption support for TxnStateStore. It is done by supporting encryption. for KeyValueStoreMemory. The encryption is currently done on operation level when the operations are being write to the underlying log file. See inline comment for the encrypted data format.

This PR depends on #7252. It is part of the effort to support TLog encryption #6942.
2022-06-14 13:26:32 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Yi Wu 607b2a0184 update comment 2022-02-07 13:32:52 -08:00
Yi Wu b32c843522 Revert "use DiskQueueVersion::V2 for KeyValueStoreMemory"
This reverts commit ba83d73acd.
2022-02-07 13:32:52 -08:00
Yi Wu eae3dab04d use DiskQueueVersion::V2 for KeyValueStoreMemory 2022-02-07 13:32:52 -08:00
Yi Wu cda68a0e4d Support xxhash3 for checksuming DiskQueue for TLogs 2022-02-07 13:32:52 -08:00
sfc-gh-tclinkenbeard 9e06b6e6e3 Make IClosable interface const-correct 2021-10-18 13:40:47 -07:00
Daniel Smith 9713a14ef1 Reverse order of read type and debug ID args 2021-10-18 12:23:09 -04:00
Daniel Smith df53cc9580 Add an enum to IKeyValueStore to indicate the source/priority of the read 2021-10-15 14:35:59 -04:00
sfc-gh-tclinkenbeard c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Steve Atherton 507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
sfc-gh-tclinkenbeard b9a22a61ef Fix many -Wreorder-ctor warnings 2021-07-23 17:33:18 -07:00
Daniel Smith 8efe3b296a Delete remaining extern declarations for noUnseed 2021-07-08 19:19:22 -04:00
sfc-gh-tclinkenbeard 41c790b299 Merge remote-tracking branch 'origin/master' into config-db 2021-06-10 22:31:23 -07:00
sfc-gh-tclinkenbeard cfc4545135 Use g_knobs for SERVER_KNOBS and CLIENT_KNOBS 2021-06-07 14:32:51 -07:00
sfc-gh-tclinkenbeard f28ac955c3 Remove unnecessary temporary objects while growing objects of type std::vector<std::pair<A, B>> 2021-05-10 16:32:50 -07:00
sfc-gh-tclinkenbeard 5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes 4ee97c0784 Use clang-tidy to automatically fix missing overrides
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.

Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.

- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
sfc-gh-tclinkenbeard 33ec968d5a Mark expectedSize methods const 2020-12-26 18:30:44 -04:00
sfc-gh-tclinkenbeard fff8e34b4d Move IKeyValueContainer from flow to fdbserver 2020-11-15 12:23:08 -08:00
sfc-gh-tclinkenbeard 575b36bf53 Move RadixTree from flow to fdbserver 2020-11-15 11:57:54 -08:00
Xin Dong 410d418c3e Revert "Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining"
This reverts commit e258dffc65, reversing
changes made to 0e96233015.

SS commit seems to be causing write performance degredation.
2020-10-19 09:28:38 -07:00
sfc-gh-tclinkenbeard dc4612712d Merge remote-tracking branch 'origin/master' into use-override-more 2020-10-09 07:12:41 -07:00
sfc-gh-tclinkenbeard ccf2ca73f3 Fix Windows CMake build 2020-10-08 23:58:58 -07:00
Steve Atherton e258dffc65
Merge pull request #3770 from dongxinEric/feature/ss-commit-pipelining
Pipeline the commit process in SS.
2020-10-07 20:34:57 -07:00
sfc-gh-tclinkenbeard 8571dcfe28 Use override where applicable in fdbserver 2020-10-07 18:41:19 -07:00
Xin Dong 34d1d04904 Commit pipelining attempt #2 2020-09-23 20:03:04 -07:00
sfc-gh-tclinkenbeard 0814841827 Replace NULL with nullptr in fdbserver 2020-09-20 11:31:49 -07:00
sfc-gh-tclinkenbeard 77e3b314f6 Mark IKeyValueStore::getStorageBytes const 2020-06-17 18:20:43 -07:00
sfc-gh-tclinkenbeard 2a393633b6 Improved const-correctness
Using the recently added IndexedSet::const_iterator
(https://github.com/apple/foundationdb/pull/3185), we can improve the
const-correctness of many functions. In this PR const is added where
applicable. Also, wherever I came across the following while adding
const, I made the following changes:

- virtual function overrides are marked as override
- NULL is replaced with nullptr
- git clang-format is applied
2020-06-17 18:20:43 -07:00
Alex Miller 283fd3af27 Add a knob which controls writing prefix compressed kvs mem snapshots.
Which will be set to on by default in 7.0
2020-05-12 17:01:52 -07:00
Steve Atherton fd08f888ea Added comments about when snapshot deltas can't be used, and fixed some whitespace oddities. 2020-04-04 03:19:01 -07:00
Steve Atherton eca107cb18 Bug fixes regarding when snapshot item deltas are okay, and added sanity checks on delta lengths. Also, rewrote the snapshot progress actor to advance its iterator to the next KV pair rather than doing an upperbound search of the previous key to find the next item. 2020-04-04 02:27:36 -07:00
Steve Atherton 5b18e56d11 KeyValueStoreMemory now uses key delta compression in partial snapshots. This is done with a new log opcode, OpSnapshotItemDelta, so old files can still be read but new files would not be readable by the old code. 2020-04-03 23:54:28 -07:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
A.J. Beamon df2b0452b4 Step 3 of fixing storage server range reads: change return type of readRange from VectorRef<KeyValueRef> to RangeResultRef. 2020-02-06 13:19:24 -08:00
mengranwo 115ff8bf65 change getKey() interface, pass in uint8_t * only 2020-01-15 13:49:45 -08:00