Commit Graph

125 Commits

Author SHA1 Message Date
Andrew Noyes 7a9217a392
Add contrib/debug_determinism (#6389)
* Add contrib/debug_determinism

Add an instrumentation-based technique for debugging unseen mismatches. Also guard a few existing sources of nondeterminism that don't affect unseen with the DEBUG_DETERMINISM macro.

Also change the simulated run loop to not run as the only task inside the real run loop, since that was a source of nondeterminism.

Also fix nondeterminism from calling timer_int

* Add StorageMetadataType::currentTime

Basically a deterministic-in-simulation version of timer_int that we can
use instead of timer_int for StorageMetadataType::createdTime
2022-02-25 12:54:31 -08:00
A.J. Beamon 250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Xiaoxi Wang 6dc5921575
createdTime based storage wiggler (#6219)
* add storagemetadata

* add StorageWiggler;

* fix serverMetadataKey bug

* add metadata tracker in storage tracker

* finish StorageWiggler

* update next storage ID

* change pid to server id

* write metadata when seed SS

* add status json fields

* remove pid based ppw iteration

* fix time expression

* fix tss metadata nonexistence; fix transaction retry when retrieving metadata

* fix checkMetadata bug when store type is wrong

* fix remove storage status json

* format code

* refactor updateNextWigglingStoragePID

* seperate storage metadata tracker and store type tracker

* rename pid

* wiggler stats

* fix completion between waitServerListChange and storageRecruiter

* solve review comments

* rename system key

* fix database lock timeout by adding lock_aware

* format code

* status json

* resolve code format/naming comments

* delete expireNow; change PerpetualStorageWiggleID's value to KeyBackedObjectMap<UID, StorageWiggleValue>

* fix omit start rount

* format code

* status json reset

* solve status json format

* improve status json latency; replace binarywriter/reader to objectwriter/reader; refactor storagewigglerstats transactions

* status timestamp
2022-02-04 15:04:30 -08:00
A.J. Beamon d8e161f89e Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state. 2021-12-17 11:57:39 -08:00
He Liu 9c835a8e17 merge fix 2021-09-22 16:55:49 -07:00
He Liu a1f6dcc2d5 merge from master 2021-09-22 13:07:38 -07:00
He Liu 95f64f97de resolved some fmt issues 2021-09-20 11:58:30 -07:00
He Liu b2afa6e2f5 throw an error is no team can be found for a dropped range 2021-09-20 11:53:27 -07:00
He Liu 4d5bf08da8 address comments 2021-09-19 17:03:06 -07:00
Xiaoge Su abf73047ca Enforce std:: specifier rather than using namespace 2021-09-16 19:40:28 -07:00
He Liu ef7fdc0781 fmt 2021-09-15 10:32:09 -07:00
He Liu c8a3413820 exclude to-be-dropped server from the random team 2021-09-15 09:07:50 -07:00
helium e838d63011 exclude to-be-removed server from target team 2021-09-14 20:57:58 -07:00
helium fd6d088945 choose team before removing server 2021-09-14 19:24:59 -07:00
helium 7e53f8662d added comments 2021-09-13 13:28:55 -07:00
helium 65e6f26f05 bug fix on unwaited actor 2021-09-09 20:43:29 -07:00
helium 2ea055371b Introduced serverKeysTrueEmptyRange to indicate the SS that the new
shard is empty.
2021-09-09 14:15:07 -07:00
helium 8e0b572a18 Added comments. 2021-09-02 22:29:07 -07:00
helium 7e7f9372de Replaced clear() with krmSetRangeCoalescing(). 2021-09-02 21:58:05 -07:00
helium a8a6f6b8a3 Remove Transaction::set() when clearing the keyrange. 2021-09-01 15:35:40 -07:00
helium 6612cc00b6 Check if the src server list will be empty before removing a failed server." 2021-09-01 14:52:07 -07:00
Evan Tschannen 0a78c84a3e
Merge pull request #5183 from sfc-gh-xwang/tlog_dev
TLog Streaming Peek
2021-08-17 11:51:34 -07:00
Josh Slocum e444d3781c Various TSS improvements from snowblower testing 2021-08-13 10:24:15 -05:00
Xiaoxi Wang 2a88033800 clean 100k simulation test. revert changes of fdbrpc.h 2021-07-31 16:46:14 -07:00
Steve Atherton 507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
Steve Atherton f596a81073 Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs. 2021-07-17 00:11:40 -07:00
sfc-gh-tclinkenbeard 8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
Neethu Haneesha Bingi decfb610ff Minor review comments. 2021-06-25 15:04:49 -07:00
Neethu Haneesha Bingi 66f2518405 exclude to work with any locality data match. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 4ad5926a25 Snake naming of keys and added comments to all new functions. 2021-06-23 18:03:27 -07:00
Neethu Haneesha Bingi 73752f441b exclude locality:clang-format, ranged loops, documentation, tracking addStoragesever for exclusion. 2021-06-23 18:03:27 -07:00
Josh Slocum db04b65ae1 Adding comments and a quarantine case I forgot 2021-06-11 10:42:33 -05:00
Josh Slocum ac209b32fd Addressing review comments 2021-06-03 15:31:16 +00:00
Josh Slocum b3e4f182ef TSS Mapping Change 2021-06-02 17:30:09 +00:00
Josh Slocum 4257ac2b4d More TSS Changes/Fixes 2021-05-25 20:37:48 +00:00
Josh Slocum ce82c9653e Testing Storage Server implementation 2021-05-25 20:28:50 +00:00
sfc-gh-tclinkenbeard 5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
Jon Fu b246e673bc Added comment to seedShardServers (taken from existing desc in .h file) 2021-03-24 15:34:19 -04:00
Jon Fu 29c626ca6a Changed code flow to fix loophole that avoided the knob guarding higher protocol versions and also added new restarting tests 2021-03-15 17:47:23 -04:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes 79cec09255 Apply clang-tidy's performance-inefficient-vector-operation fix
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.

    $ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
Andrew Noyes 08997d8602 Fix bugs turned up by _GLIBCXX_DEBUG
Compiling with -D_GLIBCXX_DEBUG enables libstc++ "debug mode", where
additional debug information is tracked with iterators and reported if
iterators are misused. This turned up two bugs.

I threw in removing dead code and avoiding an unnecessary map lookup
while I was in the neighborhood.
2021-02-04 02:37:51 +00:00
sfc-gh-tclinkenbeard 652d753daf Remove global ddEnabled flag 2020-10-17 11:23:52 -07:00
Evan Tschannen ba3e2af473 Merge commit '5288033bcfe40c3ade97c8bf2d04cf31b3f16cb1' into feature-tree-broadcast 2020-04-17 15:17:37 -07:00
Alex Miller 1439de37b5 Convert GetRangeLimits() -> TOO_MANY + ASSERT(). 2020-04-12 18:23:14 -07:00
Evan Tschannen ce4493f679 many bug fixes 2020-04-10 13:45:16 -07:00
Alex Miller 6078fd1b18 Convert UID to Tag in keyServers to reduce txnStateStore size 2020-04-05 14:30:09 -07:00
Jon Fu b1fd6b4443 addressed review comments 2019-10-18 09:43:25 -07:00
Jon Fu e04535b0a6 remove unneeded comment 2019-10-14 09:37:55 -07:00
Jon Fu 6e1af6b2d9 changed check in movekeys for matching of srcSet and intendedTeam 2019-10-10 10:58:28 -07:00