Commit Graph

73 Commits

Author SHA1 Message Date
Xiaoxi Wang 0cbd1dfcca add comment and MockDDTest base class file 2022-11-01 14:56:55 -07:00
Xiaoxi Wang 334fced572 add data api implementations; add more realistic fetchKey implementation; finish randomKeyBetween implementation 2022-11-01 14:56:55 -07:00
He Liu 7bb823edbe
Replace KeyRange with std::vector<KeyRange> in DataMoveMetaData and (#8591)
* Replace KeyRange with std::vector<KeyRange> in DataMoveMetaData and
CheckpointMetaData.

* Checked if ranges.empty().

* fmt.

* Resolved some comments.

Co-authored-by: He Liu <heliu@apple.com>
2022-10-28 15:22:55 -07:00
Lukas Joswiak 9d3c3b1efe Remove cluster ID logic from individual roles
The logic to determine the validity of a process joining a cluster now
belongs on the worker and the cluster controller. It is no longer
restricted to tlogs and storages, but instead applies to all processes
(even stateless ones).
2022-10-27 13:56:13 -07:00
Xiaoxi Wang 36d9de9072 change UNREACHABLE to ASSERT(false); change function name 2022-10-25 15:43:24 -07:00
Xiaoxi Wang e07a50573a splitStorageMetrics finish implementation (no unit test yet but 100k
test pass)
2022-10-24 09:58:41 -07:00
Xiaoxi Wang 5d90703dc8 finish getKeysLocations etc, and unit test pass. 2022-10-24 09:58:41 -07:00
Xiaoxi Wang 70eb9aef28 simple MGS IKeyLocationService implementation 2022-10-24 09:58:41 -07:00
Jingyu Zhou a8391caf23 Revert "Data loss protection v2" 2022-10-20 18:09:58 -05:00
Lukas Joswiak 72bc89cf39 Remove cluster ID logic from individual roles
The logic to determine the validity of a process joining a cluster now
belongs on the worker and the cluster controller. It is no longer
restricted to tlogs and storages, but instead applies to all processes
(even stateless ones).
2022-10-18 21:37:42 -07:00
Xiaoxi Wang 08b006efba solve compile warning 2022-10-18 09:49:07 -07:00
Xiaoxi Wang 6630ec8c1d remove virtual on raw* Methods 2022-10-17 16:43:32 -07:00
Xiaoxi Wang 9c28dd46be add assert to parallelism lock take 2022-10-17 14:42:04 -07:00
Xiaoxi Wang 5d57429949 modify boolean definition; add comments 2022-10-17 10:12:00 -07:00
Xiaoxi Wang 313c76232c skip shard size validity 2022-10-15 22:52:43 -07:00
Xiaoxi Wang 83685e6a61 add status verification; add skipDDModeCheck in getInitialDataDistribution for workload test 2022-10-15 22:52:42 -07:00
Xiaoxi Wang 4fa8d3e8ef finish mock MoveKeys implementation (not tested yet) 2022-10-15 22:52:42 -07:00
Xiaoxi Wang 00c19ec9ed expose rawStartMovement and rawFinishMovement 2022-10-15 22:52:42 -07:00
Xiaoxi Wang e959bdef8d extract startMoveKeysTransaction method 2022-10-15 22:52:42 -07:00
He Liu b52edd8658 Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency 2022-10-10 11:00:05 -07:00
He Liu 1931e3266f Resolve comments. 2022-10-10 10:28:19 -07:00
Xiaoxi Wang 21b2e11bc4 getWorkers from IDDTxnProcessor 2022-10-04 14:57:04 -07:00
Xiaoxi Wang 4cf4ccc089 correct getServerListAndProcessClasses implementation (100k pass) 2022-10-03 22:24:35 -07:00
Xiaoxi Wang 76f2dc8ce0 merge upstream/main 2022-10-02 22:07:42 -07:00
Xiaoxi Wang 1cc46a98f8 solve merge conflicts 2022-09-28 14:34:42 -07:00
Xiaoxi Wang 3578832c69 merge upstream/main 2022-09-28 14:01:50 -07:00
Xiaoxi Wang e476e814b2 merge upstream/main 2022-09-27 10:15:26 -07:00
Xiaoxi Wang 14d73193d5 waitDDTeamInfoPrintSignal, getClusterId, tryUpdateReplicasKeyForDc in IDDTxnProcessor 2022-09-26 23:00:31 -07:00
Xiaoxi Wang d9cc7ff44e merge upstream/main 2022-09-23 14:49:46 -07:00
Xiaoxi Wang 050bf22e1e add comments to test toml and moveKeys to non-const method 2022-09-22 13:11:53 -07:00
Xiaoxi Wang 9358aea097 fix busy loop with correct error handling in valley filler 2022-09-21 14:58:34 -07:00
Xiaoxi Wang f9e0230b86 DDQueue constructor with ITxnProcessor 2022-09-21 10:56:22 -07:00
Xiaoxi Wang 08f4c06450 remove unnecessary creation of transactions; call all cx and transaction methods in DDShardTracker through txnProcessor 2022-09-19 14:41:34 -07:00
Xiaoxi Wang 7dd1d7a029 solve merge problem; enable an empty top k unit tests 2022-09-14 14:13:54 -07:00
Xiaoxi Wang 85f119c13b fix destination team issue 2022-09-14 00:17:00 -07:00
Xiaoxi Wang 46eadd53cf finish getInitialDataDistribution test verification 2022-09-13 22:43:15 -07:00
Xiaoxi Wang bafcacc1eb fix MockDDTxnProcessor::getInitialDataDistribution segment fault in assertion part; IDDTxnProcessorApiWorkload get mock initData finish 2022-09-13 14:39:12 -07:00
He Liu 958b28497e Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency 2022-09-13 13:55:01 -07:00
He Liu 18cdef43be Cleanup. 2022-09-13 13:52:08 -07:00
He Liu ebabe916f1 Cleanup. 2022-09-13 09:28:41 -07:00
Xiaoxi Wang 94b81d9125 mark MockDDTxnProcessor::removeKeysFromFailedServer as unreachable() 2022-09-12 13:03:45 -07:00
He Liu e6df139204 Get sources servers by DC in DDAuditRange. 2022-09-12 11:04:23 -07:00
He Liu ad11c6e82d Cleanup. 2022-09-06 19:00:09 -07:00
He Liu 6f4d09f947 Replace Reference<> with std::shared_ptr<>. 2022-09-05 12:45:14 -07:00
Xiaoxi Wang f83e2251d6 add shard status inference comments; finish ShardsAffectedByTeamFailure::removeFailedServerForRange (no unit test yet) 2022-09-01 09:29:16 -07:00
Xiaoxi Wang fbb6016c40 pass /MockGlobalState/initialAsEmptyDatabaseMGS 2022-08-31 22:26:10 -07:00
Xiaoxi Wang d192a5630c add comment to ShardsAffectedByTeamFailure 2022-08-30 14:36:14 -07:00
Xiaoxi Wang 0ba37ca0c4 add counterpart of system keyspace 2022-08-30 10:59:14 -07:00
Xiaoxi Wang 8544cf48d4 implement DDMockTxnProcessor::getInitialDataDistribution (compiled) 2022-08-29 14:59:40 -07:00
Xiaoxi Wang 5c2b846fbd solve conflicts in upstream/main 2022-08-19 15:36:17 -07:00