Commit Graph

7540 Commits

Author SHA1 Message Date
Chaoguang Lin 2aa0a5cf58 The lock special key has the lock UID as the value 2021-09-03 12:15:56 -07:00
helium 8e0b572a18 Added comments. 2021-09-02 22:29:07 -07:00
helium 7e7f9372de Replaced clear() with krmSetRangeCoalescing(). 2021-09-02 21:58:05 -07:00
Zhe Wu 51920ffa16 RecruitFromConfigurationRetry should wait for goodRecruitmentTime to be ready 2021-09-02 09:32:08 -07:00
helium a8a6f6b8a3 Remove Transaction::set() when clearing the keyrange. 2021-09-01 15:35:40 -07:00
helium 6612cc00b6 Check if the src server list will be empty before removing a failed server." 2021-09-01 14:52:07 -07:00
yao-xiao-github f7842ff3db
Log SubCode and Severity for RocksDB erros (#5515) 2021-08-31 21:03:02 -07:00
Zhe Wu 2d298355a5
Merge pull request #5489 from halfprice/zhewu/remote-checkregion-in-health-checking
Remove checkRegions in shouldTriggerRecoveryDueToDegradedServers
2021-08-30 15:15:42 -07:00
sfc-gh-tclinkenbeard a574661bc3 Remove unnecessary temporary object from ApiCorrectnessWorkload::getMetrics 2021-08-30 13:33:25 -07:00
Zhe Wu 6e4f74c9b3 Remove checkRegions in shouldTriggerRecoveryDueToDegradedServers 2021-08-30 13:03:14 -07:00
sfc-gh-tclinkenbeard e918d20f68 Document "averaged" parameter of PerfMetric constructor 2021-08-29 15:38:21 -07:00
Jingyu Zhou e576fd45f5
Merge pull request #5496 from xis19/master
Refactor TxnStateRequest handler in commitProxyServerCore
2021-08-29 11:14:45 -07:00
Xiaoge Su e6b8a519c4 fixup! Make processTransactionStateRequestPart not blocking 2021-08-28 21:10:10 -07:00
Xiaoge Su 74d8ca87d7 Refactor TxnStateRequest handler in commitProxyServerCore 2021-08-27 23:38:34 -07:00
sfc-gh-tclinkenbeard 86c6c2b643 Merge remote-tracking branch 'origin/master' into paxos-config-db 2021-08-27 23:24:14 -07:00
FDB Formatster 69508b980f modify comments to make clang-format and coverage tool play nice 2021-08-27 17:18:00 -07:00
FDB Formatster 2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
sfc-gh-tclinkenbeard 3a880b43d4 Fix and reenable PaxosConfigTransaction::onError 2021-08-27 15:06:33 -07:00
sfc-gh-tclinkenbeard ad99f025e3 Merge remote-tracking branch 'origin/master' into paxos-config-db 2021-08-27 14:13:34 -07:00
sfc-gh-tclinkenbeard bf982644cf For now, ConfigIncrement workload passes even if clients get stuck 2021-08-27 13:46:08 -07:00
A.J. Beamon d706a7ddd7
Merge pull request #5474 from sfc-gh-ajbeamon/unit-tests-require-match
Unit tests require a match from the test pattern in order to be successful
2021-08-27 09:17:23 -07:00
sfc-gh-tclinkenbeard 29d83291a1 Add CommitUnknownResult metric for ConfigIncrement workload 2021-08-27 00:44:12 -07:00
sfc-gh-tclinkenbeard 28f3481656 Remove unnecessary debugging trace events 2021-08-26 17:13:39 -07:00
sfc-gh-tclinkenbeard 7f8a1385d5 Merge remote-tracking branch 'origin/master' into paxos-config-db 2021-08-26 17:00:43 -07:00
sfc-gh-tclinkenbeard 28daab9f5c Use ActorCollection instead of std::vector<Future<Void>> in *Quorum classes, to listen for errors 2021-08-26 16:18:19 -07:00
Fuheng Zhao e0acce98ab format 2021-08-26 13:15:05 -07:00
Fuheng Zhao d8c257e263 resolve conflicts 2021-08-26 13:13:01 -07:00
Fuheng Zhao 8a19dac2f7 update the count and precetage to countLinear and percentageLinear 2021-08-26 13:08:11 -07:00
A.J. Beamon 5ce2e44a36 Unit tests require a match from the test pattern in order to be successful. This avoids cases where a typo in the pattern might lead one to believe that their tests were all successful. 2021-08-26 12:20:09 -07:00
Steve Atherton be440ab954
Merge pull request #5260 from FuhengZhao/RedwoodHistogram
Redwood local histograms
2021-08-26 12:05:44 -07:00
Fuheng Zhao 67600d436e avoid yield 2021-08-26 09:51:07 -07:00
Fuheng Zhao bc9fd3ca96 Merge branch 'RedwoodHistogram' of https://github.com/FuhengZhao/foundationdb into RedwoodHistogram 2021-08-26 09:27:23 -07:00
Fuheng Zhao 7a043e72fb timing the log time for 33 histograms and change a void function to ACTOR 2021-08-26 09:27:00 -07:00
Steve Atherton 02c729d16b
Remove unused variable. 2021-08-25 16:03:00 -07:00
Steve Atherton ba63d3b7d7
Fix typo 2021-08-25 15:59:56 -07:00
Steve Atherton 5c29da2a0f
Merge pull request #5461 from sfc-gh-satherton/durability-bug-repro1
TLogServer popped version calculation was off-by-one for retired tags after master recovery
2021-08-25 15:56:49 -07:00
Fuheng Zhao 3b26cb5f99 add yield 2021-08-25 11:40:24 -07:00
Trevor Clinkenbeard 5d84ffe019
Merge pull request #5441 from xis19/master
Refactor ApplyMetadataMutation for better readability
2021-08-24 18:42:18 -07:00
Steve Atherton deeb6b3404 Merge branch 'master' of https://github.com/apple/foundationdb into durability-bug-repro1
# Conflicts:
#	fdbserver/TLogServer.actor.cpp
2021-08-24 16:19:16 -07:00
Fuheng Zhao 5eb92ae65c reomve registry 2021-08-24 15:47:04 -07:00
Steve Atherton ec0e39b40f Bug fix: Popped versions are exclusive, so after recovery a tag for which there is no longer data should be considered popped up until the version *after* recovery, indicating that data at the recovery version itself has been popped. 2021-08-24 15:16:20 -07:00
Chaoguang Lin 868fa93d3d Merge branch 'master' of github.com:apple/foundationdb into refactor-cache_range-command 2021-08-24 12:58:23 -07:00
Xiaoge Su fd1c09c853 fixup! Fix the code per comments 2021-08-24 12:58:14 -07:00
Xiaoge Su a2114550e0 Refactor ApplyMetadataMutation for better readability 2021-08-24 12:58:14 -07:00
sfc-gh-tclinkenbeard a047782db2 Merge remote-tracking branch 'origin/master' into paxos-config-db 2021-08-24 11:08:20 -07:00
Fuheng Zhao 57652a0cae Format 2021-08-24 10:00:24 -07:00
Fuheng Zhao b65a66fab7 log redwood histogram seperatly 2021-08-24 09:57:39 -07:00
Steve Atherton d6bdde6fe8 Fix trace event detail name collision in MutationTracking use cases. 2021-08-24 08:04:19 -07:00
Zhe Wang 7f595f4925 add_trace_event_for_role_recruitment_by_CC 2021-08-23 20:23:55 -05:00
Lukas Joswiak 7e6bc27863 Remove linear time loop 2021-08-23 14:02:41 -07:00
sfc-gh-tclinkenbeard 6c2bf4fd8d Add "Last Known Value" metric to ConfigIncrement workload 2021-08-23 13:32:57 -07:00
sfc-gh-tclinkenbeard 78b57629a1 Handle not_committed error in PaxosConfigTransaction::onError 2021-08-23 13:09:49 -07:00
Lukas Joswiak f037ac7b19 Fix capitalization 2021-08-23 12:59:01 -07:00
Lukas Joswiak 08892eab55 Move client failure cleanup 2021-08-23 12:54:03 -07:00
Lukas Joswiak eaac647558 Fix kill worker bug 2021-08-23 12:45:42 -07:00
Lukas Joswiak adc1025fa1 Clean up clientFailures periodically 2021-08-23 12:45:42 -07:00
Lukas Joswiak d004703cc8 Add worker kill unit test 2021-08-23 12:45:42 -07:00
Trevor Clinkenbeard 61a8bb6443
Merge pull request #5412 from sfc-gh-ljoswiak/features/atomic-knobs
Add atomicity to knobs
2021-08-23 12:42:47 -07:00
Daniel Smith 8bde0b3136 Disable RocksDB storage engine in simulation 2021-08-23 14:56:13 -04:00
Lukas Joswiak 8996e396b9 Remove old local configuration initialization 2021-08-23 11:41:01 -07:00
Lukas Joswiak 4c92026213 Use existing error when triggering local config restart 2021-08-23 11:41:01 -07:00
Lukas Joswiak 338fc59673 Separate test knobs into atomic and nonatomic 2021-08-23 11:41:00 -07:00
Lukas Joswiak e0f75c2b99 Remove TODO 2021-08-23 11:41:00 -07:00
Lukas Joswiak 078e614595 Add atomic clear unit test 2021-08-23 11:41:00 -07:00
Lukas Joswiak 4107c35538 Add atomic knob unit test 2021-08-23 11:41:00 -07:00
Lukas Joswiak 7998ec4232 Add atomicity to knobs 2021-08-23 11:41:00 -07:00
Daniel Smith b75fb0d105 Improve error to status conversion 2021-08-23 11:35:26 -04:00
Daniel Smith 473e2bbf5f Fix error handling of reads. 2021-08-23 11:35:26 -04:00
Chaoguang Lin 8ed041132f Rewrite cache_range related functions to template code; Remove duplicate code; fix the existing bug in cache_range; 2021-08-20 13:42:50 -07:00
Trevor Clinkenbeard 66df75c570
Merge pull request #5385 from sfc-gh-tclinkenbeard/debug-dd
Capture deep copy of `machine_info` in `printSnapshotTeamsInfo`
2021-08-20 13:25:50 -07:00
sfc-gh-tclinkenbeard 9458a6975d Remove std::map::at usage from DataDistribution.actor.cpp 2021-08-20 12:35:26 -07:00
Zhe Wu 16445984c5
Merge pull request #5425 from halfprice/zhewu/fix-cc-health-monitor-cancellation
Allow CC health monitor to handle Actor cancellation
2021-08-20 11:01:38 -07:00
Steve Atherton ebdb9064ff
Merge pull request #5359 from RenxuanW/mlr
Check in the prototype of MutationLogReader.
2021-08-19 21:11:10 -07:00
Trevor Clinkenbeard cb04f6a135
Merge pull request #5426 from sfc-gh-tclinkenbeard/paxos-config-db
Reenable ConfigIncrement test
2021-08-19 18:18:43 -07:00
neethuhaneesha 41f94e6e3b
Merge pull request #5130 from neethuhaneesha/readCancelRocksDB
Cancelling the timedout reads with rocksdb storage.
2021-08-19 16:41:22 -07:00
Zhe Wu a8655e5f4e Allow CC health monitor to handle cancellation 2021-08-19 16:32:18 -07:00
sfc-gh-tclinkenbeard 658233d0e0 Merge remote-tracking branch 'origin/master' into paxos-config-db 2021-08-19 16:18:59 -07:00
sfc-gh-tclinkenbeard 0fcca0eb60 Add IKnobCollection::clearTestKnobs to fix unit tests 2021-08-19 16:18:44 -07:00
Evan Tschannen 2adad62e59
Merge pull request #5424 from sfc-gh-jslocum/shard-metrics-timeout-fix
Using the current max shard size estimate instead of the max possible…
2021-08-19 16:00:51 -07:00
Neethu Haneesha Bingi 0fbfcc582e Increasing the read cancellation timeouts to 5 minutes in simulation. 2021-08-19 14:47:12 -07:00
sfc-gh-tclinkenbeard b6c669be23 Send ConfigBroadcastSnapshotReply to broadcaster 2021-08-19 14:45:30 -07:00
Chaoguang Lin 8c56cef23c
Merge pull request #5366 from sfc-gh-clin/add-flow-db-interface
Rewrite ThrottleApi functions using template
2021-08-19 14:28:55 -07:00
Josh Slocum 17f6f7a2ca Using the current max shard size estimate instead of the max possible if unavailable 2021-08-19 16:17:55 -05:00
Neethu Haneesha Bingi 6c97782378 Timedout reads cancellation check removed in simulation. 2021-08-19 14:11:28 -07:00
Neethu Haneesha Bingi 02b3ed3ff1 Adding deadline option to rocksdb calls. 2021-08-19 14:11:28 -07:00
Neethu Haneesha Bingi fbb393f998 Added readrange timeoout check and rocksdb read deadline option. 2021-08-19 14:11:28 -07:00
Neethu Haneesha Bingi 24ac173c95 Adding ReadRangeAction timeout, returning error and using timer_monotonic changes. 2021-08-19 14:11:28 -07:00
Neethu Haneesha Bingi 01e85610ab Cancelling the timedout reads with rocksdb storage. 2021-08-19 14:11:28 -07:00
Chaoguang Lin fbed117b21 Solve comments; Add a method to return DatabaseContext Reference 2021-08-19 19:34:25 +00:00
Daniel Smith 45b40addb7
Merge pull request #5379 from neethuhaneesha/eagerReadsDisable
Disabling option for removing eagerReads for ClearRange mutations.
2021-08-19 12:39:30 -04:00
sfc-gh-tclinkenbeard 8faf418b45 Add back LocalConfiguration::initialize 2021-08-18 22:27:38 -07:00
Josh Slocum 0a0b4cf333
Merge pull request #5410 from sfc-gh-xwang/master
fix uninitialized int
2021-08-18 17:01:20 -07:00
Xiaoxi Wang 7b0756e391 fix uninitialized int 2021-08-18 15:48:02 -07:00
A.J. Beamon 448ffe7931
Merge pull request #5408 from yao-xiao-github/time-keeper-event
Add version and timestamp to TimeKeeperCommit event.
2021-08-18 15:30:11 -07:00
yao-xiao-github 0afb63c13f
Update fdbserver/ClusterController.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2021-08-18 15:29:13 -07:00
Yao Xiao 7f0a09d1e2 Enable debugTransaction before getReadVersion. 2021-08-18 15:22:58 -07:00
Renxuan Wang 6e89dc3da0 Check in the prototype of MutationLogReader. 2021-08-18 14:59:50 -07:00
Jingyu Zhou 38f9cbd8b6
Merge pull request #5406 from Doxense/fix-windows-build-with-system-time
Fix windows build with system time
2021-08-18 14:26:27 -07:00
sfc-gh-tclinkenbeard 62303af832 Remove invalid assertion from ConfigBroadcastSnapshotRequest handling 2021-08-18 13:24:00 -07:00
yao-xiao-github b8e2575f92
Update log directory to server log directory. (#5396) 2021-08-18 13:07:56 -07:00
sfc-gh-tclinkenbeard e0981e71b2 Remove invalid assertion from LocalConfiguration::addChanges 2021-08-18 12:49:00 -07:00
sfc-gh-tclinkenbeard a7f9db8b76 Remove knob reinitialization from updateInMemoryState 2021-08-18 12:32:12 -07:00
Josh Slocum 074bd174b6 Fix worker segfault by reference counting LocalConfiguration 2021-08-18 14:07:20 -05:00
Yao Xiao 4b8dadfdfb Use Time instead of Timestamp. 2021-08-18 11:53:52 -07:00
Mohamed Oulmahdi 3a9e7150f7 Add SIGUSR1 and SIGUSR2 macros for Windows 2021-08-18 18:06:36 +02:00
Yao Xiao 977f8f494e Add version and timestamp to TimeKeeperCommit event. 2021-08-18 01:27:38 -07:00
Josh Slocum 6aabd9a03e Adding FIXME for simulation issue 2021-08-17 18:18:28 -05:00
Chaoguang Lin 7ac1d17bcb Add a local namespace in Ratekeeper.actor.cpp to differenitate name GrvProxyInfo 2021-08-17 21:05:57 +00:00
Chaoguang Lin b9628bdb50 Merge branch 'master' of github.com:apple/foundationdb into add-flow-db-interface 2021-08-17 20:24:41 +00:00
Chaoguang Lin 179c313e08 Use DatabaseContext instead of Database in template functions 2021-08-17 20:02:25 +00:00
sfc-gh-tclinkenbeard 0bacc310ef Reenable consumer in config broadcaster 2021-08-17 12:09:12 -07:00
Evan Tschannen 0a78c84a3e
Merge pull request #5183 from sfc-gh-xwang/tlog_dev
TLog Streaming Peek
2021-08-17 11:51:34 -07:00
sfc-gh-tclinkenbeard 616a01d01d Only register each worker once with config broadcaster (consumer currently disabled) 2021-08-17 11:45:50 -07:00
Xiaoxi Wang d12bda94ae disable trace log 2021-08-16 16:33:20 -07:00
Jingyu Zhou bccb09f9dd
Merge pull request #5387 from sfc-gh-tclinkenbeard/improve-worker-const-correctness
Add const qualifiers to several parameters in `worker.actor.cpp`
2021-08-16 13:32:07 -07:00
sfc-gh-tclinkenbeard 51ee619c16 Remove LocalConfiguration::initialize 2021-08-16 11:39:37 -07:00
sfc-gh-tclinkenbeard 3418c20867 Merge remote-tracking branch 'origin/master' into paxos-config-db 2021-08-16 10:49:47 -07:00
sfc-gh-tclinkenbeard 52a64eb04b Improve const-correctness for AsyncVar references 2021-08-14 23:41:39 -07:00
sfc-gh-tclinkenbeard ceb83f7f5e Make ccInterface a const reference in workerServer 2021-08-14 23:41:39 -07:00
sfc-gh-tclinkenbeard 556e4bc283 Add assertion to overlappingMachineMembers 2021-08-13 14:56:22 -07:00
sfc-gh-tclinkenbeard a0a4207ce2 Capture deep copy of machine_info in printSnapshotTeamsInfo 2021-08-13 12:13:54 -07:00
sfc-gh-tclinkenbeard 3f0e07d79c Remove dead code 2021-08-13 09:54:02 -07:00
sfc-gh-tclinkenbeard ea4f6850da Mark ServerStatus::excludeOnRecruit const 2021-08-13 09:52:07 -07:00
sfc-gh-tclinkenbeard 0aafd9d5f0 Mark TCServerInfo::isCorrectStoreType const 2021-08-13 09:49:22 -07:00
Josh Slocum e444d3781c Various TSS improvements from snowblower testing 2021-08-13 10:24:15 -05:00
Neethu Haneesha Bingi c45daf6f51 Disabling option for removing eagerReads for ClearRange mutations. 2021-08-13 01:26:50 -07:00
Steve Atherton 303449f82f
Merge pull request #5378 from FuhengZhao/RedwoodMetricsLevelFix
Redwood Metric Level
2021-08-12 20:29:08 -07:00
Fuheng Zhao 4282911ce2 level off by 1 fixed 2021-08-12 20:06:46 -07:00
sfc-gh-tclinkenbeard 904deb9516 Improve DDTeamCollection const-correctness 2021-08-12 18:52:57 -07:00
sfc-gh-fzhao 29b4f221a7
Merge branch 'apple:master' into RedwoodHistogram 2021-08-12 17:07:05 -07:00
sfc-gh-tclinkenbeard cfe677c100 storageRecruiter only responds to changes in recruitStorage endpoint 2021-08-12 16:24:03 -07:00
Xiaoxi Wang df7a801945 remove FIXME 2021-08-12 14:10:34 -07:00
sfc-gh-tclinkenbeard 45ac667271 Add IsPrimary boolean parameter 2021-08-12 14:05:04 -07:00
Andrew Noyes b39a903af0
Merge pull request #5371 from sfc-gh-anoyes/anoyes/fix-heap-use-after-free
Fix heap use after free
2021-08-12 07:27:05 -07:00
Xiaoxi Wang a97570bd06 solve mis-spelling, trace log and format problems 2021-08-11 18:26:00 -07:00
Chaoguang Lin bf0d0598dc Refactor all ThrottleAPI functions using template and remove duplicate copies 2021-08-11 23:59:25 +00:00
Andrew Noyes ca9f60baef Fix heap use after free 2021-08-11 15:42:01 -07:00
Lukas Joswiak a605fb3852
Merge pull request #5026 from sfc-gh-ljoswiak/fixes/alp6
Actor sampling
2021-08-11 13:44:17 -07:00
Lukas Joswiak 1faec36bc6 Wait for all snapshot replies before sending incremental changes 2021-08-11 11:17:51 -07:00
Lukas Joswiak c098a1128d Push snapshot changes to local configuration on refresh 2021-08-11 09:13:22 -07:00
Josh Slocum 389b71ba72
Merge pull request #5365 from sfc-gh-satherton/debugging-cleanup
Added Label field for keys and ranges in MutationTracking.
2021-08-11 07:07:36 -07:00
Trevor Clinkenbeard bd0a04d4bc
Merge pull request #5345 from sfc-gh-ljoswiak/features/broadcaster-push
Refactor config broadcaster to push updates to workers
2021-08-10 19:56:35 -07:00
Chaoguang Lin 618c72f7e2 Draft to use template to rewrite ThrottleApi 2021-08-11 01:07:36 +00:00
Steve Atherton 09a41ff61b Fix duplicate trace event fields on MutationTracking event. 2021-08-10 17:56:28 -07:00
Steve Atherton f000447414 Fix duplicate TraceEvent detail name. 2021-08-10 17:43:41 -07:00
Steve Atherton 3922fa8cdd Fix duplicate TraceEvent detail name. 2021-08-10 17:41:31 -07:00
Steve Atherton eb5c8eb78d Added Label field for keys and ranges in MutationTracking. 2021-08-10 17:32:32 -07:00
Steve Atherton 01248820a7 Added Label field for keys and ranges in MutationTracking. 2021-08-10 17:30:07 -07:00
Lukas Joswiak 7fe863c2a0 Temporarily remove failing unit test 2021-08-10 16:20:51 -07:00
Steve Atherton 2cf5a3d883 Added range tracking support to MutationTracking. 2021-08-10 15:52:30 -07:00