Commit Graph

21820 Commits

Author SHA1 Message Date
Josh Slocum 12ccd3b146 typo 2022-07-20 09:35:57 -05:00
Josh Slocum e0f6897481 fixing merge conflicts 2022-07-20 09:11:37 -05:00
Junhyun Shim b521dc2f58 Bind DeterministicRandom to OpenSSL RNG 2022-07-20 16:03:09 +02:00
Josh Slocum 6e43a50ea1 cleanup 2022-07-20 08:39:32 -05:00
Josh Slocum 5540f92e2e Refactoring randomExp to the proper place 2022-07-20 08:36:03 -05:00
Josh Slocum e3cbd63b6d fixing merge conficts 2022-07-20 08:20:48 -05:00
Josh Slocum 3f8f44dd16 Merge branch 'main' into improved_cf_testing 2022-07-20 08:19:52 -05:00
Josh Slocum 922a4523a1 disabling popped data checking again 2022-07-20 08:19:27 -05:00
Josh Slocum 0b674bf0c4 Merge branch 'main' into cf_bw_operational_fixes 2022-07-20 08:03:59 -05:00
Josh Slocum fd9201f60b Merge branch 'main' into cf_metadata_rewrite 2022-07-20 07:55:00 -05:00
Josh Slocum 44885aa719 cleanup 2022-07-20 07:46:57 -05:00
Josh Slocum 78b6a96006 Merge branch 'main' into granule_merging_batch 2022-07-20 07:42:26 -05:00
Josh Slocum 4000682578 moving comment 2022-07-20 07:35:30 -05:00
Junhyun Shim 0874e553cd Place generateRandomData() under {I|Deterministic}Random 2022-07-20 13:21:11 +02:00
Jingyu Zhou 0f094d2aa1 Fix ScopeEventFieldTypeMismatch error for TLogMetrics
Backport changes made to TLogServer in #4616 back to old TLog implementations.
Old implementations were using SpecialCounter, which is string. The new type
is int64_t. Without the change, we can have many events like:

<Event Severity="30" Time="1658165532.958152" DateTime="2022-07-18T17:32:12Z" Type="ScopeEventFieldTypeMismatch" ID="0000000000000000" EventType="TraceEvent.TLogMetrics" FieldName="QueueDiskBytesUsed" OldType="Int64" NewType="String" ThreadID="2451559325190099131" Machine="10.242.248.26:4703" LogGroup="xxxx" Roles="TL" />
2022-07-19 22:27:29 -07:00
Jingyu Zhou a7469b925b
Merge pull request #7630 from jzhou77/main
Add getMappedRange latency metrics
2022-07-19 22:13:50 -07:00
Jingyu Zhou e8baba8009
Merge pull request #7575 from xis19/lb
Notes on Load Balancer
2022-07-19 17:50:25 -07:00
Josh Slocum 12b6f386cb Refactoring granule flush to retry properly on granule rollback 2022-07-19 19:49:20 -05:00
Lukas Joswiak c357a90b30 Ensure change coordinators request came from same generation proxy
There was a rare but possible issue where a change coordinators request
could be in flight, but reading `\xff/coordinators` would return the old
set of coordinators, even though a previous attempt to write it returned
`commit_unknown_result`. This happened if there was an unrelated
recovery between setting `\xff/coordinators` and the updated
coordinators reaching each machine. The `commit_unknown_result` caused
by this recovery meant the change coordinators retry loop would read
`\xff/coordinators`, but since the request was ongoing it would read the
old set of coordinators. The fix is to have the cluster controller
ignore any change coordinators request from an old generation, meaning
when the retry loop reads the old coordiantors from `\xff/coordinators`,
it is guaranteed that the in progress change coordinators message will
be rejected when arriving at the cluster controller.
2022-07-19 17:44:48 -07:00
sfc-gh-tclinkenbeard 8d2cde32d9 Added global-tag-throttling.md 2022-07-19 17:04:04 -07:00
sfc-gh-tclinkenbeard 8a18fa23cf Reenable GlobalTagThrottler_GotClientRate trace events 2022-07-19 17:04:04 -07:00
sfc-gh-tclinkenbeard f197314cb5 Add /GlobalTagThrottler/MultiTagActiveThrottling3 unit test 2022-07-19 17:04:04 -07:00
sfc-gh-tclinkenbeard 118eb4d93b Add /GlobalTagThrottler/MultiTagActiveThrottling2 unit test and fix bug uncovered in GlobalTagThrottlerImpl::getLimitingTps 2022-07-19 17:04:04 -07:00
sfc-gh-tclinkenbeard b4e711f1c3 Make parameter more readable in GlobalTagThrottlerTesting::runClient 2022-07-19 17:04:04 -07:00
sfc-gh-tclinkenbeard 20ac60fb11 Set throttling ratio in GlobalTagThrottler::tryUpdateAutoThrottling 2022-07-19 17:04:04 -07:00
sfc-gh-tclinkenbeard f41515e66b Implement GlobalTagThrottler::busy*TagCount methods 2022-07-19 17:04:01 -07:00
Josh Slocum 166de4b704 Fixing merge convergence bugs 2022-07-19 18:44:40 -05:00
Jingyu Zhou 72e72f6d87 Add getMappedRange latency metrics 2022-07-19 16:31:24 -07:00
sfc-gh-tclinkenbeard afba51b7f8 Add tracing to GlobalTagThrottler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard fe05cc5c72 Update busy read tag reporting in status json 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 145d10e3a4 Compute cluster-wide average transaction cost for each tag only once in getClientRates 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 080518d014 Use getQuota method in GlobalTagThrottlerImpl::getDesiredTps 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard d0851a67e2 Update TransactionTagCounter::startNewInterval tracing 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 2306757e90 Fix GlobalTagThrottler::autoThrottleCount 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 4977805d8e Fix failing global tag throttling unit tests 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard c86ae6cee5 Fix /GlobalTagThrottler/Simple unit test 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard e128c517d3 New GlobalTagThrottler implementation tracking more per-storage
server metrics
2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 6a63db08c8 Added /GlobalTagThrottler/ReservedWriteQuota unit test 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 5b0eb135e0 Guarantee reserved quota in GlobalTagThrottler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 8e08033974 Add /GlobalTagThrottler/MultiTagActiveThrottling unit test 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard bac583ef0c Add new active throttling unit test for global tag throttler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 34a78a82f1 Add MockStorageServer class for global tag throttler testing 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard 88ca6376ce Add LimitType and OpType enum classes for global tag throttler 2022-07-19 16:29:11 -07:00
sfc-gh-tclinkenbeard fb6540ab43 Add throttling ratio to GlobalTagThrottler 2022-07-19 16:29:10 -07:00
sfc-gh-tclinkenbeard ab2110142a Add /GlobalTagThrottler/RemoveQuota unit test 2022-07-19 16:29:10 -07:00
sfc-gh-tclinkenbeard 019900be67 Add GlobalTagThrottlerImpl::removeUnseenTags 2022-07-19 16:29:10 -07:00
sfc-gh-tclinkenbeard 3161bcfa52 Remove outdated TODO and FIXME comments from GlobalTagThrottlerImpl 2022-07-19 16:29:06 -07:00
sfc-gh-tclinkenbeard b49c36f0b0 Add StorageQueueInfo::getWriteQueueSizeLimitRatio method 2022-07-19 16:28:27 -07:00
Jingyu Zhou bc62b538b1
Merge pull request #7627 from jzhou77/remove-metrics
Remove unused "readQueueSizeMetric"
2022-07-19 15:55:52 -07:00
Dan Lambright 7e296c2367
simple replication (#7625) 2022-07-19 18:30:34 -04:00