dd902fc772
create ShardsAffectedByTeamFailure.h
2022-07-29 15:12:15 -07:00
35a4cb91d5
Disable ShardedRocksDB in simulation when shard_encode_location_metadata is disabled ( #7726 )
...
* Disabled tests for ShardedRocks.
Cleaned up ShardedRocks TraceEvent.
Added assertion in ShardManager::validate().
* Added test trace.
* Make sure TraceEvent contains `ShardedRocks`.
* Exclude ShardedRocksDB when SHARD_ENCODE_LOCATION_METADATA is disabled.
Co-authored-by: He Liu <heliu@apple.com>
2022-07-28 13:54:29 -07:00
dcf09fd691
Move DD queue code over to using movement-reasons rather than priority ( #7614 )
...
* Use enum variables to invoke Priority checking
* add an explicit isDataMovementForReadBalancing function
* Set up RelocateShard in terms of data movement reason instead of priority
* Remove isMountainChopperPriority
* Remove isDiskRebalancePriority
* Fix formatting
* Fix misnamed DataMovementReason::TEAM_HEALTHY
Co-authored-by: Zhongxing Zhang <zhongxing.zhang@snowflake.com>
2022-07-25 00:50:37 -07:00
7b6f1ca712
Merge pull request #7621 from sfc-gh-xwang/feature/dd-refactor-incremental
...
[DD Testability] Add unittest for resuming infligh relocations
2022-07-23 22:14:45 -07:00
cd13356964
solve review comments
2022-07-20 16:09:38 -07:00
41032500d0
only do check after bootstraping test
2022-07-20 14:47:54 -07:00
7865ead54d
Merge branch 'main' into readaware
2022-07-19 13:35:33 -07:00
1de37afd52
Make TEST macros C++ only ( #7558 )
...
* proof of concept
* use code-probe instead of test
* code probe working on gcc
* code probe implemented
* renamed TestProbe to CodeProbe
* fixed refactoring typo
* support filtered output
* print probes at end of simulation
* fix missed probes print
* fix deduplication
* Fix refactoring issues
* revert bad refactor
* make sure file paths are relative
* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
aca44c8644
Increase frequency of RelocateShardStartSplit traces for better observability of splits ( #7503 )
2022-07-08 14:45:44 -07:00
bc5bfaffda
Shard based move ( #6981 )
...
* Shard based move.
* Clean up.
* Clear results on retry in getInitialDataDistribution.
* Remove assertion on SHARD_ENCODE_LOCATION_METADATA for compatibility.
* Resolved comments.
Co-authored-by: He Liu <heliu@apple.com>
2022-07-07 20:49:16 -07:00
48bcf6eaba
parameterizing min split size instead of hardcoding it to min_shard_bytes ( #7404 )
2022-06-30 12:28:22 -07:00
0c2bd73e28
Fix typo in SHARD_READ_HOT_BANDWIDTH_MIN_PER_KSECONDS name
2022-06-27 09:56:02 -07:00
9b8859f4f2
move keys out of StorageMetrics
2022-05-27 17:10:01 -07:00
13a77dd5a2
change priority knob; change PromiseStream to FutureStream; remove comments; add on_sr check
2022-05-25 17:09:34 -07:00
8950822e36
add knobs; change knobs
2022-05-17 14:49:27 -07:00
f40a2a68fc
add more statistics
2022-05-17 10:19:09 -07:00
3b241955e7
add more informative trace info
2022-05-16 21:25:56 -07:00
b0a00effa0
set max shard bandwidth
2022-05-06 16:37:12 -07:00
964e0cecef
format
2022-05-04 14:52:39 -07:00
ae66ed6c16
fix DataDistributionQueue time_out ; reset the rebalance poll time
2022-05-04 14:11:20 -07:00
1723bee639
add fetchTopKShardMetrics to dd tracker
2022-05-03 23:42:09 -07:00
00b97ec829
add storage metric compare knob; timeThrottle with constant
2022-04-27 23:37:35 -07:00
73c7240bfd
code format
2022-04-26 17:34:34 -07:00
cc05f5e9db
fix getMetrics keys bug
2022-04-24 21:21:18 -07:00
04311d001e
topK shard random selection
2022-04-21 22:37:16 -07:00
d93b57dd88
conflict solving
2022-03-24 20:45:51 -07:00
1b631a9263
solve conflict with main
2022-03-24 16:29:11 -07:00
a71099471b
Update copyright header dates
2022-03-21 13:36:23 -07:00
4b92f8f546
add relocate reason and set teamSorter in relocator
2022-03-18 16:39:31 -07:00
184f0d7586
Improve const correctness of storage wiggler classes
2022-03-11 09:20:53 -04:00
8d20ee8432
change storage metrics of read sample calculation
2022-03-07 17:36:43 -08:00
a07e413eae
metrics comparator; rebalanceReadLoad()
2022-02-24 16:41:01 -08:00
d8e161f89e
Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state.
2021-12-17 11:57:39 -08:00
fd2b27d7c4
Merge pull request #6103 from sfc-gh-ajbeamon/fix-dd-merge-too-soon
...
Fix: Merge too soon bug
2021-12-06 15:01:47 -08:00
ce2d1e1648
Fix: the shard tracker state could become inaccurate if there is an ABA type update to the bandwith state of a shard.
2021-12-03 18:42:55 -08:00
3b5d23ef88
Remove unnecessary temporary objects when appending to vector of pairs
2021-12-01 18:15:16 -08:00
c10dd0df4b
fix roll trace event issue for data distribution
...
Description
Testing
2021-09-24 15:40:17 -07:00
abf73047ca
Enforce std:: specifier rather than using namespace
2021-09-16 19:40:28 -07:00
17f6f7a2ca
Using the current max shard size estimate instead of the max possible if unavailable
2021-08-19 16:17:55 -05:00
3442ebd3b7
Fix more -Wreorder-ctor warnings across many files
2021-07-24 11:20:51 -07:00
f5d312e4a0
Avoid casting NaN to int
...
Apparently it's possible for dbSizeEstimate to be negative
2021-06-04 09:47:06 -07:00
d2f9bf7eb6
added comments and fixed style
2021-03-16 15:44:49 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
76a4293d1d
Merge branch 'release-6.3'
...
# Conflicts:
# fdbserver/fdbserver.actor.cpp
2021-02-15 02:13:06 -08:00
f4c9b88908
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# cmake/CompileBoost.cmake
# fdbserver/DataDistribution.actor.cpp
# fdbserver/fdbserver.actor.cpp
# flow/Error.cpp
2021-02-15 02:05:03 -08:00
4e184fe236
Fix memory errors
2021-02-11 02:58:21 +00:00
5b2e88b187
Use structured bindings in for loops
2020-12-27 01:46:20 -04:00
19816ccdbf
Improve DataDistribution const-correctness
2020-12-26 20:22:27 -04:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
dc2bac5670
Resolve conflicts
2020-11-24 19:09:42 +00:00
Xiaoxi Wang