f5d312e4a0
Avoid casting NaN to int
...
Apparently it's possible for dbSizeEstimate to be negative
2021-06-04 09:47:06 -07:00
d2f9bf7eb6
added comments and fixed style
2021-03-16 15:44:49 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
76a4293d1d
Merge branch 'release-6.3'
...
# Conflicts:
# fdbserver/fdbserver.actor.cpp
2021-02-15 02:13:06 -08:00
f4c9b88908
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# cmake/CompileBoost.cmake
# fdbserver/DataDistribution.actor.cpp
# fdbserver/fdbserver.actor.cpp
# flow/Error.cpp
2021-02-15 02:05:03 -08:00
4e184fe236
Fix memory errors
2021-02-11 02:58:21 +00:00
5b2e88b187
Use structured bindings in for loops
2020-12-27 01:46:20 -04:00
19816ccdbf
Improve DataDistribution const-correctness
2020-12-26 20:22:27 -04:00
877997632d
Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
...
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
dc2bac5670
Resolve conflicts
2020-11-24 19:09:42 +00:00
1f541f02be
Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
...
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
2e13aacf1e
Merge pull request #4049 from sfc-gh-tclinkenbeard/fix-shard-merge-too-soon
...
Fix HasBeenTrueFor::set function
2020-11-18 10:29:58 -08:00
d64cf8b9e3
Merge branch 6.3 into master
2020-11-17 11:22:45 -08:00
84632d63ad
Remove stale comment
2020-11-16 17:54:47 -08:00
898fbac191
Remove special handling of broken_promise errors
2020-11-16 13:28:08 -08:00
4f369e70ab
Add code coverage macro to DataDistributionTracker::SafeAccessor::operator()
2020-11-16 13:25:39 -08:00
6235d087a6
Prevent shardTracker or trackShardBytes from accidentally unsafely accessing DataDistributionTracker
2020-11-16 12:46:21 -08:00
d0391db862
Merge branch 'release-6.2' into release-6.3
2020-11-16 10:15:23 -08:00
ca8ea3b6ff
Fix memory issues caused by cancelling data distribution tracker
2020-11-15 23:52:36 -08:00
6917da9ce7
Fix HasBeenTrueFor::set
2020-11-13 13:15:51 -08:00
222da17558
Merge branch 'release-6.2' into mengxu/ha-code-read
2020-11-12 13:39:27 -08:00
c2dd7d1d38
Remove unresolved questions
2020-11-11 22:39:11 -08:00
f467524e06
Don't dereference self on broken_promise
2020-11-11 00:24:23 +00:00
4669f837fa
Add uses of makeReference
2020-11-07 22:10:18 -08:00
063700e4d6
Add comments and questions to HA and tLog code reading
...
The comments' correctness need to be confirmed by reviewers.
2020-10-30 12:14:57 -07:00
8f17a1a5d6
Merge branch 'release-6.2' into release-6.3
2020-10-16 15:25:39 -07:00
91a8367acb
Avoid slow task in ~DataDistributionTracker
2020-10-01 11:44:55 -07:00
3ac6996844
Increased some knobs to throttle the spammy read hot logging. Also added more details inside the read hot log to make it useful
2020-08-12 14:27:34 -07:00
5bc2e2e595
update comments, make isAsync() virtual, remove unused code
2020-06-12 10:12:44 -07:00
980bee1d13
Fix fetchShardMetricsList_impl and add read cache in special key space
2020-06-12 10:12:44 -07:00
298b94a044
clang-format the change
2020-05-19 20:51:02 -07:00
c230c23a3a
Add test for smaller range read in dd-stats
2020-05-19 20:49:15 -07:00
93ca130602
Delete the unnecessay endKey field in DDMetricsRef, since each begin key is the previous end key
2020-05-19 13:32:42 -07:00
ef724bf939
Merge remote-tracking branch 'upstream/master' into add-data-distribution-metrics
2020-05-08 18:39:28 -07:00
e8b62e48f4
Rename DDMetrics to DDMetricsRef
2020-05-08 17:17:27 -07:00
1927f15898
Now the ReadHotDetection tests can be run reliably. Added it to nightly bundle.
2020-04-28 13:26:02 -07:00
76eb1c4376
Address review comments from Meng
2020-04-21 11:00:19 -07:00
7dd7406c59
Merge branch 'master' into feature/hot-read-key-detection-part-2
2020-04-16 14:54:05 -07:00
b1172417f5
Merge branch 'master' into per-priority-busy-logging
...
# Conflicts:
# flow/Knobs.cpp
# flow/Knobs.h
# flow/Net2.actor.cpp
2020-04-14 14:22:12 -07:00
247ab84323
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-03-23 17:01:17 -07:00
e08f0201f1
merge release 6.2 into master
2020-03-17 12:51:47 -07:00
7adc916e18
Merge pull request #2806 from ajbeamon/improve-team-request-performance
...
Improve performance of get team requests.
2020-03-16 11:56:45 -07:00
12f2b32770
added additional logging in data distribution
2020-03-13 15:19:33 -07:00
555db50cd1
Avoid calling into SABTF so frequently. Use a cheaper call that only checks that shards exist.
2020-03-12 11:22:03 -07:00
abb75f7eb7
Add logging to indicate the time spent at each priority that exceeds some minimum busyness threshold
2020-02-07 14:34:24 -08:00
c9363e7e28
Merge branch 'master' of https://github.com/apple/foundationdb into add-data-distribution-metrics
2020-01-22 21:02:21 -08:00
b0a1af1288
Added the actual read hot detection algorithm and logging machanism.
...
- When a shard has a read bandwidth larger than a threshold value(configurable via knob), and it's read-bandwidth/byte-size ratio is also larger than a threshold(configurable via knob), the corresponding shard tracker will run the algorithm
- The algorithm will divide the shard into 10MB(configurable via knob) chunks and try to find the chunk(s) that has large aforementioned ratio
- Then those ranges will be logged into TraceEvents. This will later do more like actually cache them.
2020-01-21 11:19:52 -08:00
1d6cd1007b
Instead of using absolute value as the max bytesReadPerKSec threshold, use a pre-defined read traffic to byte size ratio to decide that value dynamically based on the actual size of the shard.
2020-01-21 11:15:52 -08:00
3f9d9d8b84
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# cmake/FlowCommands.cmake
# documentation/sphinx/source/release-notes.rst
# fdbclient/StorageServerInterface.h
# fdbserver/DataDistributionTracker.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# flow/Knobs.h
# flow/Platform.cpp
# versions.target
2020-01-16 18:37:47 -08:00
1d6ac716a1
Merge remote-tracking branch 'origin' into add-data-distribution-metrics
2020-01-15 13:20:04 -08:00
Andrew Noyes