598b2eaeb0
fdbrpc: Add warning when peer is unavailable for long time
2020-01-08 13:55:13 -08:00
83ad9caf54
implemented a load balancing algorithm which evens out the number of requests processes by each proxy
2020-01-08 01:59:01 -08:00
3c769fcf60
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/ClusterController.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# versions.target
2019-11-22 15:39:19 -08:00
27cb299d84
simulation can sometimes randomly hang or throw connection_failed, instead of always doing one or the other
2019-11-21 16:24:18 -08:00
2727b91c46
simulation tests network connections failing due to errors instead of just hanging
2019-11-21 12:33:07 -08:00
3325980c03
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/DataDistribution.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/WorkerInterface.actor.h
# fdbserver/worker.actor.cpp
# versions.target
2019-10-24 17:38:15 -07:00
325a8e4213
remove confusing USE_ODIRECT knob
2019-10-24 11:44:03 -07:00
f23392ec5a
Don't use O_DIRECT in EIO by default
2019-10-24 11:39:55 -07:00
7ad0e20e48
Added knob to disable O_DIRECT
2019-10-24 11:20:14 -07:00
f41f19b5f6
Introduced knob to set eio parallelism
2019-10-24 11:20:14 -07:00
e005ef8e9a
Merge pull request #2030 from kaomakino/kaomakino/quickack
...
TCP_QUICKACK test
2019-09-10 17:06:12 -07:00
e5bbaa293a
Add FLOW_TCP_NODELAY and FLOW_TCP_QUICKACK (Linux-only) knobs
2019-09-10 20:13:15 +00:00
8a54a57420
Merge pull request #2026 from alexmiller-apple/knobs-for-network-test
...
Make the networktest payload size a knob so that it can be changed.
2019-09-10 00:11:42 -07:00
49c623826f
Make the networktest payload size a knob so that it can be changed.
2019-08-21 18:37:50 -07:00
51cedd24c8
load balance will send reads to remote servers if more than one alternative is failed or overloaded
2019-08-19 13:59:49 -07:00
da8163fd5a
allow one requests every second to skip there all_alteratives_failed delay, because if a client has a timeout longer than the delay we will never invalidate the key servers cache
2019-08-09 13:03:40 -07:00
5f7d3498ea
The delay for all_alteratives_failed can scale all the way up to 30.0 at a much slow time ratio
2019-08-09 12:35:19 -07:00
370ba8b841
Remove --object-serializer flag from executables
2019-08-06 09:25:40 -07:00
2d136af2bd
bool knobs can now be set with the words “true” or “false” instead of just a number
2019-07-30 18:21:24 -07:00
5d3e69b6fc
Merge pull request #1820 from fzhjon/load-balance-locality
...
Introduced a knob that can turn locality on/off
2019-07-16 16:40:43 -07:00
7d37040725
split the single knob into two for finer-grained control
2019-07-16 12:46:02 -07:00
d5051b08dd
Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated.
2019-07-12 16:12:35 -07:00
19a91765f6
introduced a knob that can turn locality on/off
2019-07-09 16:16:15 -07:00
983343978e
fdbrpc: ConnectionMonitor should close unreferenced after delay
...
Potentially for cases, where it goes up to 1 immediately.
2019-07-09 14:24:16 -07:00
1f9c80f633
fdbrpc: Instead of tracking last sent data, track last sent non-ping data
...
* This will allow client to continue monitoring peer connections while
connection stays open, so that there is no period of "uncertainity"
without previous no-monitoring approach.
* Use multiplier for incoming connection idle timeout
* Update idle connection timeout values and leaked connection timeout in
simulator.
2019-07-09 14:24:16 -07:00
867986cdea
fdbrpc: Reduced connection monitoring from clients
...
This patch does two changes to connection monitoring:
1. Connection monitoring at client side will check if the connection
has been stayed idle for some time. If connection is unused for a
while, we close the connection. There is some weirdness involved here
as ping messages are by themselves are connection traffic. We get over
this by making it two-phase process, first being checking idle
reliable traffic, followed by disabling pings and then checking for
idle unreliable traffic.
2. Connection monitoring of clients from server will no longer send
pings to clients. Instead, it keep monitor the received bytes and
close after certain period of inactivity.
2019-07-09 14:24:16 -07:00
e6ff23c420
Allow next packet size to be read when receiving data
...
For large packet, allocate sizeof(uint32_t) more bytes for next packet size.
Also add knob MIN_PACKET_BUFFER_FREE_BYTES, which is used to trigger allocation
of a new arena when free bytes are lower than this threshold.
2019-06-25 10:18:56 -07:00
c2c82c89a5
Remove knob DEFAULT_PACKET_BUFFER_BYTES
2019-06-25 10:18:56 -07:00
2363326ecb
Add knobs for various packet buffer sizes
2019-06-25 10:18:56 -07:00
5a8c97480a
Merge pull request #1506 from nikolas-ioannou/feature-pagecache-lru
...
AsyncFileCached: switch from a random to an LRU cache eviction policy
2019-05-17 13:42:21 -07:00
1d5572a078
removed unused knob
2019-05-13 14:15:22 -07:00
fdb3992990
AsyncFileCached: switch to string for eviction policy knob
2019-05-03 14:04:43 +02:00
b2280e15bf
AsyncFileCached: support for lru cache eviction policy
...
- Added a knob to control page cache eviction policy (0=random default, 1=lru)
- Added page cache hit, miss, and eviction stats
2019-05-02 17:35:30 +02:00
390ab9cfed
A process will mark itself as degraded if it continually disconnects from a different process which the failure monitor thinks is healthy
2019-04-04 14:11:12 -07:00
d670b74d69
prevent trace event spam by combining huge arena samples
2019-03-30 13:36:13 -07:00
70ac5ffda0
added random logging for huge arenas
2019-03-20 11:20:47 -07:00
c8122edb6b
avoid relying on 128<<10 in knobs.cpp
2019-03-18 12:40:15 -07:00
d2eb7578fd
Occasionally log a backtrace when the fast allocators allocates a new magazine to help track down memory leaks
2019-03-17 21:59:56 -07:00
c32504f705
io: Add DISABLE_POSIX_KERNEL_AIO knob to use EIO instead of Kernel AIO
...
- Some Linux filesystems don't support O_DIRECT which is required by
Kernel AIO to function properly. Instead of using O_SYNC, EIO is
much better options in terms of performance penalty.
- Some systems may not support AIO at all. Eg. Windows Subsystem for
Linux.
FIXES #842
RELATED #274
2019-03-13 13:39:45 -07:00
268093a96d
Adjust all includes to be relative to the root.
...
Remove the use of relative paths. A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h". Adjust so that every include references such a header with the
latter form.
Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
664f64881c
Port truncate optimization from Snowflake PR in order to make quick changes for a patch release.
2018-10-05 15:05:26 -07:00
3186fac397
Make sure we still accept some connections even if we are CPU bound by high priority work
2018-08-10 17:47:21 -07:00
fb46385a39
Merge pull request #628 from alexmiller-apple/reloadcertificates
...
Reload certificates if changed.
This is a cherry-pick of #628 back to release-6.0
2018-08-06 18:04:04 -07:00
5054c194e2
Some trace events are logged before FLOW_KNOBS are initialized
2018-07-01 14:30:37 -04:00
fe956bc35a
Address review comments
2018-06-26 14:37:21 -07:00
f1dea3bed9
Rework the formatter a bit and extract out the writing logic so that it could be pluggable.
2018-05-31 13:27:35 -07:00
78839b20fd
Merge branch 'master' into trace-log-refactor
...
# Conflicts:
# flow/Trace.cpp
2018-05-31 10:46:20 -07:00
047563aa62
Support trace event overflow.
2018-05-04 15:19:56 -07:00
62726efa82
Add support for logging trace events before the network is created and the trace log opened.
2018-03-28 08:21:22 -07:00
f2c804e14f
Reverting changes from merge of master into release-5.2 ( b25810711c). Note that we never intend to release master into release-5.2, but if we did we would need to revert this commit.
2018-03-06 10:15:04 -08:00
37a6a81634
Merge commit '7f6fc3e039c911cd84b8540f7f799fc38a1c1822' into feature-remote-logs
...
# Conflicts:
# fdbserver/workloads/RestartRecovery.actor.cpp
2018-02-23 12:33:28 -08:00
0bae9880f1
remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py
2018-02-21 10:25:11 -08:00
5ac4f73978
Merge branch 'release-5.1' into feature-remote-logs
...
# Conflicts:
# fdbclient/NativeAPI.actor.cpp
# fdbrpc/Locality.h
# fdbrpc/simulator.h
# fdbserver/ApplyMetadataMutation.h
# fdbserver/ClusterController.actor.cpp
# fdbserver/LogSystemPeekCursor.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/SimulatedCluster.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/TagPartitionedLogSystem.actor.cpp
# fdbserver/WorkerInterface.h
# fdbserver/masterserver.actor.cpp
# flow/Net2.actor.cpp
# tests/fast/SidebandWithStatus.txt
# tests/rare/LargeApiCorrectnessStatus.txt
# tests/slow/DDBalanceAndRemoveStatus.txt
2018-01-05 11:33:42 -08:00
482ac38ca6
added knobs so that the client failure monitoring update rate and the server failure monitoring update rate are separate knobs
2017-12-01 13:04:32 -08:00
80e5fecfe2
Increase message limit for throttling to 100,000 for circus runs.
...
Added an optimization to use a separate set for throttled events. Since this set is expected to be small, comparison of every event against this set is going to be cheaper.
2017-10-31 10:35:26 -07:00
d852a53ae4
Merge pull request #181 from cie/throttle-spammy-logs
...
Throttle spammy logs
2017-10-25 13:45:55 -07:00
e2c1e87df6
made a large number of fixes to make fearless DR correctness clean.
2017-10-19 15:36:32 -07:00
f997cb9038
Add a string knob to hold the Log directory, and write profiles to it.
...
This is the combination of two small changes.
1. Add support for a string knob type.
2. Change profiles to be written to the log directory instead of the working
directory.
We have three options of where to write files: the working directory, the data
directory, and the log directory.
The working directory may be set to a non-writable location, and likely
contains the fdb binaries. Allowing these files to be overwritten would likely
not be a wise idea.
The data directory hosts our sqlite b-trees. It would also be very unfortunate
if these were ever overwritten by an unfortunate profile name.
The log directory contains logs. Out of the three, these matter the least if
they disappear or become corrupted.
Thus, we write to the log directory.
2017-10-16 16:05:02 -07:00
3aaa11977e
Addressed Review Comments
2017-10-12 14:56:00 -07:00
0e153cdd35
Throttle Spammy logs. Three knobs are added.
...
Trace Events are sampled and cached with an expiration set. Every TraceEvent above SevDebug is checked against this cache to see if it exceeded a set threshold. If yes, then throttle the TraceEvent.
If a TraceEvent is throttled, a warning msg is logged.
2017-10-02 18:43:11 -07:00
430bb6224e
Merge branch 'release-4.6' into release-5.0
...
# Conflicts:
# fdbrpc/AsyncFileKAIO.actor.h
# fdbrpc/Net2FileSystem.cpp
# fdbrpc/sim2.actor.cpp
2017-06-16 02:14:19 -07:00
4bdcd8fc12
Merge branch 'release-4.6' into release-5.0
...
# Conflicts:
# bindings/bindingtester/run_binding_tester.sh
# fdbrpc/AsyncFileKAIO.actor.h
2017-06-14 16:43:53 -07:00
a674cb4ef4
Initial repository commit
2017-05-25 13:48:44 -07:00
Vishesh Yadav