Commit Graph

290 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard a0c49234b2 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-03-19 20:47:53 -07:00
Evan Tschannen f2c82f4fc6 too much run loop batching slows down the speed of transactions enough that if cannot commit transactions fast enough when either TRANSACTION_LIFE_VERSIONS is buggified to be small 2021-03-18 12:06:28 -07:00
Evan Tschannen 8f94167bc9 lowered the buggified time batching 2021-03-17 09:57:11 -07:00
Evan Tschannen 412609cd62 increased max batching time 2021-03-16 19:09:12 -07:00
Evan Tschannen 13242d8b35 the sim2 runloop now updates time in batches so that multiple tasks can execute with the same now() 2021-03-15 12:33:43 -07:00
Vishesh Yadav 2cd3f45fd6 Merge remote-tracking branch 'apple/release-6.3' into master-format-final
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
2021-03-10 16:50:44 -08:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Russell Sears 36bf68ed2e
Merge branch 'release-6.3' into clang_format_the_world/6.3 2021-03-08 14:40:52 -08:00
FDB Formatster 8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
Andrew Noyes 23b85256ae Change default to 64, and only start threads if using TLS 2021-03-04 20:52:46 +00:00
Andrew Noyes cc7fed6ed4 Change default for TLS_SERVER_HANDSHAKE_THREADS to 0 2021-03-04 20:52:44 +00:00
A.J. Beamon aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
A.J. Beamon 67e783acf8 Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
# Conflicts:
#	cmake/CompileBoost.cmake
#	cmake/FDBComponents.cmake
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/storageserver.actor.cpp
#	flow/Knobs.h
#	flow/network.h
2021-02-08 09:20:28 -08:00
Meng Xu 1db4b73215
Merge pull request #4243 from sfc-gh-etschannen/feature-storage-io-timeout
throw an io_timeout if it takes more than 2 minutes to commit on the storage server
2021-02-03 18:10:16 -08:00
Chaoguang Lin 593165679a Remove unnecessary header 2021-02-02 12:42:46 -08:00
Chaoguang Lin 17671604c0 Add option to define I/O size 2021-02-02 12:39:05 -08:00
Steve Atherton b0fd6bfbcf More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-30 18:18:16 -08:00
Steve Atherton 52c2695f59 Bug fixes involving use of AFCached rate control in simulation. 2021-01-30 18:13:44 -08:00
Steve Atherton e56fe02a25 Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit 2021-01-30 18:11:54 -08:00
Steve Atherton 7ce0cca687 Merge branch 'release-6.3' of https://github.com/apple/foundationdb into afcache-write-limit 2021-01-28 07:27:32 -08:00
Evan Tschannen 3e2a0a5264 throw an io_timeout if it takes more than 2 minutes to commit on the storage server 2021-01-27 11:50:42 -08:00
sfc-gh-tclinkenbeard 511ec43e62 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-01-27 11:17:58 -08:00
Steve Atherton 989fee908a More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-27 00:01:33 -08:00
sfc-gh-tclinkenbeard 7558cd8e06 Merge remote-tracking branch 'origin/master' into simplify-global-knobs 2021-01-26 10:00:25 -08:00
Andrew Noyes adde95bea5 Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
Commit merge conflicts for now
2021-01-25 18:55:24 +00:00
sfc-gh-tclinkenbeard 88bc157bd0 Added AsyncFileEncrypted 2021-01-24 22:53:04 -08:00
Steve Atherton 0d32bd0c44 Bug fixes involving use of AFCached rate control in simulation. 2021-01-24 01:50:09 -08:00
Chaoguang Lin a09acd321e update knobs default values for testing 2021-01-22 11:45:37 -08:00
Chaoguang Lin 18f7ee22c2 Merge branch 'release-6.3' of github.com:apple/foundationdb into add-throttling-on-AsyncfileCached 2021-01-22 11:44:29 -08:00
Andrew Noyes 281151ff20 Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-6.2-to-6.3
Commit conflict markers for now
2021-01-21 17:54:26 +00:00
Lukas Joswiak e8397b3812 Move UDP listen port to knob 2021-01-20 13:57:36 -08:00
Kao Makino 4ddf3385a8 Reserve the priority queue container 2021-01-20 10:34:48 -08:00
sfc-gh-tclinkenbeard 8ff14878fe Merge remote-tracking branch 'origin/master' into simplify-global-knobs 2021-01-13 14:39:35 -08:00
sfc-gh-tclinkenbeard e29ed3bf99 Remove createGlobal*Knobs functions 2021-01-13 12:14:04 -08:00
Chaoguang Lin ced946c918 Move flow knobs under AsyncFileCached part 2020-12-09 23:02:13 -08:00
Chaoguang Lin 13819171fa Update knob names to FLOW_CACHEDFILE_WRITE_WINDOW_LIMIT and FLOW_CACHEDFILE_WRITE_WINDOW_SECONDS 2020-12-09 11:45:09 -08:00
Chaoguang Lin 90f56d0e84 Add STORAGE_WRITE_WINDOW_SECONDS and STORAGE_WRITE_WINDOW_SECONDS for rate control 2020-12-08 16:35:08 -08:00
Andrew Noyes 877997632d Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
Andrew Noyes 1f541f02be Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
sfc-gh-tclinkenbeard b46384e1b8 Initialize client and flow knobs 2020-11-22 09:56:18 -08:00
sfc-gh-tclinkenbeard b1e3478267 Simplified global knobs 2020-11-21 13:27:48 -08:00
David Youngworth 50e515c29a Add DateTime to trace, initial commit 2020-11-18 16:54:49 -08:00
David Youngworth d64cf8b9e3 Merge branch 6.3 into master 2020-11-17 11:22:45 -08:00
David Youngworth d0391db862 Merge branch 'release-6.2' into release-6.3 2020-11-16 10:15:23 -08:00
Vishesh Yadav 1976f2c371 FlowTransport: Sample connect latencies 2020-11-10 12:12:01 -08:00
Young Liu 8cc3e4d3c6 Merge release-6.3 into master 2020-10-19 22:51:56 -07:00
Jingyu Zhou 8f17a1a5d6 Merge branch 'release-6.2' into release-6.3 2020-10-16 15:25:39 -07:00
Evan Tschannen 52828f9e03 Added bytesSent to the PingLatency logging; increasing the logging interval from 1 second to 3 seconds 2020-10-11 23:05:21 -07:00
Evan Tschannen f03a355c4e Merge branch 'release-6.2' into release-6.3
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/downloads.rst
#	fdbrpc/FlowTransport.actor.cpp
#	fdbrpc/FlowTransport.h
#	flow/Arena.h
#	flow/Knobs.cpp
#	flow/Knobs.h
#	packaging/msi/FDBInstaller.wxs
2020-10-07 12:28:22 -07:00
Evan Tschannen 29d4d87b56 added a knob for the number of samples 2020-10-06 17:38:09 -07:00
Evan Tschannen 9efda1fec5 added logging for the ping latencies for all network connections 2020-10-06 13:58:05 -07:00
Evan Tschannen 7ba06a4434 fix: min and max compute estimate logging on the proxy was always zero
added comments and fixed formatting
2020-10-05 12:35:10 -07:00
Evan Tschannen 5807b1ec3d changed the recent requests to be the per second amount; increased precision of cpu estimate 2020-10-04 19:31:40 -07:00
Evan Tschannen 614c8bc895 Get read versions requests must be load balanced on the number of requests because ratekeeper gives out an equal budget to each proxy 2020-10-04 16:20:24 -07:00
Lukas Joswiak efde86340a Add knob to disable span serialization 2020-09-04 17:37:34 -07:00
Evan Tschannen 5c97461d18
Merge pull request #3653 from etschannen/feature-proxy-busy-loadbalance
Changed proxy load balancing to balance on CPU usage
2020-08-31 10:39:08 -07:00
Evan Tschannen 29eec30183 Merge branch 'release-6.2' into release-6.3
# Conflicts:
#	CMakeLists.txt
#	build/Dockerfile
#	build/Dockerfile.devel
#	documentation/sphinx/source/downloads.rst
#	fdbserver/Knobs.cpp
#	fdbserver/LogSystem.h
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/TagPartitionedLogSystem.actor.cpp
#	fdbserver/WaitFailure.actor.cpp
#	fdbserver/fdbserver.vcxproj
#	fdbserver/fdbserver.vcxproj.filters
#	packaging/msi/FDBInstaller.wxs
2020-08-31 01:10:29 -07:00
Evan Tschannen c9ff450a36 do not reject a connection as redundant if our existing connection is more than 15 seconds old 2020-08-30 18:49:49 -07:00
Evan Tschannen 947a625968
Update flow/Knobs.cpp
Co-authored-by: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-08-24 11:33:49 -07:00
Evan Tschannen 9e2ee1ed4c fixed lastedZeroBusy; added a knob 2020-08-17 23:16:59 -07:00
Evan Tschannen c72068d6b5 clients load balance across proxies based on process busyness instead of number of requests 2020-08-12 17:17:21 -07:00
Evan Tschannen ee9b3d1a9d do not let a single failed handshake prevent reconnection for 10 seconds 2020-07-27 10:58:30 -07:00
Steve Atherton d375ee7939 Rolled back the knob and logic for doing multiple write() calls in write loops before yielding, because they are not necessary. The testing that suggested otherwise was in error. 2020-07-16 22:54:14 -07:00
Steve Atherton c4f3553826 TLS handshake limit changed to regular flow lock, added new knob for simultaneous handshakes limit and removed old knobs. Added knob to control how much data the FlowTransport connectionWriter() can write using multiple write() calls before yielding to other tasks, and updated p2p network test to use the same logic. Changed knob defaults for send limit on single writes and TLS handshake thread count. 2020-07-16 02:31:17 -07:00
Steve Atherton 5151f1167c Added TLS option to limit the number of glibc malloc arenas if TLS background threads are created. TLS initialization now has three phases in order to avoid creating TLS threads or limit the malloc arena count unnecessarily. 2020-06-28 17:29:13 -07:00
Steve Atherton 813dbfb297 TLS handshake threads are now created with TLS initialization instead of when the network is created since they may not be needed and it avoids calling startThread() with a custom stack size too early in process execution. StartThread() now logs a warning if the given stack size can't be used. Reorganized TLS handshake thread state near the other TLS related stuff. 2020-06-23 03:09:56 -07:00
Steve Atherton c3ce0034bf Background threads for TLS handshakes use a stack size based on a knob. Platform::startThread() now accepts a stack size. The generic thread pool implementation takes an optional stack size override which is used for each added thread. 2020-06-23 02:08:01 -07:00
Steve Atherton 3853860cf1 Changed default TLS handshake thread count. 2020-06-22 16:28:30 -07:00
Steve Atherton 6330f8b458 TLS handshakes can now be done using up to a configured number of background threads if any of them are not busy. 2020-06-21 06:23:32 -07:00
Evan Tschannen aed2d34bcb Merge branch 'master' into feature-proxy-load-balance
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	flow/Knobs.cpp
2020-05-01 09:19:39 -07:00
Evan Tschannen 76fb345dd1 Merge branch 'master' into feature-tree-broadcast
# Conflicts:
#	fdbrpc/FailureMonitor.actor.cpp
2020-04-29 09:51:22 -07:00
Evan Tschannen ba3e2af473 Merge commit '5288033bcfe40c3ade97c8bf2d04cf31b3f16cb1' into feature-tree-broadcast 2020-04-17 15:17:37 -07:00
Vishesh Yadav 8c8f23bff2 Merge remote-tracking branch 'apple/master' into task/issue-1017-slow-machine-poisoning 2020-04-16 00:45:35 -07:00
A.J. Beamon d8690d31cd Merge branch 'master' into per-priority-busy-logging
# Conflicts:
#	flow/Net2.actor.cpp
2020-04-15 08:31:30 -07:00
A.J. Beamon b1172417f5 Merge branch 'master' into per-priority-busy-logging
# Conflicts:
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/Net2.actor.cpp
2020-04-14 14:22:12 -07:00
A.J. Beamon e104a2e3a6 Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into run-loop-busy-profiler
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/NativeAPI.actor.h
#	fdbserver/fdbserver.actor.cpp
#	flow/Net2.actor.cpp
2020-04-14 14:02:24 -07:00
Evan Tschannen 07cc0a8d74 code cleanup 2020-04-10 17:02:11 -07:00
Vishesh Yadav 13447f439f fdbrpc: Add a constant to onFailedFor()
Since, we mark an address as failed when connection is failed, this
patch adds a contant to compensate the time needed to reconnect and
make sure endpoint is actually down. This contant is equal to
FAILURE_MIN_DELAY which was used by centralized
FailureMonitoringClient earlier removed.
2020-04-08 19:34:40 -07:00
Vishesh Yadav 975e6b1d9a Merge remote-tracking branch 'apple/master' into task/issue-1017-slow-machine-poisoning
Removed merge conflict with old build system.
2020-04-08 19:25:13 -07:00
Vishesh Yadav fdc1048f75 Add knob to turn off marking unstable connections 2020-04-03 15:53:00 -07:00
Vishesh Yadav 1d35f2ff5a Mark a connection as failed for X seconds if closes too often 2020-04-03 15:53:00 -07:00
tclinken 884e92bb49 Atomically update dependent knobs 2020-04-01 15:18:49 -07:00
Evan Tschannen e08f0201f1 merge release 6.2 into master 2020-03-17 12:51:47 -07:00
A.J. Beamon d8cfabe73b Extend the allocation tracing disabling flag to cover more parts of trace logging as a precaution. Make it possible to disable via knob. 2020-03-16 13:59:31 -07:00
Evan Tschannen 303df197cf Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	bindings/c/test/mako/mako.c
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogRouter.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/flow.vcxproj
#	flow/flow.vcxproj.filters
#	versions.target
2020-03-06 18:22:46 -08:00
Evan Tschannen 39050308ff lower accept batch size just to be conservative with the change 2020-03-05 18:17:49 -08:00
Evan Tschannen 820957025f accept connections in batches of 20 to improve performance 2020-03-04 14:24:57 -08:00
Evan Tschannen 924d335aa7 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	flow/Knobs.cpp
#	flow/Knobs.h
2020-02-25 18:25:19 -08:00
Evan Tschannen 65fbe0d0bc revert AcceptSocket priority change because of bad performance results 2020-02-21 19:22:14 -08:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
Evan Tschannen f04e311a1e Merge commit 'b46d6e25e24993ab5a5f04091fd3235050b7cd09' into feature-boost-ssl
# Conflicts:
#	fdbserver/SimulatedCluster.actor.cpp
#	flow/Net2.actor.cpp
2020-02-20 17:36:38 -08:00
Evan Tschannen 08c318d28a re-added the connect lock in the fdbcli so that the timeout is not spent before a connection has been initiated (because of the handshake lock) 2020-02-20 10:43:34 -08:00
Evan Tschannen 69b5a1fbe3 more priority improvements 2020-02-20 10:11:43 -08:00
Evan Tschannen fbd45963d8 The cluster controller waits until no new workers register for 1.0 before starting a bad recruitment 2020-02-19 16:48:30 -08:00
Alex Miller fe78524bbc
Merge pull request #2678 from sears/networktest_perf
Add some tuning knobs to networktestclient; also, measure latency directly
2020-02-19 14:38:09 -08:00
Evan Tschannen 693e469003 Changed the handshake lock to a BoundedFlowLock, which will enforce that old handshakes complete before starting to initiate new handshakes 2020-02-14 16:49:52 -08:00
Russell Sears 7724c644e5 Add some tuning knobs to networktestclient; also, measure latency directly. 2020-02-13 13:11:54 -08:00
Andrew Noyes 1248d2b8b4 Remove USE_OBJECT_SERIALIZER knob 2020-02-12 10:41:52 -08:00
A.J. Beamon abb75f7eb7 Add logging to indicate the time spent at each priority that exceeds some minimum busyness threshold 2020-02-07 14:34:24 -08:00
Evan Tschannen 69de430057 separate handshaking from connection to improve pipelining 2020-02-06 16:45:54 -08:00
Evan Tschannen 53d0867a17 limit the number of connections a process can attempt to establish in parallel 2020-02-04 18:15:10 -08:00
Evan Tschannen 4524831456
Merge pull request #2518 from vishesh/task/failmon-remove-server
FailureMonitoring: Server processes no longer need to talk to ClusterController
2020-02-03 17:22:50 -08:00
A.J. Beamon 182dac7cd5 Convert the slow task profiler into a run loop profiler that also logs when the run loop is 100% busy for a knob-configurable duration. 2020-01-28 12:09:37 -08:00
Evan Tschannen 78adbea834 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	flow/Knobs.h
#	versions.target
2020-01-21 21:38:19 -08:00
Evan Tschannen afd3ec13ff added knobs 2020-01-21 18:58:34 -08:00
Vishesh Yadav daef5f011a Merge remote-tracking branch 'apple/master' into task/failmon-remove-server 2020-01-21 13:20:15 -08:00
Evan Tschannen 3f9d9d8b84 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	cmake/FlowCommands.cmake
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/StorageServerInterface.h
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Knobs.h
#	flow/Platform.cpp
#	versions.target
2020-01-16 18:37:47 -08:00
Evan Tschannen 0e916fdbed throttle client TLS errors longer than server errors so that when both happen simultaneously the server throttling will be disabled when the client makes its next attempt 2020-01-12 22:12:18 -08:00
Balachandar Namasivayam 741aa523e6 Establishing TLS connection through the handshake process is expensive and the fdbserver process can get easily saturated with doing repeated TLS handshakes with only a few hundreds of clients have bad certificate. Hence throttle the number of handshakes done on the server per client ip if it has a bad certificate. 2020-01-10 16:19:41 -08:00
Vishesh Yadav 598b2eaeb0 fdbrpc: Add warning when peer is unavailable for long time 2020-01-08 13:55:13 -08:00
Evan Tschannen 83ad9caf54 implemented a load balancing algorithm which evens out the number of requests processes by each proxy 2020-01-08 01:59:01 -08:00
mpilman 7a62d3b526 Changed failure monitor ping delay to 1 second 2019-12-11 11:23:24 -08:00
Evan Tschannen 3c769fcf60 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	versions.target
2019-11-22 15:39:19 -08:00
Evan Tschannen 27cb299d84 simulation can sometimes randomly hang or throw connection_failed, instead of always doing one or the other 2019-11-21 16:24:18 -08:00
Evan Tschannen 2727b91c46 simulation tests network connections failing due to errors instead of just hanging 2019-11-21 12:33:07 -08:00
Meng Xu c4d1e6e1a9 Trace:Severity:Include SevNoInfo to mute trace
Define SevFRMutationInfo to trace mutations in restore.
2019-11-04 16:18:40 -08:00
Evan Tschannen 3325980c03 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/OldTLogServer_6_0.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/WorkerInterface.actor.h
#	fdbserver/worker.actor.cpp
#	versions.target
2019-10-24 17:38:15 -07:00
mpilman 325a8e4213 remove confusing USE_ODIRECT knob 2019-10-24 11:44:03 -07:00
mpilman f23392ec5a Don't use O_DIRECT in EIO by default 2019-10-24 11:39:55 -07:00
mpilman 7ad0e20e48 Added knob to disable O_DIRECT 2019-10-24 11:20:14 -07:00
mpilman f41f19b5f6 Introduced knob to set eio parallelism 2019-10-24 11:20:14 -07:00
Evan Tschannen 688940b685 merge 6.2 into master 2019-10-21 11:43:46 -07:00
Evan Tschannen 42b7acf7b7
Merge pull request #2202 from etschannen/feature-share-mutations
Backup and DR would not share mutations if started on different versions of FDB
2019-10-16 20:28:39 -07:00
Evan Tschannen 35ef9b32de fix: if establishing a TLS connection took longer than 10ms, we could spend all our CPU establishing new connections instead of pinging to maintain existing connections, leading to an infinite loop 2019-10-16 17:26:01 -07:00
Evan Tschannen b88b1614c1 bool knobs would improperly report a SevWarnAlways that they were unrecognized 2019-10-01 18:53:04 -07:00
Vishesh Yadav e005ef8e9a
Merge pull request #2030 from kaomakino/kaomakino/quickack
TCP_QUICKACK test
2019-09-10 17:06:12 -07:00
Kao Makino e5bbaa293a Add FLOW_TCP_NODELAY and FLOW_TCP_QUICKACK (Linux-only) knobs 2019-09-10 20:13:15 +00:00
Vishesh Yadav 8a54a57420
Merge pull request #2026 from alexmiller-apple/knobs-for-network-test
Make the networktest payload size a knob so that it can be changed.
2019-09-10 00:11:42 -07:00
Alex Miller b4de920da6 Knobs style is to use `e` or shift instead of writing long numbers. 2019-08-21 18:39:02 -07:00
Alex Miller 49c623826f Make the networktest payload size a knob so that it can be changed. 2019-08-21 18:37:50 -07:00
Evan Tschannen 41b908752e increased move keys parallelism to be less of a decrease just in case lowering this could effect normal data distribution
raised target durability lag versions to give more time for batch limiting to come into play before this limit is hit
changed max_bad_options to better reflect the name
2019-08-21 14:55:21 -07:00
Evan Tschannen 54282288cb disabled zone_id load balancing, because it can cause hot read shards 2019-08-19 14:04:21 -07:00
Evan Tschannen 51cedd24c8 load balance will send reads to remote servers if more than one alternative is failed or overloaded 2019-08-19 13:59:49 -07:00
Evan Tschannen da8163fd5a allow one requests every second to skip there all_alteratives_failed delay, because if a client has a timeout longer than the delay we will never invalidate the key servers cache 2019-08-09 13:03:40 -07:00
Evan Tschannen 36b7b98d3c Buggified the idle_timeout so that it happens in simulation 2019-08-09 12:35:55 -07:00
Evan Tschannen 5f7d3498ea The delay for all_alteratives_failed can scale all the way up to 30.0 at a much slow time ratio 2019-08-09 12:35:19 -07:00
mpilman 370ba8b841 Remove --object-serializer flag from executables 2019-08-06 09:25:40 -07:00
Evan Tschannen 2d136af2bd bool knobs can now be set with the words “true” or “false” instead of just a number 2019-07-30 18:21:24 -07:00
A.J. Beamon 94be9560ea Add cache_memory parameter to fdbserver to control the size of the (4K) page cache. Change the default slighty from 2000 MiB to 2GiB. 2019-07-23 15:05:21 -07:00
Evan Tschannen 5d3e69b6fc
Merge pull request #1820 from fzhjon/load-balance-locality
Introduced a knob that can turn locality on/off
2019-07-16 16:40:43 -07:00
Jon Fu 7d37040725 split the single knob into two for finer-grained control 2019-07-16 12:46:02 -07:00
A.J. Beamon d5051b08dd Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated. 2019-07-12 16:12:35 -07:00
Jon Fu 19a91765f6 introduced a knob that can turn locality on/off 2019-07-09 16:16:15 -07:00
Vishesh Yadav 983343978e fdbrpc: ConnectionMonitor should close unreferenced after delay
Potentially for cases, where it goes up to 1 immediately.
2019-07-09 14:24:16 -07:00
Vishesh Yadav 1f9c80f633 fdbrpc: Instead of tracking last sent data, track last sent non-ping data
* This will allow client to continue monitoring peer connections while
connection stays open, so that there is no period of "uncertainity"
without previous no-monitoring approach.

* Use multiplier for incoming connection idle timeout

* Update idle connection timeout values and leaked connection timeout in
simulator.
2019-07-09 14:24:16 -07:00
Vishesh Yadav 867986cdea fdbrpc: Reduced connection monitoring from clients
This patch does two changes to connection monitoring:

1. Connection monitoring at client side will check if the connection
has been stayed idle for some time. If connection is unused for a
while, we close the connection. There is some weirdness involved here
as ping messages are by themselves are connection traffic. We get over
this by making it two-phase process, first being checking idle
reliable traffic, followed by disabling pings and then checking for
idle unreliable traffic.

2. Connection monitoring of clients from server will no longer send
pings to clients. Instead, it keep monitor the received bytes and
close after certain period of inactivity.
2019-07-09 14:24:16 -07:00
Jingyu Zhou e6ff23c420 Allow next packet size to be read when receiving data
For large packet, allocate sizeof(uint32_t) more bytes for next packet size.
Also add knob MIN_PACKET_BUFFER_FREE_BYTES, which is used to trigger allocation
of a new arena when free bytes are lower than this threshold.
2019-06-25 10:18:56 -07:00
Jingyu Zhou c2c82c89a5 Remove knob DEFAULT_PACKET_BUFFER_BYTES 2019-06-25 10:18:56 -07:00
Jingyu Zhou 2363326ecb Add knobs for various packet buffer sizes 2019-06-25 10:18:56 -07:00
A.J. Beamon 603721e125 Merge branch 'master' into thread-safe-random-number-generation
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/genericactors.actor.cpp
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	flow/ActorCollection.actor.cpp
#	flow/Net2.actor.cpp
#	flow/Trace.cpp
#	flow/flow.cpp
2019-05-23 08:35:47 -07:00