Commit Graph

1460 Commits

Author SHA1 Message Date
Andrew Noyes 6bffbdf7e3
Revert "Actually close files in simulation" 2021-05-04 15:38:24 -07:00
sfc-gh-tclinkenbeard d56906cd54 Addressed review comments 2021-05-03 15:26:27 -07:00
sfc-gh-tclinkenbeard e5d6c5ed17 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-05-03 14:46:19 -07:00
Jingyu Zhou 1c92588cca
Merge pull request #4562 from sfc-gh-mpilman/bugfixes/simulator-close-files
Actually close files in simulation
2021-05-03 13:47:44 -07:00
Jingyu Zhou d49e0091ce
Merge pull request #4727 from sfc-gh-etschannen/fix-rewrite-bme
Simulation could still stall writes for 10 seconds even when speedUpSimulation was on
2021-05-03 13:37:04 -07:00
Lukas Joswiak 8dcd779fc4
Merge branch 'master' into features/actor-lineage 2021-05-02 14:11:42 -07:00
Lukas Joswiak cf4218dfd1 Fixes simulation failures
Fixes the following issues:

1. Use the right index when initializing the WriteOnlySet's vector of
   atomics. Also switch to std::atomic_init to initialize each atomic in
   the vector (cannot default construct the atomics in the vector
   because std::atomic does not have a copy constructor).
2. Add failure check for when items cannot be inserted into the
   WriteOnlySet due to capacity constraints. This situation occurs when
   `copy` is not called on the WriteOnlySet, such as when sampling is
   disabled. The `copy` function is what clears the WriteOnlySet.
3. Remove a global config feature I added to update the ClientDBInfo
   object used by the global config listener function. This needs more
   investigation, but the effect of this change could be that global
   config changes are not correctly recognized on fdbserver processes.
4. Add various ASSERTs to verify data in WriteOnlySet.
2021-05-01 15:26:28 -07:00
Andrew Noyes 904a39e473
Merge pull request #4667 from sfc-gh-ajbeamon/feature-mvc-monitor-protocol-version
Use fewer connections in the multi-version client
2021-04-28 14:13:17 -07:00
Evan Tschannen 65fcf4014e Fix: simulation could still stall writes for 10 seconds even when speedUpSimulation was on
Fix: disable connection failures in simulation when there are too many generations outstanding
2021-04-28 12:41:48 -07:00
A.J. Beamon 7158bfa82e Merge branch 'master' into load-balance-remove-make-request-actor 2021-04-28 10:31:41 -07:00
A.J. Beamon 135cc9c69a Make parameter const& 2021-04-28 10:30:30 -07:00
Markus Pilman 4fab2ecd30 Merge remote-tracking branch 'origin/master' into features/actor-lineage 2021-04-28 09:20:54 -06:00
A.J. Beamon 9009780aa8 Fix bug that could cause the server to crash when an old client connected 2021-04-27 11:15:16 -07:00
Evan Tschannen a02da36e85 fixed the problem with the GrvProxyClass the proper way my keeping the enum the same between versions 2021-04-26 18:45:44 -07:00
Evan Tschannen 451609e6be code cleanup 2021-04-26 10:16:18 -07:00
A.J. Beamon a794fca932 Support 5.0 (and earlier) client versions by adding GRV probing for old versions. Update the C bindings implementation of get_server_protocol to convert the ProtocolVersion object into a uint64_t. Rename a misleading protocol version alias. 2021-04-23 15:00:21 -07:00
sfc-gh-tclinkenbeard d6fa06afdd Add IConfigTransaction::getRange (not yet tested) 2021-04-23 11:39:26 -07:00
sfc-gh-tclinkenbeard 050eb079bd Add ConfigFollowerCompactRequest 2021-04-21 12:18:52 -07:00
A.J. Beamon 28f8a2716e For old incompatible connections, set the correct protocol version on the version async var 2021-04-21 11:54:05 -07:00
Evan Tschannen e18c9961b4 rewrote tlog recruitment logic so that it is deterministic, to prevent better master exists from triggering spuriously 2021-04-21 00:22:33 -07:00
A.J. Beamon eaaae2e16d Merge branch master into 'feature-mvc-monitor-protocol-version' 2021-04-20 15:07:02 -07:00
Markus Pilman 7307750e5e Merge remote-tracking branch 'origin/master' into features/actor-lineage 2021-04-19 11:29:52 -06:00
sfc-gh-tclinkenbeard f54f082159 Build interfaces for full config update pipeline 2021-04-16 17:58:00 -07:00
Steve Atherton db610355cf Keep simulated disk write delay high until speedUp is set. 2021-04-16 14:19:37 -07:00
Lukas Joswiak 551268b0f2 Add well known endpoint for worker communication 2021-04-15 13:50:50 -07:00
A.J. Beamon b2d6930103 The multi-version client monitors the cluster's protocol version and only activates the client library that can connect. 2021-04-15 11:45:14 -07:00
sfc-gh-tclinkenbeard 18f17a4ea2 First draft of config-db 2021-04-14 22:06:37 -07:00
Steve Atherton de236894cb Merge commit 'eeee15f524ff769248495d70efa0501170fb5ea2' into correctness-fix 2021-04-14 11:50:49 -07:00
Steve Atherton 1958fde5c6 Added parentheses for clarity. 2021-04-13 20:49:04 -07:00
Steve Atherton f74748ebac Applied clang-format. 2021-04-13 20:43:12 -07:00
Steve Atherton 9475b6a5dd Correctness fix, prevent AsyncFileNonDurable from always making file writes take up to 5 seconds. 2021-04-13 20:15:19 -07:00
Markus Pilman ed8d43cb87
Merge pull request #5 from sfc-gh-ljoswiak/features/network-actors
Sample actors waiting on network
2021-04-09 15:07:53 -06:00
Evan Tschannen a90c26f1d0 The master, proxies, and resolver all need to have the same machine class fitness function besides best fit to ensure recruitment is deterministic
if the first GRV proxy or resolver is forced to share a process, it should prefer to share with the commit proxy so that the commit proxy has more potential options it can share with
2021-04-08 14:29:12 -07:00
A.J. Beamon 931499fb3f Merge branch 'master' into load-balance-remove-make-request-actor 2021-04-08 09:11:35 -07:00
A.J. Beamon 040ba0c587 Rearrange things no that the backoff delay has no impact unless it's needed. 2021-04-07 15:23:50 -07:00
Lukas Joswiak 433872e17d Sample actors waiting on network 2021-04-06 17:28:28 -07:00
Markus Pilman 9bcde529f8
Merge pull request #4 from sfc-gh-ljoswiak/features/current-actor
Sample running actor
2021-04-05 11:36:48 -06:00
Markus Pilman 41d1aee609 delete dead code 2021-04-01 14:06:13 -06:00
Markus Pilman 1987682e1e Merge remote-tracking branch 'origin/master' into bugfixes/simulator-close-files 2021-04-01 11:14:28 -06:00
Markus Pilman dc35af3760 Merge remote-tracking branch 'origin/master' into features/actor-lineage 2021-04-01 11:01:31 -06:00
Markus Pilman ce8fce94c8
Merge pull request #4596 from sfc-gh-etschannen/fix-starting-config
Fixed simulations which timeout setting starting configuration
2021-03-31 10:31:28 -06:00
Evan Tschannen e774262046 fix: g_simulator.disableRemote did not contain the rest of the configuration 2021-03-30 21:11:26 -07:00
sfc-gh-tclinkenbeard d4191899d9 Add comments for AsyncFileEncrypted changes 2021-03-28 22:14:37 -07:00
sfc-gh-tclinkenbeard 82420e5572 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-03-27 21:02:19 -07:00
Markus Pilman 1033db9fba Revert change 2021-03-25 14:00:07 -06:00
Markus Pilman 1385a776da only remove files from the open map if they have no modifications in flight 2021-03-25 13:22:29 -06:00
Markus Pilman b51e4aa590 handle file renames properly 2021-03-24 19:57:24 -06:00
Markus Pilman 6a344ddeab fix typo 2021-03-24 16:56:11 -06:00
Markus Pilman f7d3b31ef8 Actually close files in simulation 2021-03-24 16:27:35 -06:00
A.J. Beamon 36f4c17ef1 Reduce the number of actor calls in load balancing to improve performance. 2021-03-24 15:04:45 -07:00
Lukas Joswiak 2dfd420882 Add sampling profiler thread 2021-03-24 14:52:42 -07:00
A.J. Beamon f1166f2bf6
Merge pull request #4545 from sfc-gh-anoyes/anoyes/fix-truncate-simulation
In simulation, fix treatment of extending a file with truncate as a "pending modification"
2021-03-24 12:35:32 -07:00
Andrew Noyes eb80321ea3 Attempt to fix windows build 2021-03-24 18:48:10 +00:00
Andrew Noyes c186d363c6 Add unit test 2021-03-24 17:32:07 +00:00
Andrew Noyes 170c197c4c Truncate marks everything after size modified 2021-03-23 21:07:12 +00:00
Andrew Noyes e83de2b799 Fix bug: minSizeAfterPendingModifications needs to be maxed 2021-03-23 21:00:21 +00:00
Evan Tschannen a893309112 Opening a file with OPEN_ATOMIC_WRITE_AND_CREATE should create a new file handle, so that if a file with the same name is still in use, operations against it will not happen to the new file. This can happen when the disk queue replaces a file. 2021-03-23 13:47:46 -07:00
Andrew Noyes 0daf6cf632 Consider extending a file with truncate as a "pending modification"
Before this, truncating and reading concurrently could cause to read
uninitialized memory. So could truncating then reading, since the effect
of the truncate in the actual file was allowed to be delayed. Now reads
will wait for a truncate that extends the file to complete if they
intersect the newly-zeroed region.
2021-03-23 19:44:36 +00:00
sfc-gh-tclinkenbeard a0c49234b2 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-03-19 20:47:53 -07:00
Evan Tschannen e1ebe2f487 clang-format 2021-03-19 13:17:39 -07:00
Evan Tschannen 78e81e514a fix: OPEN_ATOMIC_WRITE_AND_CREATE did not create a new file handle for the replacement file, so when the disk queue calls replaceFile, truncates against the old file handle will happen on the new file resulting in corruption 2021-03-19 13:11:52 -07:00
Evan Tschannen d7491a8f30 removed logging 2021-03-19 13:08:22 -07:00
Evan Tschannen 22f5033c6a add filehandle 2021-03-18 23:29:22 -07:00
Evan Tschannen 335b59eafe log all file size changes 2021-03-18 22:52:19 -07:00
Evan Tschannen b1ac27cec1 attempt to avoid having batching consume extra simulated time 2021-03-18 13:05:29 -07:00
Evan Tschannen 00f114b976 update mutex usage 2021-03-18 11:18:07 -07:00
Evan Tschannen c53dd4a46f check isStopped between each task 2021-03-18 10:49:24 -07:00
Evan Tschannen 488fe6f008 give more time to cleanup tasks when rebooting 2021-03-17 21:50:19 -07:00
Evan Tschannen c7ef8377d2 add back in the machine check 2021-03-17 18:48:56 -07:00
Evan Tschannen 67967b5272 slightly adjusted seconds check 2021-03-17 18:06:42 -07:00
Evan Tschannen 06fe6917ab switch to a separate queue for ordered tasks 2021-03-17 17:45:04 -07:00
Evan Tschannen c44035a27b ordered tasks in a batch are executed first and in their creation order 2021-03-17 17:07:25 -07:00
Evan Tschannen 5af2962d04 ordered tasks are executed at the highest priority instead of disabling batching 2021-03-17 16:44:49 -07:00
Evan Tschannen bf4fcbdb5e fix compile error 2021-03-17 16:31:44 -07:00
Evan Tschannen ec4c29361c do not allow batching with tasks that must be ordered 2021-03-17 16:29:33 -07:00
Evan Tschannen 3233fa339e temporarily disable stable reordering to make sure the result of the PR is correctness clean 2021-03-17 16:02:07 -07:00
Evan Tschannen bf75ee2cc6 fixed formatting 2021-03-17 15:36:25 -07:00
Evan Tschannen 151018a36a do not reorder on the machine process 2021-03-17 15:35:34 -07:00
Evan Tschannen a7178b3e5f changed the logic for when stable can be randomized 2021-03-17 15:01:33 -07:00
Evan Tschannen e16c4d71f1 simulation framework delays still need to be ordered 2021-03-17 14:34:22 -07:00
Evan Tschannen fb883b482d avoid possible collisions between stable numbers 2021-03-17 14:09:46 -07:00
Evan Tschannen 7702f0151f randomize execution order of tasks with the same priority 2021-03-17 13:59:00 -07:00
Evan Tschannen 514e80d8a5 sort by time after priority to better match Net2 2021-03-17 13:45:04 -07:00
Evan Tschannen 3275cd7b94 fix spacing 2021-03-17 13:19:45 -07:00
Evan Tschannen 524662e871 use the stable value from instead task instead of stable_sort 2021-03-17 13:18:28 -07:00
Evan Tschannen 056764462d enable randLog 2021-03-17 12:15:20 -07:00
Evan Tschannen a096d1f403 switch to stable_sort 2021-03-17 11:38:23 -07:00
Evan Tschannen c390cf7c6c removed an assert 2021-03-17 11:05:02 -07:00
Evan Tschannen 2ea3c971d1 execute tasks within a batch in priority order 2021-03-17 11:01:06 -07:00
Markus Pilman eb036b7b02 Merge remote-tracking branch 'origin/master' into features/actor-lineage 2021-03-17 11:59:54 -06:00
Evan Tschannen a3c48772e1 Merge branch 'master' into feature-sim-time-batching 2021-03-17 09:55:07 -07:00
Evan Tschannen b3301fe361 fix: do not allow actualTime to go backward 2021-03-16 19:00:37 -07:00
Evan Tschannen 394e43a18d updated how simulation does the batch to better match the real runloop 2021-03-16 18:50:47 -07:00
Evan Tschannen 9cf59b44be do not batch with tasks created as a result of other tasks in the same batch 2021-03-16 17:27:14 -07:00
Evan Tschannen 3a218e4b32 limit the number of tasks that can be executed with the same now() 2021-03-16 16:55:53 -07:00
Andrew Noyes e7abffbe71
Merge pull request #4494 from sfc-gh-etschannen/feature-fix-sim-reliable
Fixed a bug in isReliable() of a simulated process
2021-03-16 10:19:28 -07:00
A.J. Beamon 25c4880ebe Merge branch 'release-6.3' into merge-release-6.3-into-master (temporarily discard all changes to BackupContainer.actor.cpp)
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbserver/Knobs.h
2021-03-15 16:41:04 -07:00
Evan Tschannen 13242d8b35 the sim2 runloop now updates time in batches so that multiple tasks can execute with the same now() 2021-03-15 12:33:43 -07:00
Evan Tschannen 6a372e3fc7 fixed a simulation bug where a process on an unreliable machine would be considered reliable by the simulator 2021-03-15 11:07:36 -07:00
Markus Pilman d0cc649ca2 fixed comment 2021-03-12 09:45:02 -07:00
A.J. Beamon 3b390168e8 Update fdbrpc/FlowTransport.actor.cpp 2021-03-12 09:40:47 -07:00
A.J. Beamon 7031b8f437 Update fdbrpc/FlowTransport.actor.cpp 2021-03-12 09:40:35 -07:00
Markus Pilman 3342d9438e address review comments 2021-03-12 09:39:22 -07:00
Markus Pilman bb53f49aa2 don't start deliver actor for unknown endpoint 2021-03-12 09:39:04 -07:00
Markus Pilman 4bee976b37 completely ignore messages to unknown endpoint 2021-03-12 09:35:53 -07:00
Vishesh Yadav 2cd3f45fd6 Merge remote-tracking branch 'apple/release-6.3' into master-format-final
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
2021-03-10 16:50:44 -08:00
Markus Pilman cfbaaa6001 Merge remote-tracking branch 'origin/release-6.2' into release-6.3 2021-03-10 14:30:47 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Vishesh Yadav 2bb4f2e59f Merge branch 'release-6.3-pre-format' into master-format
This merges release-6.3 branch right before it was fully formatted.
There were quite a few conflicts that are resolved here. CoroFlow had
a check for OOM errors introduced in 6.3, but didn't seem applicable in
the new implmentation which seems to use boost.
2021-03-10 09:37:41 -08:00
Steve Atherton e9c9bd5299 Merge commit 'e2d3ddee2cb4da0bba65346d29ad2212fbce16f0' into file-backup-async-rename
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbclient/Knobs.cpp
#	fdbrpc/AsyncFileNonDurable.actor.h
#	fdbrpc/sim2.actor.cpp
2021-03-08 02:00:40 -08:00
Steve Atherton 318b862aa3 Apply clang-format to backup changes. 2021-03-08 01:49:29 -08:00
Steve Atherton a397d8625f AsyncFileNonDurable now has a flag for whether or not to emulate AIO mode, so the previous behavior limits are preserved when this flag is set. AsyncFileDurable writes which are selected to be durable are again done in a single write, and the logic to handle non-aligned writes for non-AIO mode is more clear and commented. In simulation, backup files on local filesystems are written with random buffer sizes. Backup container unit test now uses more random file sizes and limits memory consumption for all files being written in parallel. 2021-03-07 22:21:10 -08:00
Steve Atherton 1d00c41385 Bug fixes: Avoid empty writes, avoid 0 length truncation, and make AsyncFileNonDurable support writes at arbitrary offsets and lengths. Increase randomness of IBackupFile appends in backup container unit test. 2021-03-05 17:45:36 -08:00
FDB Formatster 8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
FDB Formatster 0c214d71da apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 16:31:35 -06:00
Steve Atherton f33ed86210
Merge pull request #4420 from apple/release-6.2
Merge 6.2 into 6.3
2021-03-03 15:57:36 -08:00
Markus Pilman 08c98ec2a9 Unit test for rename function 2021-03-03 14:25:00 -07:00
Markus Pilman fd1a865737 fix botched include 2021-03-03 09:36:04 -07:00
Markus Pilman 7f758328c0 add renameFile to WinASIO 2021-03-03 09:35:19 -07:00
Markus Pilman 7cb529ddb3 address review comment 2021-03-02 17:34:30 -07:00
Markus Pilman a33bf68444 make sure rename code is actually called 2021-03-02 17:34:23 -07:00
Markus Pilman d2bd63d0cb fix includes 2021-03-02 17:05:27 -07:00
Markus Pilman cc47332478 Added an actor to allow for async file renames 2021-03-02 16:38:51 -07:00
Evan Tschannen 346a4e3ecd Merge branch 'release-6.3'
# Conflicts:
#	fdbcli/fdbcli.actor.cpp
#	fdbrpc/LoadBalance.actor.h
#	fdbrpc/MultiInterface.h
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/masterserver.actor.cpp
2021-03-01 18:52:06 -08:00
Evan Tschannen 3cd819df25
Merge pull request #4301 from sfc-gh-anoyes/anoyes/iterator-fix-6.3
Fix bugs turned up by _GLIBCXX_DEBUG
2021-03-01 13:53:00 -08:00
Andrew Noyes bec1cd15b9 Use default stack size for libeio threads 2021-03-01 16:41:09 +00:00
Zhe Wu 4f2ed263cc Update comment 2021-02-24 15:15:56 -08:00
Zhe Wu 78721530e6 Resolve merge conflict 2021-02-24 15:13:58 -08:00
Zhe Wu a17b17f896 add more clarification 2021-02-23 12:23:10 -08:00
Zhe Wu 25a7f87433 Revise documentation in QueueModel to clarify the meaning of 'outstanding' and 'penalty' 2021-02-23 12:20:05 -08:00
Zhe Wu 8ee7ee9e45 Apply clang-format to fdbrpc/QueueModel.h 2021-02-23 10:21:35 -08:00
Zhe Wu 309c6fddc9 Add documentation to client side load balancing algorithm 2021-02-22 15:29:18 -08:00
Meng Xu ef0bf2728e Merge branch 'release-6.3' into mengxu/ha-code
Resolve Conflicts:
	fdbserver/LogRouter.actor.cpp: Only conflicts at comments
2021-02-19 21:47:09 -08:00
Meng Xu 471a3489fb Resolve review comments and add trace fields to MasterRecoveryState 2021-02-17 14:44:14 -08:00
Markus Pilman d13e240496 use libcoroutine for Windows
The Windows CI still doesn't have boost context installed
so we use the old implementation for now
2021-02-17 09:23:41 -07:00
Markus Pilman f514194aec Merge remote-tracking branch 'origin/master' into features/coroutine2 2021-02-17 09:03:07 -07:00
Meng Xu 9122be4d81 Add comments to HA code and loadBalance code 2021-02-10 13:51:36 -08:00
A.J. Beamon e6d9ebd3d2
Merge pull request #4302 from sfc-gh-ajbeamon/make-ping-stable-interface
Make the ping enpoint a stable interface
2021-02-08 18:10:33 -08:00
A.J. Beamon 9f0f36c9a6 Make the ping enpoint a stable interface. Fix bug where we would could start scanning for packets before the connect packet was finished. 2021-02-08 17:10:33 -08:00
Andrew Noyes cb196daefe Fix bugs turned up by _GLIBCXX_DEBUG
Compiling with -D_GLIBCXX_DEBUG enables libstc++ "debug mode", where
additional debug information is tracked with iterators and reported if
iterators are misused. This turned up two bugs.

I threw in removing dead code and avoiding an unnecessary map lookup
while I was in the neighborhood.
2021-02-09 00:41:21 +00:00
A.J. Beamon aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
Markus Pilman 6e571ab87b Merge branch 'master' of github.com:apple/foundationdb into features/coroutine2 2021-02-08 10:25:29 -07:00
A.J. Beamon 67e783acf8 Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
# Conflicts:
#	cmake/CompileBoost.cmake
#	cmake/FDBComponents.cmake
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/storageserver.actor.cpp
#	flow/Knobs.h
#	flow/network.h
2021-02-08 09:20:28 -08:00
Steve Atherton 8547abbbe4 Bug fix: In simulation, KVStoreSQLite can outlive its process during a simulated reboot so its file references must be cleared explicitly during shutdown instead of implicitly during destruction. Re-enabled a useful trace event. Some cleanup, removed unnecessary member var initializer and removed unnecessary method. 2021-02-05 21:26:39 -08:00
Andrew Noyes 08997d8602 Fix bugs turned up by _GLIBCXX_DEBUG
Compiling with -D_GLIBCXX_DEBUG enables libstc++ "debug mode", where
additional debug information is tracked with iterators and reported if
iterators are misused. This turned up two bugs.

I threw in removing dead code and avoiding an unnecessary map lookup
while I was in the neighborhood.
2021-02-04 02:37:51 +00:00
sfc-gh-tclinkenbeard 7c0e331e07 Disable copying StreamCipher::Key 2021-02-03 18:13:39 -08:00
Meng Xu 1db4b73215
Merge pull request #4243 from sfc-gh-etschannen/feature-storage-io-timeout
throw an io_timeout if it takes more than 2 minutes to commit on the storage server
2021-02-03 18:10:16 -08:00
sfc-gh-tclinkenbeard 5be4df6f84 Move global key into StreamCipher 2021-02-03 17:39:59 -08:00
Chaoguang Lin 593165679a Remove unnecessary header 2021-02-02 12:42:46 -08:00
Chaoguang Lin 17671604c0 Add option to define I/O size 2021-02-02 12:39:05 -08:00
Markus Pilman 534fa45291 Merge branch 'master' of github.com:apple/foundationdb into features/coroutine2 2021-02-01 16:31:16 -07:00
sfc-gh-tclinkenbeard 5a0502cbf4 Improve SimExternalConnection::read performance 2021-01-31 16:08:03 -08:00
Steve Atherton b0fd6bfbcf More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-30 18:18:16 -08:00
Steve Atherton 52c2695f59 Bug fixes involving use of AFCached rate control in simulation. 2021-01-30 18:13:44 -08:00
Steve Atherton e56fe02a25 Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit 2021-01-30 18:11:54 -08:00
sfc-gh-tclinkenbeard 4563eeba5b Moved AsyncFileEncrypted::RandomCache implementation to cpp file 2021-01-30 00:12:05 -08:00
sfc-gh-tclinkenbeard bbf8230462 Use random eviction policy for cached AsyncFileEncrypted blocks 2021-01-29 23:57:24 -08:00
Evan Tschannen 80c302e5c9 removed assert 2021-01-28 17:52:06 -08:00
Evan Tschannen aa9d97bf22 validate that failDisk is being triggered in simulation 2021-01-28 12:34:25 -08:00
Evan Tschannen cf6a8e5df7 waitUntilDiskReady is not an actor 2021-01-27 14:36:02 -08:00
Evan Tschannen 36e4f82115 more complete support for simulated disk failures 2021-01-27 14:29:43 -08:00
Evan Tschannen f30ae07a2d fixed compile error 2021-01-27 14:10:36 -08:00
Evan Tschannen b9a4c7418a fixed typo 2021-01-27 14:10:06 -08:00
Evan Tschannen 3ee5831287 fault injection can now cause a disk to stop responding 2021-01-27 13:57:54 -08:00
sfc-gh-tclinkenbeard 511ec43e62 Merge remote-tracking branch 'origin/master' into encrypt-backup-files 2021-01-27 11:17:58 -08:00
Steve Atherton 989fee908a More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-27 00:01:33 -08:00
Markus Pilman d3a85b555a CoroFlow and remove libcoro 2021-01-26 17:23:17 -07:00
Steve Atherton d718e7baf3
Merge pull request #4179 from sfc-gh-tclinkenbeard/run-minio-joshua
Support Testing S3BlobStore backup in simulation
2021-01-26 14:39:57 -08:00
sfc-gh-tclinkenbeard 27314b111c Fix Windows build issues 2021-01-26 13:41:37 -08:00
Andrew Noyes 3560003c50 Fix missing override in fdbrpc/FlowTests.actor.cpp 2021-01-26 17:29:54 +00:00
sfc-gh-tclinkenbeard bb1dd21c22 Define BOOST_DATE_TIME_NO_LIB in sim2.actor.cpp 2021-01-25 18:06:55 -08:00
Andrew Noyes 4ee97c0784 Use clang-tidy to automatically fix missing overrides
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.

Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.

- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
sfc-gh-tclinkenbeard 88cc2f752a Define BOOST_DATE_TIME_NO_LIB in SimExternalConnection.actor.cpp 2021-01-25 16:23:36 -08:00
sfc-gh-tclinkenbeard 1612c44988 Disable StreamCipher for Windows 2021-01-25 12:36:35 -08:00
sfc-gh-tclinkenbeard 037279c843 Disable StreamCipher if TLS is disabled 2021-01-25 10:42:14 -08:00
Jon Fu c3775ede37 add override keyword properly to squash compiler warnings 2021-01-25 13:24:15 -05:00
sfc-gh-tclinkenbeard 88bc157bd0 Added AsyncFileEncrypted 2021-01-24 22:53:04 -08:00
Steve Atherton 0d32bd0c44 Bug fixes involving use of AFCached rate control in simulation. 2021-01-24 01:50:09 -08:00
sfc-gh-tclinkenbeard 51cbc494f1 Merge remote-tracking branch 'origin/master' into run-minio-joshua 2021-01-23 11:31:19 -08:00
sfc-gh-tclinkenbeard 82a8f1c763 Fix -Wreturn-type warning 2021-01-22 12:13:29 -08:00
sfc-gh-tclinkenbeard fb9b4b7626 Merge remote-tracking branch 'origin/master' into misc-changes 2021-01-21 10:45:50 -08:00
Lukas Joswiak 3f7e19a182 Add tags to spans to provide additional metadata 2021-01-20 13:57:36 -08:00
Lukas Joswiak 98e6ba6962 Add POSIX send functionality 2021-01-20 13:57:36 -08:00
Lukas Joswiak d2143f184a Add fast UDP tracer 2021-01-20 13:57:36 -08:00
Markus Pilman fb64902d5c Assign roles 2021-01-19 16:04:09 -07:00
Steve Atherton 523a8c08db Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit
# Conflicts:
#	flow/Knobs.h
2021-01-17 05:30:55 -08:00
sfc-gh-tclinkenbeard 191f215b9d Remove chrono dependency 2021-01-13 21:54:22 -08:00
Markus Pilman 8e93573aad Merge branch 'master' of github.com:apple/foundationdb into features/actor-lineage 2021-01-05 10:24:45 -07:00
sfc-gh-tclinkenbeard 61a29ecfc8 Merge remote-tracking branch 'origin/master' into run-minio-joshua 2021-01-01 09:38:36 -04:00
sfc-gh-tclinkenbeard 86c7c1e946 Fix IAsyncFileSystem method signatures 2020-12-28 01:57:42 -04:00
sfc-gh-tclinkenbeard 5bfa6cea98 Merge remote-tracking branch 'origin/master' into misc-changes 2020-12-26 20:47:00 -04:00
sfc-gh-tclinkenbeard 913401aaee Move SendBufferIterator out of network.h
This class is only needed for implementation in Net2.actor.cpp and
SimExternalConnection.actor.cpp, so this class should not be included
everywhere network.h is included
2020-12-26 16:30:00 -04:00
sfc-gh-tclinkenbeard 0b45c969b9 Remove invalid assertion from SimExternalConnection::read 2020-12-24 21:20:04 -04:00
sfc-gh-tclinkenbeard a7202a5b42 Use connectExernal in INetworkConnections::connect 2020-12-24 21:20:04 -04:00
sfc-gh-tclinkenbeard 1dfa2acb93 Reduce sleep in SimExternalConnection::write 2020-12-24 21:20:04 -04:00
sfc-gh-tclinkenbeard d9b5309f4f Add random delays to SimExternalConnection 2020-12-24 21:20:04 -04:00
sfc-gh-tclinkenbeard 917da45c3f Miscellaneous improvements to SimExternalConnection unit test 2020-12-24 21:20:04 -04:00
sfc-gh-tclinkenbeard 555c3d95fc Added fdbrpc/SimExternalClient unit test 2020-12-24 21:20:03 -04:00
sfc-gh-tclinkenbeard 201857541d Remove unnecessary copies from INetworkConnections interface 2020-12-24 21:20:03 -04:00
sfc-gh-tclinkenbeard 94d7c17978 Implemented SimExternalConnection::onReadable 2020-12-24 21:20:03 -04:00