4bee976b37
completely ignore messages to unknown endpoint
2021-03-12 09:35:53 -07:00
a76c7b9754
Fix several memory leaks and a thread leak
2021-03-11 23:58:32 +00:00
d7252da951
clang-format: Fix the TEST() macros which require comments in line
2021-03-10 16:50:53 -08:00
2cd3f45fd6
Merge remote-tracking branch 'apple/release-6.3' into master-format-final
...
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
2021-03-10 16:50:44 -08:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
2bb4f2e59f
Merge branch 'release-6.3-pre-format' into master-format
...
This merges release-6.3 branch right before it was fully formatted.
There were quite a few conflicts that are resolved here. CoroFlow had
a check for OOM errors introduced in 6.3, but didn't seem applicable in
the new implmentation which seems to use boost.
2021-03-10 09:37:41 -08:00
36bf68ed2e
Merge branch 'release-6.3' into clang_format_the_world/6.3
2021-03-08 14:40:52 -08:00
a7ccb6669d
Change Net2::connect to only call initTLS for tls
2021-03-08 16:27:04 +00:00
8a8c488ede
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-05 18:13:38 -06:00
23b85256ae
Change default to 64, and only start threads if using TLS
2021-03-04 20:52:46 +00:00
cc7fed6ed4
Change default for TLS_SERVER_HANDSHAKE_THREADS to 0
2021-03-04 20:52:44 +00:00
79cec09255
Apply clang-tidy's performance-inefficient-vector-operation fix
...
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.
$ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
4806299821
Merge pull request #4382 from sfc-gh-nwijetunga/optimize-watches-server
...
Optimize Watches Server
2021-03-02 14:25:21 -07:00
a03b5f5426
Merge pull request #4349 from sfc-gh-ljoswiak/fixes/log-file-tracing
...
Stop running log file tracer in simulation
2021-03-02 14:00:00 -07:00
d676cb32e1
Separate tracers that will run in simulation from those that won't
2021-03-02 10:35:37 -08:00
4adce4eb83
Limit named thread support to Linux and add a comment documenting that.
2021-03-01 20:59:25 +00:00
179dea5a1b
Name the RocksDB background threads
2021-03-01 20:35:55 +00:00
2d84e5e1f1
Merge pull request #4254 from sfc-gh-tclinkenbeard/capture-forked-process-output
...
Capture forked process output in traces
2021-02-24 14:34:24 -07:00
6c65b0e8e0
inital commit
2021-02-22 20:58:59 -05:00
4a6ec52e0d
Merge pull request #4242 from sfc-gh-mpilman/features/coroutine2
...
Replace libcoro with boost::coroutine2
2021-02-17 15:45:19 -08:00
7a40190500
Don't test log file tracing in simulation
2021-02-17 09:50:46 -08:00
f514194aec
Merge remote-tracking branch 'origin/master' into features/coroutine2
2021-02-17 09:03:07 -07:00
4657db6705
Add comment warning against relying on TracerType order
2021-02-16 19:09:24 -08:00
f9c8674bc9
Use auto
2021-02-16 16:48:51 -08:00
30d2d9ff82
Remove async tracer
2021-02-16 16:36:33 -08:00
76a4293d1d
Merge branch 'release-6.3'
...
# Conflicts:
# fdbserver/fdbserver.actor.cpp
2021-02-15 02:13:06 -08:00
f4c9b88908
Merge branch 'release-6.2' into release-6.3
...
# Conflicts:
# cmake/CompileBoost.cmake
# fdbserver/DataDistribution.actor.cpp
# fdbserver/fdbserver.actor.cpp
# flow/Error.cpp
2021-02-15 02:05:03 -08:00
dd0e692f2a
Log std::exception::what when creating an unknown_error
...
unknown_error usually gets created when an exception of some type other
than `Error` is thrown. If the current exception is a std::exception
(and it usually is), we can log more information about it - which is
(incredibly!) helpful for figuring out what the heck happened.
2021-02-13 17:42:11 +00:00
4e184fe236
Fix memory errors
2021-02-11 02:58:21 +00:00
53739c9da5
Merge pull request #4241 from sfc-gh-clin/add-c-function-snapshot
...
Add c function for snapshot
2021-02-09 15:49:20 -08:00
a56184c0b7
log TLSVerifyFailure as a warning (severity 20) since it generally indicates a miconfiguration or important network problem
2021-02-08 22:01:48 +00:00
601a548cfe
Revert "Move new config to config.h.cmake"
...
This reverts commit 2951ebb4e7
2021-02-08 14:01:02 -08:00
2951ebb4e7
Move new config to config.h.cmake
2021-02-08 13:34:17 -08:00
aaf0a9aa7b
Merge branch 'release-6.3' into merge-release-6.3-into-master
...
# Conflicts:
# build/docker-compose.yaml
# cmake/ConfigureCompiler.cmake
# fdbclient/FileBackupAgent.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/IAsyncFile.h
# fdbrpc/IRateControl.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
449e4b98b4
Attempt to fix a build warning that's now blocking the compile due to werror.
2021-02-08 12:06:51 -08:00
67e783acf8
Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
...
# Conflicts:
# cmake/CompileBoost.cmake
# cmake/FDBComponents.cmake
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/simulator.h
# fdbserver/KeyValueStoreSQLite.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/storageserver.actor.cpp
# flow/Knobs.h
# flow/network.h
2021-02-08 09:20:28 -08:00
a85458dd1e
Merge branch 'master' of github.com:apple/foundationdb into add-c-function-snapshot
2021-02-05 14:14:52 -08:00
97925de9d4
Update fdb_database_create_snapshot to take a UID as input
2021-02-05 14:14:38 -08:00
5e80e16802
Revert "CDs reject requests from external cluster or requests need to be forwarded based on an old forward request."
2021-02-05 13:05:33 -08:00
8091d8179d
Merge pull request #4252 from dongxinEric/feature/4251/cds-reject-requests-from-other-cluster
...
CDs reject requests from external cluster or requests need to be forwarded based on an old forward request.
2021-02-04 17:33:07 -08:00
db82da5bc2
Only link to jemalloc in fdbserver
...
Including jemalloc in libfdb_c seems to be problematic, since it's
apparently not recommended to dlopen jemalloc:
https://github.com/jemalloc/jemalloc/issues/937#issuecomment-311242450
2021-02-05 01:30:31 +00:00
c017b51d66
Fix typo
2021-02-04 13:35:40 -08:00
e4a55908ff
Merge pull request #4222 from sfc-gh-mpilman/features/jemalloc
...
Replace standard malloc with jemalloc
2021-02-04 12:47:29 -08:00
7b489da13b
introduce USE_JEMALLOC and some refactoring
2021-02-04 11:53:28 -07:00
1db4b73215
Merge pull request #4243 from sfc-gh-etschannen/feature-storage-io-timeout
...
throw an io_timeout if it takes more than 2 minutes to commit on the storage server
2021-02-03 18:10:16 -08:00
675e567305
Merge pull request #4218 from sfc-gh-etschannen/feature-low-priority-reads-6.2
...
Reduce read priority when durability lag is high
2021-02-03 18:09:39 -08:00
068647ba1b
updated naming and comments
2021-02-03 14:24:39 -08:00
c4b622e7b8
added a comment and updated a variable name
2021-02-03 13:58:04 -08:00
a0a71ff7f9
simplified the algorithm based on testing
2021-02-02 17:53:07 -08:00
6f0e2dcdd0
Merge pull request #4268 from sfc-gh-clin/add-knob-for-IO-size
...
Add option to define I/O size
2021-02-02 16:22:24 -08:00
593165679a
Remove unnecessary header
2021-02-02 12:42:46 -08:00
17671604c0
Add option to define I/O size
2021-02-02 12:39:05 -08:00
090eee33e7
rearrange include ordering
2021-02-02 14:58:51 -05:00
d10b452758
include <errno.h> instead of <sys/errno.h>
2021-02-02 14:55:02 -05:00
761d9991a3
change to SevWarn instead of SevError
2021-02-02 14:47:19 -05:00
5a28d783b0
add readable description of error to trace event
2021-02-02 14:34:56 -05:00
5fa7c73810
add tracing to fileopen failure before throwing error for more info
2021-02-02 14:30:12 -05:00
534fa45291
Merge branch 'master' of github.com:apple/foundationdb into features/coroutine2
2021-02-01 16:31:16 -07:00
c25a3f11c3
Merge pull request #4259 from sfc-gh-satherton/ratelimit-6.2
...
Backport AFCache write rate control to 6.2
2021-01-31 20:55:16 -07:00
04b06c0c3c
Merge remote-tracking branch 'origin/master' into capture-forked-process-output
2021-01-31 16:14:38 -08:00
e4b953ab38
Merge pull request #4229 from sfc-gh-satherton/afcache-write-limit
...
AsyncFileCached RateControl support and bug fixes
2021-01-31 15:13:03 -07:00
97781bde76
Removed stray knob added unintentionally in a merge conflict.
2021-01-30 18:35:16 -08:00
b0fd6bfbcf
More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent.
2021-01-30 18:18:16 -08:00
52c2695f59
Bug fixes involving use of AFCached rate control in simulation.
2021-01-30 18:13:44 -08:00
e56fe02a25
Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit
2021-01-30 18:11:54 -08:00
14af5857ed
Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why.
2021-01-30 18:06:05 -08:00
e72d486215
Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments.
2021-01-30 18:06:04 -08:00
ff0667b28d
winbuild: define BOOST_USE_WINDOWS_H, WIN32_LEAN_AND_MEAN and NOMINMAX to fix various macro expansions issues
...
- NOMINMAX to fix confusion between std::max(..) and max(..) macro redefinition
- BOOST_USE_WINDOWS_H to prevent boost from redefining win32 APIs
- WIN32_LEAN_AND_MEAN to fix include ordering issues with winsock.h
2021-01-30 15:15:02 +01:00
9f837a3b84
Coordinators will not serve following requests after this change:
...
- A request that needs to be forwarded after the forward was set for a long(configured via Knobs) time.
- A request that has a different connection string than the CD's local connection string.
Simulation has been made to work with this change.
2021-01-29 13:24:45 -08:00
4f3da7c93b
also kill tlogs if a commit takes too long
2021-01-29 12:18:06 -08:00
acac02587d
Trace output from forked processes
2021-01-29 01:31:26 -08:00
301e25334d
fixed asan build
2021-01-28 09:01:40 -07:00
7ce0cca687
Merge branch 'release-6.3' of https://github.com/apple/foundationdb into afcache-write-limit
2021-01-28 07:27:32 -08:00
7c64e3943b
compile boost for sanitizers
2021-01-27 19:13:02 -07:00
75b24da109
fixed cast
2021-01-27 14:12:18 -08:00
3e2a0a5264
throw an io_timeout if it takes more than 2 minutes to commit on the storage server
2021-01-27 11:50:42 -08:00
989fee908a
More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent.
2021-01-27 00:01:33 -08:00
06a52449d1
typo
2021-01-26 17:38:03 -07:00
8dcded849e
Update flow/CMakeLists.txt
2021-01-26 17:37:17 -07:00
341069fcce
WIP asan on boost coroutine2
2021-01-26 17:10:13 -07:00
d718e7baf3
Merge pull request #4179 from sfc-gh-tclinkenbeard/run-minio-joshua
...
Support Testing S3BlobStore backup in simulation
2021-01-26 14:39:57 -08:00
0207f89904
don't use jemalloc with valgrind, asan, etc
2021-01-26 11:50:40 -07:00
7558cd8e06
Merge remote-tracking branch 'origin/master' into simplify-global-knobs
2021-01-26 10:00:25 -08:00
564c70acb4
Merge branch 'master' into anoyes/modernize-use-override
2021-01-25 18:26:18 -08:00
4ee97c0784
Use clang-tidy to automatically fix missing overrides
...
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.
Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.
- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
0ef44739ea
Fix OPEN_FOR_IDE build in preparation for using clang-tidy
2021-01-26 02:04:11 +00:00
17962712fb
Merge pull request #4227 from sfc-gh-tclinkenbeard/encapsulate-packetwriter
...
Encapsulate PacketWriter methods
2021-01-25 15:08:46 -08:00
5bf5545b87
Merge pull request #4236 from sfc-gh-anoyes/anoyes/merge-6.3-to-master
...
Merge 6.3 to master
2021-01-25 15:53:43 -07:00
d346c4955d
Merge pull request #4235 from sfc-gh-kmakino/kaomakino/aarch64_update
...
Fix minor issues for aarch64
2021-01-25 15:20:08 -07:00
29976f0b41
Fix minor issues for aarch64
2021-01-25 12:10:57 -08:00
adde95bea5
Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
...
Commit merge conflicts for now
2021-01-25 18:55:24 +00:00
c3775ede37
add override keyword properly to squash compiler warnings
2021-01-25 13:24:15 -05:00
0d32bd0c44
Bug fixes involving use of AFCached rate control in simulation.
2021-01-24 01:50:09 -08:00
51cbc494f1
Merge remote-tracking branch 'origin/master' into run-minio-joshua
2021-01-23 11:31:19 -08:00
b88a47f7db
Encapsulation PacketWriter methods
2021-01-22 18:48:23 -08:00
5f96c4b721
Add override keyword to DummyThreadPool methods
2021-01-22 13:19:29 -08:00
589eaf54d7
Switch to BoringSSL by default
2021-01-22 20:39:12 +00:00
a09acd321e
update knobs default values for testing
2021-01-22 11:45:37 -08:00
18f7ee22c2
Merge branch 'release-6.3' of github.com:apple/foundationdb into add-throttling-on-AsyncfileCached
2021-01-22 11:44:29 -08:00
4dd9c4889e
Revert "Make Profiler::active_profiler a unique pointer" and add comment
2021-01-21 14:32:46 -08:00
Markus Pilman