Commit Graph

2226 Commits

Author SHA1 Message Date
Nim Wijetunga 17f9da6962 remove traces 2021-03-16 16:47:50 +00:00
Trevor Clinkenbeard ffb8e27f43
Merge pull request #4461 from sfc-gh-anoyes/anoyes/fix-thread-leak
Fix several leaks
2021-03-15 22:03:45 -07:00
Nim Wijetunga cc6e395664 atomic network busyness 2021-03-16 02:16:43 +00:00
Andrew Noyes 5f64a0942f Align FastAllocator memory to 4096 for size >= 4096 2021-03-16 00:44:35 +00:00
Nim Wijetunga f0d79b3d86 Inital implementation of network busyness 2021-03-15 23:23:56 +00:00
Evan Tschannen 13242d8b35 the sim2 runloop now updates time in batches so that multiple tasks can execute with the same now() 2021-03-15 12:33:43 -07:00
Trevor Clinkenbeard a82d6cf7d4
Merge pull request #4486 from johscheuer/fix-typo
Fix typo
2021-03-14 15:38:48 -07:00
Johannes M. Scheuermann 4aa98f5035 Fix typo 2021-03-14 19:30:35 +00:00
Andrew Noyes d09603b6b3 Explain FastAllocator<4096> special case 2021-03-13 00:36:29 +00:00
Andrew Noyes a63dc3cfd6 Inform LSAN that we have unaligned pointers 2021-03-13 00:33:47 +00:00
FDB Formatster 7867cd454e apply clang-format to flow/Platform.h 2021-03-12 15:16:33 -08:00
A.J. Beamon 74f427d317 Change the macro that forbids exit() calls to be a static assertion 2021-03-12 14:47:19 -08:00
Markus Pilman 4bee976b37 completely ignore messages to unknown endpoint 2021-03-12 09:35:53 -07:00
Andrew Noyes a76c7b9754 Fix several memory leaks and a thread leak 2021-03-11 23:58:32 +00:00
Vishesh Yadav d7252da951 clang-format: Fix the TEST() macros which require comments in line 2021-03-10 16:50:53 -08:00
Vishesh Yadav 2cd3f45fd6 Merge remote-tracking branch 'apple/release-6.3' into master-format-final
release-6.3 was recently merged, and there were two PRs which were
merged in between and got those changes in here. Hence, since all the
changes were in, discarded the incoming changes and accepted all
current.
2021-03-10 16:50:44 -08:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Vishesh Yadav 2bb4f2e59f Merge branch 'release-6.3-pre-format' into master-format
This merges release-6.3 branch right before it was fully formatted.
There were quite a few conflicts that are resolved here. CoroFlow had
a check for OOM errors introduced in 6.3, but didn't seem applicable in
the new implmentation which seems to use boost.
2021-03-10 09:37:41 -08:00
Russell Sears 36bf68ed2e
Merge branch 'release-6.3' into clang_format_the_world/6.3 2021-03-08 14:40:52 -08:00
Andrew Noyes a7ccb6669d Change Net2::connect to only call initTLS for tls 2021-03-08 16:27:04 +00:00
FDB Formatster 8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
Andrew Noyes 23b85256ae Change default to 64, and only start threads if using TLS 2021-03-04 20:52:46 +00:00
Andrew Noyes cc7fed6ed4 Change default for TLS_SERVER_HANDSHAKE_THREADS to 0 2021-03-04 20:52:44 +00:00
Andrew Noyes 79cec09255 Apply clang-tidy's performance-inefficient-vector-operation fix
I ran this command in my build directory after compiling with
OPEN_FOR_IDE. It took a few small tweaks to get it to compile, which is
outside the scope of this commit.

    $ python run-clang-tidy.py -j $(nproc) -checks='-*,performance-inefficient-vector-operation' -fix
2021-03-04 03:58:25 +00:00
Markus Pilman 4806299821
Merge pull request #4382 from sfc-gh-nwijetunga/optimize-watches-server
Optimize Watches Server
2021-03-02 14:25:21 -07:00
Markus Pilman a03b5f5426
Merge pull request #4349 from sfc-gh-ljoswiak/fixes/log-file-tracing
Stop running log file tracer in simulation
2021-03-02 14:00:00 -07:00
Lukas Joswiak d676cb32e1 Separate tracers that will run in simulation from those that won't 2021-03-02 10:35:37 -08:00
Daniel Smith 4adce4eb83 Limit named thread support to Linux and add a comment documenting that. 2021-03-01 20:59:25 +00:00
Daniel Smith 179dea5a1b Name the RocksDB background threads 2021-03-01 20:35:55 +00:00
Markus Pilman 2d84e5e1f1
Merge pull request #4254 from sfc-gh-tclinkenbeard/capture-forked-process-output
Capture forked process output in traces
2021-02-24 14:34:24 -07:00
Nim Wijetunga 6c65b0e8e0 inital commit 2021-02-22 20:58:59 -05:00
Andrew Noyes 4a6ec52e0d
Merge pull request #4242 from sfc-gh-mpilman/features/coroutine2
Replace libcoro with boost::coroutine2
2021-02-17 15:45:19 -08:00
Lukas Joswiak 7a40190500 Don't test log file tracing in simulation 2021-02-17 09:50:46 -08:00
Markus Pilman f514194aec Merge remote-tracking branch 'origin/master' into features/coroutine2 2021-02-17 09:03:07 -07:00
Lukas Joswiak 4657db6705 Add comment warning against relying on TracerType order 2021-02-16 19:09:24 -08:00
Lukas Joswiak f9c8674bc9 Use auto 2021-02-16 16:48:51 -08:00
Lukas Joswiak 30d2d9ff82 Remove async tracer 2021-02-16 16:36:33 -08:00
Steve Atherton 76a4293d1d Merge branch 'release-6.3'
# Conflicts:
#	fdbserver/fdbserver.actor.cpp
2021-02-15 02:13:06 -08:00
Steve Atherton f4c9b88908 Merge branch 'release-6.2' into release-6.3
# Conflicts:
#	cmake/CompileBoost.cmake
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Error.cpp
2021-02-15 02:05:03 -08:00
Andrew Noyes dd0e692f2a Log std::exception::what when creating an unknown_error
unknown_error usually gets created when an exception of some type other
than `Error` is thrown. If the current exception is a std::exception
(and it usually is), we can log more information about it - which is
(incredibly!) helpful for figuring out what the heck happened.
2021-02-13 17:42:11 +00:00
Andrew Noyes 4e184fe236 Fix memory errors 2021-02-11 02:58:21 +00:00
Andrew Noyes 53739c9da5
Merge pull request #4241 from sfc-gh-clin/add-c-function-snapshot
Add c function for snapshot
2021-02-09 15:49:20 -08:00
Russell Sears a56184c0b7 log TLSVerifyFailure as a warning (severity 20) since it generally indicates a miconfiguration or important network problem 2021-02-08 22:01:48 +00:00
A.J. Beamon 601a548cfe Revert "Move new config to config.h.cmake"
This reverts commit 2951ebb4e7.
2021-02-08 14:01:02 -08:00
A.J. Beamon 2951ebb4e7 Move new config to config.h.cmake 2021-02-08 13:34:17 -08:00
A.J. Beamon aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
2021-02-08 12:58:34 -08:00
A.J. Beamon 449e4b98b4 Attempt to fix a build warning that's now blocking the compile due to werror. 2021-02-08 12:06:51 -08:00
A.J. Beamon 67e783acf8 Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
# Conflicts:
#	cmake/CompileBoost.cmake
#	cmake/FDBComponents.cmake
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/storageserver.actor.cpp
#	flow/Knobs.h
#	flow/network.h
2021-02-08 09:20:28 -08:00
Chaoguang Lin a85458dd1e Merge branch 'master' of github.com:apple/foundationdb into add-c-function-snapshot 2021-02-05 14:14:52 -08:00
Chaoguang Lin 97925de9d4 Update fdb_database_create_snapshot to take a UID as input 2021-02-05 14:14:38 -08:00
A.J. Beamon 5e80e16802
Revert "CDs reject requests from external cluster or requests need to be forwarded based on an old forward request." 2021-02-05 13:05:33 -08:00
Evan Tschannen 8091d8179d
Merge pull request #4252 from dongxinEric/feature/4251/cds-reject-requests-from-other-cluster
CDs reject requests from external cluster or requests need to be forwarded based on an old forward request.
2021-02-04 17:33:07 -08:00
Andrew Noyes db82da5bc2 Only link to jemalloc in fdbserver
Including jemalloc in libfdb_c seems to be problematic, since it's
apparently not recommended to dlopen jemalloc:
https://github.com/jemalloc/jemalloc/issues/937#issuecomment-311242450
2021-02-05 01:30:31 +00:00
Xin Dong c017b51d66 Fix typo 2021-02-04 13:35:40 -08:00
Andrew Noyes e4a55908ff
Merge pull request #4222 from sfc-gh-mpilman/features/jemalloc
Replace standard malloc with jemalloc
2021-02-04 12:47:29 -08:00
Markus Pilman 7b489da13b introduce USE_JEMALLOC and some refactoring 2021-02-04 11:53:28 -07:00
Meng Xu 1db4b73215
Merge pull request #4243 from sfc-gh-etschannen/feature-storage-io-timeout
throw an io_timeout if it takes more than 2 minutes to commit on the storage server
2021-02-03 18:10:16 -08:00
Meng Xu 675e567305
Merge pull request #4218 from sfc-gh-etschannen/feature-low-priority-reads-6.2
Reduce read priority when durability lag is high
2021-02-03 18:09:39 -08:00
Evan Tschannen 068647ba1b updated naming and comments 2021-02-03 14:24:39 -08:00
Evan Tschannen c4b622e7b8 added a comment and updated a variable name 2021-02-03 13:58:04 -08:00
Evan Tschannen a0a71ff7f9 simplified the algorithm based on testing 2021-02-02 17:53:07 -08:00
Steve Atherton 6f0e2dcdd0
Merge pull request #4268 from sfc-gh-clin/add-knob-for-IO-size
Add option to define I/O size
2021-02-02 16:22:24 -08:00
Chaoguang Lin 593165679a Remove unnecessary header 2021-02-02 12:42:46 -08:00
Chaoguang Lin 17671604c0 Add option to define I/O size 2021-02-02 12:39:05 -08:00
Jon Fu 090eee33e7 rearrange include ordering 2021-02-02 14:58:51 -05:00
Jon Fu d10b452758 include <errno.h> instead of <sys/errno.h> 2021-02-02 14:55:02 -05:00
Jon Fu 761d9991a3 change to SevWarn instead of SevError 2021-02-02 14:47:19 -05:00
Jon Fu 5a28d783b0 add readable description of error to trace event 2021-02-02 14:34:56 -05:00
Jon Fu 5fa7c73810 add tracing to fileopen failure before throwing error for more info 2021-02-02 14:30:12 -05:00
Markus Pilman 534fa45291 Merge branch 'master' of github.com:apple/foundationdb into features/coroutine2 2021-02-01 16:31:16 -07:00
Markus Pilman c25a3f11c3
Merge pull request #4259 from sfc-gh-satherton/ratelimit-6.2
Backport AFCache write rate control to 6.2
2021-01-31 20:55:16 -07:00
sfc-gh-tclinkenbeard 04b06c0c3c Merge remote-tracking branch 'origin/master' into capture-forked-process-output 2021-01-31 16:14:38 -08:00
Markus Pilman e4b953ab38
Merge pull request #4229 from sfc-gh-satherton/afcache-write-limit
AsyncFileCached RateControl support and bug fixes
2021-01-31 15:13:03 -07:00
Steve Atherton 97781bde76 Removed stray knob added unintentionally in a merge conflict. 2021-01-30 18:35:16 -08:00
Steve Atherton b0fd6bfbcf More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-30 18:18:16 -08:00
Steve Atherton 52c2695f59 Bug fixes involving use of AFCached rate control in simulation. 2021-01-30 18:13:44 -08:00
Steve Atherton e56fe02a25 Merge branch 'add-throttling-on-AsyncfileCached' of github.com:sfc-gh-clin/foundationdb into afcache-write-limit 2021-01-30 18:11:54 -08:00
Steve Atherton 14af5857ed Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why. 2021-01-30 18:06:05 -08:00
Steve Atherton e72d486215 Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments. 2021-01-30 18:06:04 -08:00
Christophe Chevalier ff0667b28d winbuild: define BOOST_USE_WINDOWS_H, WIN32_LEAN_AND_MEAN and NOMINMAX to fix various macro expansions issues
- NOMINMAX to fix confusion between std::max(..) and max(..) macro redefinition
- BOOST_USE_WINDOWS_H to prevent boost from redefining win32 APIs
- WIN32_LEAN_AND_MEAN to fix include ordering issues with winsock.h
2021-01-30 15:15:02 +01:00
Xin Dong 9f837a3b84 Coordinators will not serve following requests after this change:
- A request that needs to be forwarded after the forward was set for a long(configured via Knobs) time.
- A request that has a different connection string than the CD's local connection string.

Simulation has been made to work with this change.
2021-01-29 13:24:45 -08:00
Evan Tschannen 4f3da7c93b also kill tlogs if a commit takes too long 2021-01-29 12:18:06 -08:00
sfc-gh-tclinkenbeard acac02587d Trace output from forked processes 2021-01-29 01:31:26 -08:00
Markus Pilman 301e25334d fixed asan build 2021-01-28 09:01:40 -07:00
Steve Atherton 7ce0cca687 Merge branch 'release-6.3' of https://github.com/apple/foundationdb into afcache-write-limit 2021-01-28 07:27:32 -08:00
Markus Pilman 7c64e3943b compile boost for sanitizers 2021-01-27 19:13:02 -07:00
Evan Tschannen 75b24da109 fixed cast 2021-01-27 14:12:18 -08:00
Evan Tschannen 3e2a0a5264 throw an io_timeout if it takes more than 2 minutes to commit on the storage server 2021-01-27 11:50:42 -08:00
Steve Atherton 989fee908a More bug fixes in SQLite file handle lifetimes after storage server restart (including simulated process reboot) when a write rate limit is used. Added several debug trace events, renamed some VFSAsync* events to be more consistent. 2021-01-27 00:01:33 -08:00
Markus Pilman 06a52449d1
typo 2021-01-26 17:38:03 -07:00
Markus Pilman 8dcded849e
Update flow/CMakeLists.txt 2021-01-26 17:37:17 -07:00
Andrew Noyes 341069fcce WIP asan on boost coroutine2 2021-01-26 17:10:13 -07:00
Steve Atherton d718e7baf3
Merge pull request #4179 from sfc-gh-tclinkenbeard/run-minio-joshua
Support Testing S3BlobStore backup in simulation
2021-01-26 14:39:57 -08:00
Markus Pilman 0207f89904 don't use jemalloc with valgrind, asan, etc 2021-01-26 11:50:40 -07:00
sfc-gh-tclinkenbeard 7558cd8e06 Merge remote-tracking branch 'origin/master' into simplify-global-knobs 2021-01-26 10:00:25 -08:00
Andrew Noyes 564c70acb4
Merge branch 'master' into anoyes/modernize-use-override 2021-01-25 18:26:18 -08:00
Andrew Noyes 4ee97c0784 Use clang-tidy to automatically fix missing overrides
Use `clang-tidy -p . $file -checks='-*,modernize-use-override' -header-filter='.*' -fix`
to fix missing overrides, and then use git clang-format to reformat just
those changes. This went pretty well for most files.

Formatting the following files went off the rails, so I'm going to
follow up with a commit that's just clang-tidy and no clang-format.

- fdbclient/DatabaseBackupAgent.actor.cpp
- fdbclient/FileBackupAgent.actor.cpp
- fdbserver/OldTLogServer_4_6.actor.cpp
- fdbmonitor/SimpleIni.h
- fdbserver/workloads/ClientTransactionProfileCorrectness.actor.cpp
2021-01-26 02:04:12 +00:00
Andrew Noyes 0ef44739ea Fix OPEN_FOR_IDE build in preparation for using clang-tidy 2021-01-26 02:04:11 +00:00
Andrew Noyes 17962712fb
Merge pull request #4227 from sfc-gh-tclinkenbeard/encapsulate-packetwriter
Encapsulate PacketWriter methods
2021-01-25 15:08:46 -08:00
Markus Pilman 5bf5545b87
Merge pull request #4236 from sfc-gh-anoyes/anoyes/merge-6.3-to-master
Merge 6.3 to master
2021-01-25 15:53:43 -07:00
Markus Pilman d346c4955d
Merge pull request #4235 from sfc-gh-kmakino/kaomakino/aarch64_update
Fix minor issues for aarch64
2021-01-25 15:20:08 -07:00
Kao Makino 29976f0b41 Fix minor issues for aarch64 2021-01-25 12:10:57 -08:00
Andrew Noyes adde95bea5 Merge branch 'release-6.3' into anoyes/merge-6.3-to-master
Commit merge conflicts for now
2021-01-25 18:55:24 +00:00
Jon Fu c3775ede37 add override keyword properly to squash compiler warnings 2021-01-25 13:24:15 -05:00
Steve Atherton 0d32bd0c44 Bug fixes involving use of AFCached rate control in simulation. 2021-01-24 01:50:09 -08:00
sfc-gh-tclinkenbeard 51cbc494f1 Merge remote-tracking branch 'origin/master' into run-minio-joshua 2021-01-23 11:31:19 -08:00
sfc-gh-tclinkenbeard b88a47f7db Encapsulation PacketWriter methods 2021-01-22 18:48:23 -08:00
sfc-gh-tclinkenbeard 5f96c4b721 Add override keyword to DummyThreadPool methods 2021-01-22 13:19:29 -08:00
Russell Sears 589eaf54d7 Switch to BoringSSL by default 2021-01-22 20:39:12 +00:00
Chaoguang Lin a09acd321e update knobs default values for testing 2021-01-22 11:45:37 -08:00
Chaoguang Lin 18f7ee22c2 Merge branch 'release-6.3' of github.com:apple/foundationdb into add-throttling-on-AsyncfileCached 2021-01-22 11:44:29 -08:00
sfc-gh-tclinkenbeard 4dd9c4889e Revert "Make Profiler::active_profiler a unique pointer" and add comment 2021-01-21 14:32:46 -08:00
sfc-gh-tclinkenbeard 065c7e570c Revert "Make g_net2 a unique_ptr" and add comment 2021-01-21 14:31:03 -08:00
Markus Pilman b149069b58 link statically against jemalloc 2021-01-21 13:35:27 -07:00
sfc-gh-tclinkenbeard fb9b4b7626 Merge remote-tracking branch 'origin/master' into misc-changes 2021-01-21 10:45:50 -08:00
Andrew Noyes 281151ff20 Merge remote-tracking branch 'upstream/release-6.2' into anoyes/merge-6.2-to-6.3
Commit conflict markers for now
2021-01-21 17:54:26 +00:00
Lukas Joswiak 4d6fbf099c Use StringRef for span tags 2021-01-20 13:57:36 -08:00
Lukas Joswiak 34ad4b617b Add support for longer strings 2021-01-20 13:57:36 -08:00
Lukas Joswiak 5f5f487967 Allow zero length strings 2021-01-20 13:57:36 -08:00
Lukas Joswiak e27f2ca3fb Remove some tracers from Windows 2021-01-20 13:57:36 -08:00
Lukas Joswiak b9c6ca2481 Fix Windows compilation 2021-01-20 13:57:36 -08:00
Lukas Joswiak 3f7e19a182 Add tags to spans to provide additional metadata 2021-01-20 13:57:36 -08:00
Lukas Joswiak f6f53c7a73 Style updates 2021-01-20 13:57:36 -08:00
Lukas Joswiak e8397b3812 Move UDP listen port to knob 2021-01-20 13:57:36 -08:00
Lukas Joswiak 8cc55a3ed5 Fix merge issues 2021-01-20 13:57:36 -08:00
Lukas Joswiak e8038cd2c1 Remove debug code 2021-01-20 13:57:36 -08:00
Lukas Joswiak de8bf9f6cf Write span context as separate 64-bit ints 2021-01-20 13:57:36 -08:00
Lukas Joswiak ecdb96a7b9 Add IP address and port to serialized spans 2021-01-20 13:57:36 -08:00
Lukas Joswiak 98e6ba6962 Add POSIX send functionality 2021-01-20 13:57:36 -08:00
Lukas Joswiak 13858a853c Backoff future send attempts if a send failure occurs 2021-01-20 13:57:36 -08:00
Lukas Joswiak d2143f184a Add fast UDP tracer 2021-01-20 13:57:36 -08:00
Lukas Joswiak 4ea5267bfd Add TEST macro 2021-01-20 13:57:36 -08:00
Lukas Joswiak 6800a4dcb3 Fix incorrect simulation check for first byte
The array representing fields of the Span can be size 4 or 5, depending
on whether the Span has any parents or not.
2021-01-20 13:57:36 -08:00
Lukas Joswiak 3c11b4b1bd Cleanup 2021-01-20 13:57:36 -08:00
Lukas Joswiak 3132c38bff Rename fluentd -> UDP 2021-01-20 13:57:36 -08:00
Lukas Joswiak 069d75b6f2 Update format of serialized spans
Write span context as a string instead of as two 64-bit integers. Also
removes some debug code.
2021-01-20 13:57:36 -08:00
Lukas Joswiak abedd7a147 Fix msgpack array size format
Was being written in little-endian, should be written in big-endian.
2021-01-20 13:57:36 -08:00
Lukas Joswiak e2f57b0845 Modify trace calls 2021-01-20 13:57:36 -08:00
Lukas Joswiak 45699f82be Abandon sending traces after first failed attempt 2021-01-20 13:57:36 -08:00
Lukas Joswiak cfba197302 Catch errors around async_send 2021-01-20 13:57:36 -08:00
Lukas Joswiak 3b8a84a153 Add per-test buffers created trace 2021-01-20 13:57:36 -08:00
Lukas Joswiak aa16ea6024 Add additional trace calls 2021-01-20 13:57:36 -08:00
Lukas Joswiak b3879d7919 Remove actor ready assertions 2021-01-20 13:57:35 -08:00
Lukas Joswiak 753bdd235e Add UDP server to check received packets in simulation 2021-01-20 13:57:35 -08:00
Lukas Joswiak 96029ca08f Serialize span as array instead of as map 2021-01-20 13:57:35 -08:00
Lukas Joswiak add80be134 Add support for arrays with up to 65535 elements 2021-01-20 13:57:35 -08:00
Lukas Joswiak 55746506c2 Log queue size every couple seconds 2021-01-20 13:57:35 -08:00
Lukas Joswiak d473735f06 Fix gcc compilation issue 2021-01-20 13:57:35 -08:00
Lukas Joswiak 1313de15b8 Fix bug with incorrect parent container size 2021-01-20 13:57:35 -08:00
Lukas Joswiak 1961c3fc80 Serialize spans to msgpack and send via UDP 2021-01-20 13:57:35 -08:00
Kao Makino 4ddf3385a8 Reserve the priority queue container 2021-01-20 10:34:48 -08:00
Markus Pilman c3efbe3040 fixed minor bug 2021-01-19 16:52:30 -07:00
Markus Pilman f40d8c2f49 make profiler signal handler reentrant safe 2021-01-19 16:04:21 -07:00
Markus Pilman fb64902d5c Assign roles 2021-01-19 16:04:09 -07:00
Markus Pilman aea57d282f Merge branch 'master' of github.com:apple/foundationdb into features/actor-lineage 2021-01-19 11:04:25 -07:00
Steve Atherton 11927f6fff Another refactor of SQLite injected error handling to address failures of the previous attempt. Some code cleanup. Added documentation of injected error handling works and why. 2021-01-17 04:17:13 -08:00
Steve Atherton f1adafaf8c Refactored how injected fault state is managed to make debugging injected fault handling easier. Moved injected error tracking for open() into an INetwork global since it should be unique per simulated process. Fixed bug where injected error in WAL file was not recognized as injected because it occurred before SQLiteDB's vfsWAL pointer was initialized. Simplified logic in SQLiteDB. Added comments. 2021-01-16 05:04:30 -08:00
sfc-gh-tclinkenbeard 191f215b9d Remove chrono dependency 2021-01-13 21:54:22 -08:00
sfc-gh-tclinkenbeard 95eaa5e866 Merge remote-tracking branch 'origin/master' into misc-changes 2021-01-13 21:14:36 -08:00
sfc-gh-tclinkenbeard 477bd8bc8e Make Knobs::Knobs() protected 2021-01-13 17:43:33 -08:00
sfc-gh-tclinkenbeard 8ff14878fe Merge remote-tracking branch 'origin/master' into simplify-global-knobs 2021-01-13 14:39:35 -08:00
sfc-gh-tclinkenbeard e29ed3bf99 Remove createGlobal*Knobs functions 2021-01-13 12:14:04 -08:00
Markus Pilman 2a24857cfa
Merge pull request #4182 from skunkwerks/fix/avoid-preprocesor-clash
build: don't clash with FreeBSD native defines
2021-01-12 10:33:21 -07:00
Markus Pilman 2609c3d619
Merge pull request #4072 from sfc-gh-tclinkenbeard/improve-type-safety
Make enums automatically binary serializable
2021-01-12 10:31:34 -07:00
Markus Pilman 8e93573aad Merge branch 'master' of github.com:apple/foundationdb into features/actor-lineage 2021-01-05 10:24:45 -07:00
Dave Cottlehuber 2daa4d33af flow: fix compilation error on FreeBSD 2021-01-04 13:41:53 +00:00
Dave Cottlehuber 96a37e4d46 build: don't clash with FreeBSD native defines
FreeBSD has an existing CACHE_LINE_SIZE parameter as a platform-specific define.

```
# /usr/include/machine/param.h
/*
 * CACHE_LINE_SIZE is the compile-time maximum cache line size for an
 * architecture.  It should be used with appropriate caution.
 */
#define CACHE_LINE_SHIFT        6
#define CACHE_LINE_SIZE         (1 << CACHE_LINE_SHIFT)
```
2021-01-04 13:13:06 +00:00
sfc-gh-tclinkenbeard 61a29ecfc8 Merge remote-tracking branch 'origin/master' into run-minio-joshua 2021-01-01 09:38:36 -04:00
sfc-gh-tclinkenbeard 21d7620922 TDMetric optimizations 2020-12-26 22:50:34 -04:00
sfc-gh-tclinkenbeard 5bfa6cea98 Merge remote-tracking branch 'origin/master' into misc-changes 2020-12-26 20:47:00 -04:00
sfc-gh-tclinkenbeard 913401aaee Move SendBufferIterator out of network.h
This class is only needed for implementation in Net2.actor.cpp and
SimExternalConnection.actor.cpp, so this class should not be included
everywhere network.h is included
2020-12-26 16:30:00 -04:00
sfc-gh-tclinkenbeard a7202a5b42 Use connectExernal in INetworkConnections::connect 2020-12-24 21:20:04 -04:00
sfc-gh-tclinkenbeard 201857541d Remove unnecessary copies from INetworkConnections interface 2020-12-24 21:20:03 -04:00
sfc-gh-tclinkenbeard 1e7ea8ece1 Implemented SimExternalConnection::write 2020-12-24 21:20:02 -04:00
sfc-gh-tclinkenbeard fed1a03950 Added SimExternalConnection (most methods not implemented yet) 2020-12-24 21:20:02 -04:00
Andrew Noyes 9601769b01
Merge pull request #3858 from sfc-gh-rchen/stable_interfaces
Stable interfaces
2020-12-11 09:34:27 -08:00
Andrew Noyes 128c758edc Set special sanitizer annotations in stacktrace.amalgamation.cpp
That's the only place they're needed. This fixes at least the USE_TSAN
build and possibly other sanitizer builds
2020-12-11 02:08:02 +00:00
Markus Pilman f8e1df6c4f Support for actor stack traces 2020-12-10 10:42:04 -07:00
Chaoguang Lin ced946c918 Move flow knobs under AsyncFileCached part 2020-12-09 23:02:13 -08:00
Markus Pilman 945d0246cd add actor stacktrace feature 2020-12-09 13:28:15 -07:00
Chaoguang Lin 13819171fa Update knob names to FLOW_CACHEDFILE_WRITE_WINDOW_LIMIT and FLOW_CACHEDFILE_WRITE_WINDOW_SECONDS 2020-12-09 11:45:09 -08:00
Markus Pilman 9344a5fd0a Fix Coro ucontext bug introduced in #4109 2020-12-09 12:25:31 -07:00
Markus Pilman 0d324cee80 Annotation framework and role lineage 2020-12-09 10:19:59 -07:00
Chaoguang Lin 90f56d0e84 Add STORAGE_WRITE_WINDOW_SECONDS and STORAGE_WRITE_WINDOW_SECONDS for rate control 2020-12-08 16:35:08 -08:00
Andrew Noyes 922057bb8a
Merge pull request #4109 from sfc-gh-mpilman/features/flow-config
Move compiler definitions into config file
2020-12-08 13:19:25 -08:00
sfc-gh-tclinkenbeard afa8e78911 Fix Debouncer move constructor and move assignment operator
Moving the Debouncer did not previously move the worker future
2020-12-08 09:09:33 -08:00
sfc-gh-tclinkenbeard c8786a65c3 Remove unnecessary uses of virtual 2020-12-08 09:09:33 -08:00
sfc-gh-tclinkenbeard 5059228e9e Use unique_ptr for TraceEvent::tmpEventMetric 2020-12-08 09:09:32 -08:00
sfc-gh-tclinkenbeard b8a55dd097 Use unique_ptr for PImpl 2020-12-08 09:09:32 -08:00
sfc-gh-tclinkenbeard 882ea49b2e Make Profiler::active_profiler a unique pointer 2020-12-08 09:09:32 -08:00
sfc-gh-tclinkenbeard fad43687d1 Make g_net2 a unique_ptr 2020-12-08 09:09:31 -08:00
sfc-gh-tclinkenbeard c324e80663 Use unique_ptr for g_tracer 2020-12-08 09:09:31 -08:00
sfc-gh-tclinkenbeard baf751e706 Use unique_ptr for DynamicEventMetric fields 2020-12-08 09:09:30 -08:00
sfc-gh-tclinkenbeard 29e10d1418 Modernize TDMetric signatures 2020-12-08 09:09:30 -08:00
sfc-gh-tclinkenbeard d15441e85c Replace non-standard sealed with final 2020-12-08 09:09:30 -08:00
sfc-gh-tclinkenbeard 2b92cfbde8 Fix Net2 method signatures 2020-12-08 09:09:30 -08:00
Trevor Clinkenbeard 9581bc0279
Merge pull request #4129 from sfc-gh-anoyes/anoyes/merge-release-6.3-master
Merge release 6.3 master
2020-12-08 08:57:54 -08:00
Markus Pilman a1f7c74ac4
Merge pull request #4125 from sfc-gh-ljoswiak/features/tracing-options
Let client disable tracing
2020-12-07 17:40:09 -07:00
Markus Pilman d837e923ad minor bugfix 2020-12-07 15:23:18 -07:00
Markus Pilman 05f77f905f Added actor lineage 2020-12-07 15:15:25 -07:00
Markus Pilman f97cb9b41a remove USE_UCONTEXT (as this didn't exist before) 2020-12-07 11:26:08 -07:00
Andrew Noyes 7fbc4d7391 Resolve conflicts 2020-12-04 23:58:42 +00:00
Andrew Noyes 877997632d Merge branch 'release-6.3' into anoyes/merge-release-6.3-master
Include conflict markers for review purposes
2020-12-04 01:38:07 +00:00
Lukas Joswiak 3669615e4b Refactor and clean up 2020-12-03 14:06:11 -08:00
Richard Chen c77d9e4abe merge conflicts 2020-12-02 21:53:19 +00:00
Markus Pilman b04f09b29b address review comments 2020-11-25 16:23:36 -07:00
Markus Pilman dae8ea24ad Move compiler definitions into config file 2020-11-25 15:06:59 -07:00
Markus Pilman 82f7f541c3 started lineage implementation 2020-11-25 11:38:08 -07:00
Markus Pilman 18ba83fc3d
Merge pull request #4105 from sfc-gh-anoyes/anoyes/release-6.3-merge
Merge release-6.2 into release-6.3 and fix conflicts
2020-11-25 11:16:05 -07:00
Andrew Noyes 5a717b3ba9 Change dd_tracker_cancelled to 1215 to avoid conflict
This is safe because it's not been in a release, and it doesn't get
serialized.
2020-11-24 16:27:21 -08:00
Andrew Noyes dc2bac5670 Resolve conflicts 2020-11-24 19:09:42 +00:00
Andrew Noyes 1f541f02be Merge branch 'anoyes/merge-6.2-to-6.3' into anoyes/release-6.3-merge
Merge, leaving conflict markers for now
2020-11-24 16:55:34 +00:00
sfc-gh-tclinkenbeard aa07df6a91 Backport to 6.3 the ability to read xxhash3 checksum for sqlite pages 2020-11-24 00:28:49 -08:00
Andrew Noyes 823337a2e1 Reserve error code 1215 for dd_tracker_cancelled 2020-11-23 14:51:40 -08:00
Andrew Noyes 231a7a10d0
Merge pull request #4075 from sfc-gh-tclinkenbeard/sqlite-xxhash3-checksum
Sqlite xxhash3 checksum
2020-11-23 10:43:35 -08:00
sfc-gh-tclinkenbeard b46384e1b8 Initialize client and flow knobs 2020-11-22 09:56:18 -08:00
sfc-gh-tclinkenbeard 156a617ed8 Replace xxhash64 with xxhash3
The goal here is to improve performance
2020-11-21 18:25:35 -08:00
sfc-gh-tclinkenbeard b1e3478267 Simplified global knobs 2020-11-21 13:27:48 -08:00
Markus Pilman 615029a393
Merge pull request #4082 from sfc-gh-dyoungworth/dyoungworth/merge_6_3_master
merge 6 3 master
2020-11-19 20:53:51 -07:00
Markus Pilman 67944f6899
Merge pull request #4087 from sfc-gh-dyoungworth/dyoungworth/addDateTimeToTrace2
add date time to trace (release 6.2)
2020-11-19 17:44:44 -07:00
David Youngworth b1256b5dcd Address review comments, simplify DateTime 2020-11-18 16:55:47 -08:00
David Youngworth 5ade54f767 Fix windows build, DateTime to use UTC 2020-11-18 16:55:12 -08:00
David Youngworth 490fe61032 Fix bug in rolled Trace code 2020-11-18 16:55:06 -08:00
David Youngworth 50e515c29a Add DateTime to trace, initial commit 2020-11-18 16:54:49 -08:00
sfc-gh-tclinkenbeard bcda617b80 Benchmark hashing algorithms in flowbench 2020-11-17 23:14:55 -08:00
David Youngworth fc9b78737f Fix some merge bugs 2020-11-17 14:53:02 -08:00
Markus Pilman 659f58d28d
Merge pull request #4001 from sfc-gh-ljoswiak/features/simulation-tracing
Open random tracer in simulation
2020-11-17 14:27:18 -07:00
Lukas Joswiak 1506ed5a46 Don't write traces to log file in simulation 2020-11-17 11:26:10 -08:00
David Youngworth d64cf8b9e3 Merge branch 6.3 into master 2020-11-17 11:22:45 -08:00
Lukas Joswiak 71d0b1da85 Open random tracer in simulation 2020-11-17 09:51:19 -08:00
David Youngworth 489ba20641 Fix several merge issues 2020-11-16 14:46:36 -08:00
David Youngworth d0391db862 Merge branch 'release-6.2' into release-6.3 2020-11-16 10:15:23 -08:00
sfc-gh-tclinkenbeard ca8ea3b6ff Fix memory issues caused by cancelling data distribution tracker 2020-11-15 23:52:36 -08:00
sfc-gh-tclinkenbeard fff8e34b4d Move IKeyValueContainer from flow to fdbserver 2020-11-15 12:23:08 -08:00
sfc-gh-tclinkenbeard 575b36bf53 Move RadixTree from flow to fdbserver 2020-11-15 11:57:54 -08:00
sfc-gh-tclinkenbeard eab75d4ee1 Make enums automatically binary serializable 2020-11-14 19:22:04 -08:00
Markus Pilman d3598bfaa4 address review comments 2020-11-13 15:37:03 -07:00
Jingyu Zhou 9f2399f951
Merge pull request #4061 from xis19/reportHistogramPeriodically
Report histogram periodically
2020-11-13 14:14:25 -08:00
Xiaoge Su 3a6948c199 Report histogram periodically 2020-11-12 17:04:33 -08:00
Xiaoge Su 4a0fa57989 Add bytes_per_second unit in histograms 2020-11-12 15:38:51 -08:00
Markus Pilman 54df295186 remove dead code 2020-11-12 14:00:34 -07:00
Markus Pilman 0f205620bf
Merge branch 'master' into features/udp 2020-11-12 13:51:17 -07:00
Markus Pilman 1343f40117 don't allow empty coments 2020-11-11 14:07:54 -07:00
Jingyu Zhou 32a089dde1
Merge pull request #4030 from vishesh/task/issue-connection-logging-count
FlowTransport: Sample connect latencies
2020-11-10 20:00:58 -08:00
Vishesh Yadav 1976f2c371 FlowTransport: Sample connect latencies 2020-11-10 12:12:01 -08:00
Markus Pilman 16affd3575 Make test macros enforce unique comments 2020-11-10 11:13:34 -07:00
Xiaoge Su ebca63e3c7
Merge pull request #4026 from sears/histogram_fixes
Histogram fixes
2020-11-09 13:12:38 -08:00
Russell Sears 08e8a06933 PR comments; add missing addRef when re-using Histogram in HistogramRegistry; Instrument UnsentPacketQueue in FlowTransport 2020-11-08 08:37:16 -08:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
sfc-gh-tclinkenbeard a42a431b57 Add makeReference (similar to std::make_shared) 2020-11-07 22:09:53 -08:00