Commit Graph

236 Commits

Author SHA1 Message Date
Meng Xu 862336de8f Merge branch 'master' into mengxu/merge-to-master-PR 2020-09-24 17:06:00 -07:00
Steve Atherton 58e043c7a5 Enable run loop profiler for test and multitest roles. 2020-09-24 14:14:55 -07:00
A.J. Beamon 5f7e784742
Merge pull request #3796 from xis19/master
Refactor fdbserver.actor.cpp for better readability
2020-09-24 09:12:47 -07:00
Xiaoge Su 581427c880 fixup! Remove "\n" and extra spaces in printUsage 2020-09-23 12:03:09 -07:00
Xiaoge Su e2ea6bbdd0 fixup! Use currLine.size() for the first currLength in printOptionUsage 2020-09-23 11:19:02 -07:00
Xiaoge Su 2bf4ebbd51 fixup! Add missing comma 2020-09-21 20:43:07 -07:00
Xiaoge Su 3c505786ca Refactor fdbserver.actor.cpp 2020-09-21 20:19:47 -07:00
sfc-gh-tclinkenbeard 0814841827 Replace NULL with nullptr in fdbserver 2020-09-20 11:31:49 -07:00
Young Liu 35bef73a1c Rename proxy to commit proxy 2020-09-10 17:44:15 -07:00
Evan Tschannen a49cb41de7 Merge branch 'release-6.3'
# Conflicts:
#	CMakeLists.txt
#	cmake/ConfigureCompiler.cmake
#	fdbserver/Knobs.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	flow/ThreadHelper.actor.h
#	flow/serialize.h
#	tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
Steve Atherton 38cade1146 Merged tag release-6.3.3 into tls-background-handshake. 2020-07-17 23:39:57 -07:00
Alex Miller 0131f0dc48 Make simulation able to load and run .toml files 2020-07-12 14:47:39 -07:00
Markus Pilman cc60597739
Merge branch 'master' into bugfixes/visibility-perf-regression 2020-07-09 17:04:04 -06:00
Meng Xu 1b68665228 Merge branch 'master' into release-6.3 2020-07-08 18:52:05 -07:00
Markus Pilman 0fbe7101c3 Revert "Revert "Request tracing""
This reverts commit 327cc31e35.
2020-07-07 10:06:13 -06:00
sfc-gh-tclinkenbeard dd5cd762a8 Removed dead code 2020-07-04 12:50:40 -07:00
Steve Atherton 0da4c91ad5 Performance improvements in socket use and buffering which results in TLS network connections using around 3.5x less CPU.
PacketBuffers are 16k minimum sizes, using FastAlloc.  All calling paths to PacketBuffer creation now pass a size hint if known.  P2PNetworkTest improved in many ways, errors go to TraceEvents instead of stdout, error counts are reported in stats, range min/max parameters are combined into "min:max" strings, new options to "stutter" connections by delaying randomly before socket read or write, message size to send is now configured separately for incoming vs outgoing sessions.
2020-06-25 20:44:43 -07:00
Jingyu Zhou 327cc31e35
Revert "Request tracing" 2020-06-16 12:32:42 -07:00
Meng Xu 8595813b7d
Merge pull request #3355 from apple/release-6.3
Merge Release 6.3 into master branch
2020-06-12 20:08:47 -07:00
Jingyu Zhou 444a5a0ec8 Set up blob credentials for backup workers
This is needed so that backup workers (fdbserver) can upload files to blobs.
2020-06-11 17:26:21 -07:00
Markus Pilman 09c136e434 Framework for transaction tracing 2020-06-08 16:09:37 -07:00
A.J. Beamon d128252e90 Merge release-6.3 into master 2020-05-22 09:25:32 -07:00
A.J. Beamon b589c3c940 Remove IncludeVersions.h, which is no longer needed for cmake-only builds 2020-05-20 13:23:02 -07:00
tclinken 3ff2fa9c2a Removed uses of outdated cpuTicks and errorCounts 2020-05-13 23:27:44 -07:00
Alex Miller 27da91ab9e Merge remote-tracking branch 'upstream/master' into mutation-debugging 2020-05-13 12:51:44 -07:00
Jingyu Zhou a833724322
Merge pull request #3078 from xumengpanda/mengxu/fr-circus-stall-PR
Performant restore: Various improvements based on circus test
2020-05-07 20:07:23 -07:00
Meng Xu a93c23d239 Resovle review comments 2020-05-07 15:06:59 -07:00
Alex Miller 78d644f673
Merge branch 'master' into bugfixes/cmake-includes 2020-05-04 02:11:46 -07:00
Dave Cottlehuber 98639645b1 fdbserver: update headers 2020-04-30 18:11:23 +00:00
Evan Tschannen b7f5f3be48 merge in master 2020-04-28 13:11:47 -07:00
Evan Tschannen c87aa33941 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	bindings/go/src/fdb/generated.go
#	documentation/sphinx/source/api-common.rst.inc
#	documentation/sphinx/source/api-ruby.rst
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/FailureMonitorClient.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/vexillographer/fdb.options
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/OldTLogServer_6_0.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	versions.target
2020-04-23 13:47:53 -07:00
Alex Miller 2ce539ef6d Respect flow<->fdbrpc module boundaries.
Which fixes a compilation error due to a circular dependency between
flow.a and fdbrpc.a.  However, this is now done at the cost of newNet2
users have to remember to add Net2FileSystem::stop() as a callback.
2020-04-20 02:53:07 -07:00
Evan Tschannen ba3e2af473 Merge commit '5288033bcfe40c3ade97c8bf2d04cf31b3f16cb1' into feature-tree-broadcast 2020-04-17 15:17:37 -07:00
A.J. Beamon e104a2e3a6 Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into run-loop-busy-profiler
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/NativeAPI.actor.h
#	fdbserver/fdbserver.actor.cpp
#	flow/Net2.actor.cpp
2020-04-14 14:02:24 -07:00
Evan Tschannen 07cc0a8d74 code cleanup 2020-04-10 17:02:11 -07:00
tclinken 52860043c9 Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs 2020-04-08 12:26:21 -07:00
Meng Xu e5b2cd81d5 FastRestore:Cleanup debug code 2020-04-07 15:56:44 -07:00
Markus Pilman e4611e8ae4 fix versions.h stupidity 2020-04-06 10:28:55 -07:00
Markus Pilman 8b5780c36c don't include source and binary dir
This forces users to use include paths from the sources root.

So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
Meng Xu 536e65cd76 FastRestore:Introduce debugFRMutation for debug keys 2020-04-05 15:00:36 -07:00
Meng Xu a81ec332a9 FastRestore:Fix:Master cannot throttle on in progress version batches when it release batches out of order in simulation 2020-04-04 17:34:26 -07:00
tclinken 884e92bb49 Atomically update dependent knobs 2020-04-01 15:18:49 -07:00
Alex Miller 72e5891058 Clean up and rework the debugMutation API.
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process.  This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.

As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match.  A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.

This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 03:30:28 -07:00
Evan Tschannen 303df197cf Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	bindings/c/test/mako/mako.c
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogRouter.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/flow.vcxproj
#	flow/flow.vcxproj.filters
#	versions.target
2020-03-06 18:22:46 -08:00
Evan Tschannen dbfc0cbcc0
Merge pull request #2781 from alexmiller-apple/certificate-refresh
Refresh certificates used for handshaking when they change on disk
2020-03-06 11:12:04 -08:00
A.J. Beamon effb6d2d49 Add ResolverMetrics trace event 2020-03-05 10:49:21 -08:00
Alex Miller 595dd77ed1 Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh 2020-03-04 20:25:42 -08:00
Alex Miller 9b5ef3416e Refactor TLSParams into TLSConfig + LoadedTLSConfig
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.

initTLS now just takes the in-memory bytes and applies them to the ssl
context.

This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
A.J. Beamon 58e621eca1 Invalid knobs or knob values are treated as warnings rather than errors. Apply this change to backup as well. 2020-03-04 15:50:04 -08:00
Evan Tschannen c11c24b79d removed the fdbrpc version of platform.h 2020-02-28 14:56:10 -08:00