bbf5bdf6da
fix stable interfaces test and corresponding changes in simulator
2020-10-12 18:25:12 +00:00
41843f07e6
add simulator support for different process versions and ProtocolVersion test
2020-10-12 18:19:31 +00:00
862336de8f
Merge branch 'master' into mengxu/merge-to-master-PR
2020-09-24 17:06:00 -07:00
58e043c7a5
Enable run loop profiler for test and multitest roles.
2020-09-24 14:14:55 -07:00
5f7e784742
Merge pull request #3796 from xis19/master
...
Refactor fdbserver.actor.cpp for better readability
2020-09-24 09:12:47 -07:00
581427c880
fixup! Remove "\n" and extra spaces in printUsage
2020-09-23 12:03:09 -07:00
e2ea6bbdd0
fixup! Use currLine.size() for the first currLength in printOptionUsage
2020-09-23 11:19:02 -07:00
2bf4ebbd51
fixup! Add missing comma
2020-09-21 20:43:07 -07:00
3c505786ca
Refactor fdbserver.actor.cpp
2020-09-21 20:19:47 -07:00
0814841827
Replace NULL with nullptr in fdbserver
2020-09-20 11:31:49 -07:00
35bef73a1c
Rename proxy to commit proxy
2020-09-10 17:44:15 -07:00
a49cb41de7
Merge branch 'release-6.3'
...
# Conflicts:
# CMakeLists.txt
# cmake/ConfigureCompiler.cmake
# fdbserver/Knobs.cpp
# fdbserver/StorageCache.actor.cpp
# fdbserver/storageserver.actor.cpp
# flow/ThreadHelper.actor.h
# flow/serialize.h
# tests/CMakeLists.txt
2020-07-29 00:31:55 -07:00
38cade1146
Merged tag release-6.3.3 into tls-background-handshake.
2020-07-17 23:39:57 -07:00
0131f0dc48
Make simulation able to load and run .toml files
2020-07-12 14:47:39 -07:00
cc60597739
Merge branch 'master' into bugfixes/visibility-perf-regression
2020-07-09 17:04:04 -06:00
1b68665228
Merge branch 'master' into release-6.3
2020-07-08 18:52:05 -07:00
0fbe7101c3
Revert "Revert "Request tracing""
...
This reverts commit 327cc31e35
2020-07-07 10:06:13 -06:00
dd5cd762a8
Removed dead code
2020-07-04 12:50:40 -07:00
0da4c91ad5
Performance improvements in socket use and buffering which results in TLS network connections using around 3.5x less CPU.
...
PacketBuffers are 16k minimum sizes, using FastAlloc. All calling paths to PacketBuffer creation now pass a size hint if known. P2PNetworkTest improved in many ways, errors go to TraceEvents instead of stdout, error counts are reported in stats, range min/max parameters are combined into "min:max" strings, new options to "stutter" connections by delaying randomly before socket read or write, message size to send is now configured separately for incoming vs outgoing sessions.
2020-06-25 20:44:43 -07:00
327cc31e35
Revert "Request tracing"
2020-06-16 12:32:42 -07:00
8595813b7d
Merge pull request #3355 from apple/release-6.3
...
Merge Release 6.3 into master branch
2020-06-12 20:08:47 -07:00
444a5a0ec8
Set up blob credentials for backup workers
...
This is needed so that backup workers (fdbserver) can upload files to blobs.
2020-06-11 17:26:21 -07:00
09c136e434
Framework for transaction tracing
2020-06-08 16:09:37 -07:00
d128252e90
Merge release-6.3 into master
2020-05-22 09:25:32 -07:00
b589c3c940
Remove IncludeVersions.h, which is no longer needed for cmake-only builds
2020-05-20 13:23:02 -07:00
3ff2fa9c2a
Removed uses of outdated cpuTicks and errorCounts
2020-05-13 23:27:44 -07:00
27da91ab9e
Merge remote-tracking branch 'upstream/master' into mutation-debugging
2020-05-13 12:51:44 -07:00
a833724322
Merge pull request #3078 from xumengpanda/mengxu/fr-circus-stall-PR
...
Performant restore: Various improvements based on circus test
2020-05-07 20:07:23 -07:00
a93c23d239
Resovle review comments
2020-05-07 15:06:59 -07:00
78d644f673
Merge branch 'master' into bugfixes/cmake-includes
2020-05-04 02:11:46 -07:00
98639645b1
fdbserver: update headers
2020-04-30 18:11:23 +00:00
b7f5f3be48
merge in master
2020-04-28 13:11:47 -07:00
c87aa33941
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/go/src/fdb/generated.go
# documentation/sphinx/source/api-common.rst.inc
# documentation/sphinx/source/api-ruby.rst
# documentation/sphinx/source/release-notes.rst
# fdbclient/FailureMonitorClient.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/vexillographer/fdb.options
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/OldTLogServer_6_0.actor.cpp
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-04-23 13:47:53 -07:00
2ce539ef6d
Respect flow<->fdbrpc module boundaries.
...
Which fixes a compilation error due to a circular dependency between
flow.a and fdbrpc.a. However, this is now done at the cost of newNet2
users have to remember to add Net2FileSystem::stop() as a callback.
2020-04-20 02:53:07 -07:00
ba3e2af473
Merge commit '5288033bcfe40c3ade97c8bf2d04cf31b3f16cb1' into feature-tree-broadcast
2020-04-17 15:17:37 -07:00
e104a2e3a6
Merge commit 'cf01233f28a2c42908656a39f458a4475c1d44a3' into run-loop-busy-profiler
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/NativeAPI.actor.h
# fdbserver/fdbserver.actor.cpp
# flow/Net2.actor.cpp
2020-04-14 14:02:24 -07:00
07cc0a8d74
code cleanup
2020-04-10 17:02:11 -07:00
52860043c9
Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs
2020-04-08 12:26:21 -07:00
e5b2cd81d5
FastRestore:Cleanup debug code
2020-04-07 15:56:44 -07:00
e4611e8ae4
fix versions.h stupidity
2020-04-06 10:28:55 -07:00
8b5780c36c
don't include source and binary dir
...
This forces users to use include paths from the sources root.
So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
536e65cd76
FastRestore:Introduce debugFRMutation for debug keys
2020-04-05 15:00:36 -07:00
a81ec332a9
FastRestore:Fix:Master cannot throttle on in progress version batches when it release batches out of order in simulation
2020-04-04 17:34:26 -07:00
884e92bb49
Atomically update dependent knobs
2020-04-01 15:18:49 -07:00
72e5891058
Clean up and rework the debugMutation API.
...
As a relatively unknown debugging tool for simulation tests, one could
have simulation print when a particular key is handled in various stages
of the commit process. This functionality was enabled by changing a 0
to a 1 in an #if, and changing a constant to the key in question.
As a proxy and storage server handle mutations, they call debugMutation
or debugKeyRange, which then checks against the mutation against the key
in question, and logs if they match. A mixture of printfs and
TraceEvents would then be emitted, and for this to actually be usable,
one also needs to comment out some particularly spammy debugKeyRange()
calls.
This PR reworks the API of debugMutation/debugKeyRange, pulls it out
into its own file, and trims what is logged by default into something
useful and understandable:
* debugMutation() now returns a TraceEvent, that one can add more details to before it is logged.
* Data distribution and storage server cleanup operations are no longer logged by default
2020-03-27 03:30:28 -07:00
303df197cf
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# bindings/c/test/mako/mako.c
# documentation/sphinx/source/release-notes.rst
# fdbbackup/backup.actor.cpp
# fdbclient/NativeAPI.actor.cpp
# fdbclient/NativeAPI.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/Knobs.cpp
# fdbserver/Knobs.h
# fdbserver/LogRouter.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/fdbserver.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/flow.vcxproj
# flow/flow.vcxproj.filters
# versions.target
2020-03-06 18:22:46 -08:00
dbfc0cbcc0
Merge pull request #2781 from alexmiller-apple/certificate-refresh
...
Refresh certificates used for handshaking when they change on disk
2020-03-06 11:12:04 -08:00
effb6d2d49
Add ResolverMetrics trace event
2020-03-05 10:49:21 -08:00
595dd77ed1
Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh
2020-03-04 20:25:42 -08:00
9b5ef3416e
Refactor TLSParams into TLSConfig + LoadedTLSConfig
...
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.
initTLS now just takes the in-memory bytes and applies them to the ssl
context.
This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
58e621eca1
Invalid knobs or knob values are treated as warnings rather than errors. Apply this change to backup as well.
2020-03-04 15:50:04 -08:00
c11c24b79d
removed the fdbrpc version of platform.h
2020-02-28 14:56:10 -08:00
6054c05963
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# fdbserver/fdbserver.actor.cpp
# versions.target
2020-02-28 12:11:05 -08:00
c3299b8ebe
if tls cannot be initialized, throw an error from createDatabase
2020-02-26 18:53:06 -08:00
bf5a95e6df
Merge commit 'dc39bdfbbf94a7f470386f439df08c044d08d90c' into feature-tls-environment-vars
...
# Conflicts:
# flow/Net2.actor.cpp
2020-02-26 18:02:56 -08:00
d1598e7c99
set_verify_peers throws an error instead of returning a value
2020-02-26 16:06:16 -08:00
2586bade68
re-added support for configuration TLS options with environment variables
2020-02-26 15:33:48 -08:00
0f5c999d4b
Better containment of boost errors related to TLS.
2020-02-26 12:26:43 -08:00
96258b9809
Merge branch 'release-6.2'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbcli/fdbcli.actor.cpp
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/FlowTransport.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/DataDistribution.actor.h
# fdbserver/DataDistributionQueue.actor.cpp
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/QuietDatabase.actor.cpp
# fdbserver/SkipList.cpp
# fdbserver/StorageMetrics.actor.h
# fdbserver/TLogServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# fdbserver/storageserver.actor.cpp
# fdbserver/workloads/KVStoreTest.actor.cpp
# flow/CMakeLists.txt
# flow/Knobs.cpp
# flow/Knobs.h
# flow/genericactors.actor.cpp
# flow/serialize.h
2020-02-21 19:09:16 -08:00
927cff3317
Report errors on TLS misconfigurations ... or at least try to.
2020-02-20 16:57:29 -08:00
08c318d28a
re-added the connect lock in the fdbcli so that the timeout is not spent before a connection has been initiated (because of the handshake lock)
2020-02-20 10:43:34 -08:00
761da5a059
code cleanup
2020-02-19 17:59:45 -08:00
88d36af9c7
Fix --tls_password and add better error logging
...
This refactors all tls settings into a TLSParams object so that we can
set the password before loading any certificates.
It turns out that the FDBLibTLS code did really nice things with error
logging, but I just didn't understand openssl enough before to realize
what pieces I should be copying.
2020-02-19 00:57:05 -08:00
31a6ec34b7
Merge branch 'master' into mengxu/fast-restore-agent-PR
2020-02-18 16:17:59 -08:00
e71fe44ee3
Merge branch 'master' into features/icc
2020-02-08 21:33:02 -08:00
e390dbd36c
Add a non-FDBLibTLS verify peers framework to new TLS impl
2020-02-06 21:06:52 -08:00
38d8d0d675
fixed simulation
2020-02-06 19:29:31 -08:00
844c8511c4
Merge pull request #2588 from jzhou77/backup-worker
...
Integrate new backup worker with existing backup command
2020-02-05 14:14:43 -08:00
84853dd1fd
switched SSL implementation to use boost ssl
2020-02-04 14:56:40 -08:00
d09e07f1f5
Merge remote-tracking branch 'upstream/master' into features/icc
2020-02-04 10:26:18 -08:00
7c10683c77
Backup workers save logs into right containers
...
The mutation logs of backup workers are saved into "mlogs" directory under the
container directory. The backup worker has been restructured to handle multiple
backups, where each one is stored in a separate backup container.
In the backup worker, mutations pulled from TLogs are buffered in a message
queue. When writing out to different containers, their corresponding mutation
ranges are used to check if a mutation should be written. When a new backup
is submitted by the client, "backupStartedKey" is updated. The worker monitors
this key, updates its internal map of backups, and then next pull from TLog
needs to wait for the readiness of the new backup. This is to ensure when
worker 0 sets the backup is started, all workers have already been logging
mutations for the backup.
2020-02-03 20:27:14 -08:00
3b57bf1781
Merge branch 'master' into mengxu/fast-restore-agent-PR
2020-02-03 17:23:54 -08:00
4524831456
Merge pull request #2518 from vishesh/task/failmon-remove-server
...
FailureMonitoring: Server processes no longer need to talk to ClusterController
2020-02-03 17:22:50 -08:00
9c2046b11b
FastRestore:Minic fdbd to monitor coordintors
...
Before we start a fdb restore process.
2020-02-03 14:48:31 -08:00
559b95c61a
FastRestore:RestoreRole:Mimic how fdbd starts
2020-02-01 10:23:48 -08:00
8ef56e5cae
FastRestore:LeaderElection:Log election progress
2020-01-30 20:03:14 -08:00
ff92401ed5
FastRestore:Add FastRestoreClass and blob option
...
To simplify test in circus framework, we need a fastrestore class;
To get data from blob in real mode, restore worker should set up
blob credentials in order to call BackupContainer interface to
get all backup files.
2020-01-28 20:25:05 -08:00
182dac7cd5
Convert the slow task profiler into a run loop profiler that also logs when the run loop is 100% busy for a knob-configurable duration.
2020-01-28 12:09:37 -08:00
31a1106286
Save mutations to backup files in simulation
...
This is the first step in the new backup's data pipeline. Verification of file
content is needed in future commits. A clear documentation of file format is a
work in progress.
2020-01-22 19:38:45 -08:00
daef5f011a
Merge remote-tracking branch 'apple/master' into task/failmon-remove-server
2020-01-21 13:20:15 -08:00
3f9d9d8b84
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# cmake/FlowCommands.cmake
# documentation/sphinx/source/release-notes.rst
# fdbclient/StorageServerInterface.h
# fdbserver/DataDistributionTracker.actor.cpp
# fdbserver/MasterProxyServer.actor.cpp
# fdbserver/fdbserver.actor.cpp
# flow/Knobs.h
# flow/Platform.cpp
# versions.target
2020-01-16 18:37:47 -08:00
17e97f24e4
Merge pull request #2526 from etschannen/feature-dd-improvements
...
Data distribution improvements
2020-01-10 17:53:22 -08:00
e4fa4ad0c9
Data distribution will not merge a shard unless it has been low bandwidth for 5 minutes
2020-01-09 17:02:49 -08:00
6e6cfaff16
Cleanup old Failure Monitoring code
2020-01-07 15:53:32 -08:00
9866d1ce27
Revert change to make g_trace_clock thread_local, instead checking we are on the correct thread when getting the time.
2019-12-06 10:15:49 -08:00
ed8d3f163c
Rename hgVersion to sourceVersion.
2019-11-15 12:26:51 -08:00
de8921b660
Move RestoreWorkerInterface to fdbclient
2019-10-25 10:42:22 -07:00
d4de608bb6
Fix OPEN_FOR_IDE build
2019-10-25 10:42:22 -07:00
bb0ae31002
Removed dead code.
2019-10-18 17:06:48 -07:00
3ba8fd95b5
Add script to parse output from enabling ALLOC_INSTRUMENTATION_STDOUT
2019-10-08 15:50:47 -07:00
e26270b17f
Fix compilation errors after merge 6.2
2019-10-02 13:28:24 -07:00
d0147e5e5d
Merge branch 'release-6.2' into mengxu/merge-release620-to-master-v3
...
Resolved Conflicts:
documentation/sphinx/source/release-notes.rst
fdbserver/DataDistribution.actor.cpp
versions.target
2019-10-02 13:22:56 -07:00
9a64dc7f24
Merge pull request #1792 from ajbeamon/remove-dl-iterate-phdr-hack
...
Remove dl_iterate_phdr caching used by slow task profiler
2019-09-17 16:51:30 -07:00
eb29541acb
Update fdbserver/fdbserver.actor.cpp
...
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-09-13 15:10:24 -07:00
8875fe7aac
Don't require at leats 2GiB over cache_memory
2019-09-13 13:02:06 -07:00
8c1c0296ed
Verify that we have enough memory if cache_memory is set
2019-09-13 12:54:37 -07:00
048c341a7d
FastRestore:Bug fix after merge with master
...
Include RestoreWorkerInterface.h instead of RestoreInterface.h into fdbserver.actor.cpp
Report warning instead of error when unlockDatabase throws error.
2019-09-04 21:14:09 -07:00
24aad14f06
Merge branch 'release-6.2'
...
# Conflicts:
# CMakeLists.txt
# documentation/sphinx/source/release-notes.rst
# versions.target
2019-08-30 17:23:58 -07:00
45373bd04c
Attempt to fix some merge madness
2019-08-22 10:17:15 -07:00
e465cbef07
Merge branch 'master' into remove-dl-iterate-phdr-hack
...
# Conflicts:
# fdbserver/fdbserver.actor.cpp
2019-08-22 10:13:23 -07:00
f02799455e
Add --loggroup to fdbserver and fdbbackup help text.
2019-08-19 12:59:14 -07:00
2a9eeddf5a
Sort roles
2019-08-09 13:10:12 -07:00
7bd7ef7952
Group like CLIOptions members
2019-08-09 13:09:59 -07:00
4ed83752a5
Extract argv parsing code from main
...
Currently main has many non-const local variables that represent parsed
options. These variables get changed by parsing code intermixed with
code that does things with these variables. This commit extracts the
parsing code from main and moves the local variables to a const struct.
2019-08-08 10:09:50 -07:00
370ba8b841
Remove --object-serializer flag from executables
2019-08-06 09:25:40 -07:00
476641a087
Merge pull request #1929 from jzhou77/fix-warning
...
Fix compiler warnings
2019-08-01 11:15:41 -07:00
37450be706
Fix format usage for currentProtocolVersion
...
ProtocolVersion now is a class.
2019-08-01 10:19:46 -07:00
a731adeb8f
--machine_id now sets locality_machineid
2019-07-30 16:11:09 -07:00
25f93f7f1b
Revert change to machine_id documentation (to be fixed in separate PR).
2019-07-30 15:20:57 -07:00
924c51274d
Move memory and locality arguments from --dev-help to --help. Also update -i/--machine_id to note that it modifies the zone identifier key (depite the name of the parameter, which I'm not changing now).
2019-07-30 14:34:27 -07:00
cfc985cdf1
re-enabled flat buffers, fixed the latencyBandConfig serialization
2019-07-27 17:48:24 -07:00
9871045cc7
flat buffers is causing an infinite loop when serializing LatencyBandConfig::GrvConfig
2019-07-27 16:34:18 -07:00
1a4ca05a04
Merge pull request #1889 from ajbeamon/add-cache-memory-parameter
...
Add cache_memory parameter to fdbserver
2019-07-26 13:34:24 -07:00
9afd162e2f
remove snap v1 related code
2019-07-25 17:29:31 -07:00
a92b6cd3d1
Merge branch 'master' into add-cache-memory-parameter
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
2019-07-25 13:41:57 -07:00
869f77aef1
Few cosmetic edits and fixes
2019-07-24 15:36:28 -07:00
a954cf4e06
snap v2: restore related changes for the simulator
2019-07-24 15:36:28 -07:00
94be9560ea
Add cache_memory parameter to fdbserver to control the size of the (4K) page cache. Change the default slighty from 2000 MiB to 2GiB.
2019-07-23 15:05:21 -07:00
1ac2d01b03
Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2
2019-07-18 09:50:08 -07:00
d5caf0c1b4
Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2
2019-07-16 14:47:40 -07:00
d5051b08dd
Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated.
2019-07-12 16:12:35 -07:00
6a899ddff3
Remove dl_iterate_phdr results caching that is used by slow task profiler, instead favoring disabling and reenabling profiling around the call. Add a mechanism to handle deferred profile requests.
2019-07-03 12:48:36 -07:00
bf883d7055
Merge remote-tracking branch 'upstream/master' into flowlock-api
2019-06-25 14:26:50 -07:00
7a500cd37f
A giant translation of TaskFooPriority -> TaskPriority::Foo
...
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
2019-06-25 02:47:35 -07:00
5ce45a8a2d
Addressed review comments.
2019-06-20 23:03:49 -07:00
844dd60202
FDB compiling with intel compiler
2019-06-20 09:29:01 -07:00
4832404c85
Make public address and listen address a comma separated list
2019-06-18 18:15:15 -07:00
6ea75713cb
Overall framework and first buggify entries
2019-06-16 09:09:09 -07:00
6b4d30c3ae
failmon: Identify client vs server when starting failure monitoring client
2019-06-09 00:43:12 -07:00
42c551a996
handle isRestoring & BackupFailed not being set
...
restartInfo.in->BackupFailed and isRestoring may not be
set in all cases, handle the absence of them.
2019-05-28 22:07:46 -07:00
ceac68c990
restore - remove emtpy snapdir,snap loop retry fix
...
- remove partially snapped directories to avoid no cluster file assert
- snap create to retry max 3 times for not_fully_recovered and keep
retrying for the other failures
2019-05-28 22:07:46 -07:00
bb474dc323
if recovery < fully_recovered then fail the exec
...
Will do more cleanup, pushing it for a test run in CI
2019-05-28 22:07:46 -07:00
936ffc2dde
rebase related changes
2019-05-28 22:07:46 -07:00
ec7834e2f7
code re-orgnaization and address comments
2019-05-28 22:07:46 -07:00
61e93a9304
Address review comments and minor fixes
2019-05-28 22:07:46 -07:00
898bed66c1
Allow only whitelisted binary path for exec op
2019-05-28 22:07:46 -07:00
a60145b9a1
Restore the cluster in single region configuration
2019-05-28 22:07:46 -07:00
382b246930
trace change and retain fitness file after restore
2019-05-28 22:07:46 -07:00
281c785f94
'--restoring' cmd line arg removed for fdbserver
...
'--restoring' command line option was introduced to indicate
simulated fdbserver to restore from snapshot and restart the cluster.
As part of this change that option is removed and restore
information is stored in the restartInfo.ini.
2019-05-28 22:07:46 -07:00
4bc4c615da
exec op to all tlog, restore change in test &other
...
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
2019-05-28 22:07:46 -07:00
603721e125
Merge branch 'master' into thread-safe-random-number-generation
...
# Conflicts:
# fdbclient/ManagementAPI.actor.cpp
# fdbrpc/AsyncFileCached.actor.h
# fdbrpc/genericactors.actor.cpp
# fdbrpc/sim2.actor.cpp
# fdbserver/DiskQueue.actor.cpp
# fdbserver/workloads/BulkSetup.actor.h
# flow/ActorCollection.actor.cpp
# flow/Net2.actor.cpp
# flow/Trace.cpp
# flow/flow.cpp
2019-05-23 08:35:47 -07:00
5a8c97480a
Merge pull request #1506 from nikolas-ioannou/feature-pagecache-lru
...
AsyncFileCached: switch from a random to an LRU cache eviction policy
2019-05-17 13:42:21 -07:00
53b97fe506
Extend support for parentpid
...
This adds support for the `--parentpid` option to non-Windows platforms.
This option is intended for testing layer implementations. When running
higher level CI chains its useful to ensure that any ephemeral instances
of fdbserver are automatically reaped.
2019-05-16 14:24:11 -10:00
20c3f7f264
remove mixed-mode support
2019-05-13 14:15:23 -07:00
9eeb48c43d
Allow to turn on object serializer
...
This commit includes functionality to turn on
the object serializer for network communication.
This is done the following way:
- On incoming connections, a process will detect
whether the client supports the object serializer
and will only serialize responses with it, if it does
- On outgoing connections, the command line flag is used
to determine whether the object serializer should be used
to send data.
This way, a cluster can run in mixed mode. To upgrade one
can upgrade one process at a time and set the flag one process
at a time.
This is how this is tested on the simulator:
- The command line flag can take three options: on, off,
and random.
- For off, the object serializer will never we used.
- For on, the object serializer will be always used.
- For random, the simulator will flip a coin for each
process it starts up.
2019-05-13 14:15:22 -07:00
067cdf9cde
Simplified cache eviction policy knob arg check.
2019-05-13 08:50:04 +02:00
5f55f3f613
Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used.
2019-05-10 14:01:52 -07:00
5793b1a55e
Validate cache eviction policy value after knob args have been set.
2019-05-07 08:32:57 +02:00
bf378952cb
fdbserver: fix some print/scan format warnings
...
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
2d5043c665
Merge branch 'release-6.1'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# versions.target
2019-04-30 18:27:04 -07:00
1f37f82b87
invalid knob overrides do not prevent fdbserver from starting
2019-04-25 17:08:13 -07:00
6870e132b2
Merge branch 'master' into pprof
2019-04-19 14:06:44 -07:00
ab834c4f7e
Move profiling option help message to devhelp
2019-04-09 13:26:12 -07:00
6220a5ce0f
Merge pull request #1370 from jzhou77/fix-unreferenced
...
Remove unused functions
2019-04-09 11:49:45 -07:00
4b08042a88
Change memory profiling threshold to a flag
2019-04-05 16:33:51 -07:00
1c16f87a4e
Remove trace-calls to printable (in non-workloads)
2019-04-05 13:12:19 -07:00
f7f8ddd894
Fix warnings on unused variables
...
Found by -Wunused-variable flag.
2019-04-01 14:00:20 -07:00
7c02ee6fdd
Fix compiler warning about unreferenced exception variable
2019-03-26 13:43:47 -07:00
24c92a1870
Merge pull request #1352 from etschannen/feature-network-address-list
...
Changed NetworkAddressList to at most two addresses for performance
2019-03-24 10:22:38 -07:00
50a4403661
fix: missing parathesis
2019-03-23 21:52:15 -07:00
1fc6937802
changed NetworkAddressList to at most two addresses for performance
2019-03-23 17:54:46 -07:00
ac8ad07b45
Address review comments.
2019-03-22 18:48:49 -07:00
4ed323ac52
Fixed bug and addressed review comments.
2019-03-22 18:48:49 -07:00
d75020b44a
Fix bug where accessing shared memory created by boost 1.52 leads to error when accessed by boost 1.67.
2019-03-22 18:48:49 -07:00
fea18e7be0
fix: fdbserver segfault when started with wrong arguments
...
Public address is required for roles FDBD, NetworkTestServer and
Restore only. Therefore, check those cases, and for others follow the
earlier behaviour of using default ip address 0.
FIXES #1305
2019-03-19 02:05:11 -07:00
c5a18945b6
Merge pull request #1260 from vishesh/task/tls-upgrade
...
Allows cluster string to contain coordinators with different TLS states
2019-03-12 13:45:08 -07:00
ebffe8c633
print correct pahes in alloc instrumentation
2019-03-08 15:03:17 -08:00
ed49d603a0
Allows cluster string to contain coordinators with different TLS states
...
During live TLS upgrades, we can hence switch one coordinator at a time
to TLS than all of them together.
2019-03-06 16:05:10 -08:00
981426bac9
More ide fixes
2019-03-05 18:03:57 -08:00
a9562f61be
fix: missing argument to printf in fdbserver
2019-03-05 14:03:09 -08:00
57832e625d
net: Support IPv6 #963
...
- NetworkAddress now contains IPAddress object which can be either
IPv4 or IPv6 address. 128bits are used even for IPv4 addresses,
however only 32bits are used when using/serializing IPv4 address.
- ConnectPacket is updated to store IPv6 address. Backward compatible
with old format since the first 32bits of IP address field is used
for serialization of IPv4.
- Mainly updates rest of the code to use IPAddress structure instead
of plain uint32_t.
- IPv6 address/pair ports should be represented as `[ip]:port` as per
convention. This applies to both cluster files and command line
arguments.
2019-03-04 14:12:41 -08:00
c46252eacd
fix: Don't disallow FDB listen addresses which are not in cluster file #1189
...
Rejects listen addresses if the IP:PORT matches with that found in
cluster file, but the TLS state is different.
2019-02-26 21:26:27 -08:00
999ea09bfd
Use correct fwd decls in TesterInterface
...
Also TesterInterface.h -> TesterInterface.actor.h
2019-02-19 15:16:59 -08:00
699216f713
Use fwd decls in workloads
...
Also workloads.h -> workloads.actor.h
2019-02-19 15:16:59 -08:00
3f0fd2a20c
Use fwd decls in WorkerInterface
...
Also WorkerInterface.h -> WorkerInterface.actor.h
2019-02-19 15:16:59 -08:00
27a3153719
Use ACTOR forward declarations in MoveKeys
...
Also MoveKeys.h -> MoveKeys.actor.h
2019-02-19 15:16:59 -08:00
0bb60e5a3b
Use proper fwd decl in NativeAPI
...
Also NativeAPI.h -> NativeAPI.actor.h
2019-02-19 15:16:59 -08:00
0898686c9b
Remove old TODO
2019-02-18 15:43:27 -08:00
d34a658357
Add Restore role previous removed by mistake
2019-02-15 20:25:03 -08:00
c03de6c7b6
Update CLI to take addresses with mixed TLS states
2019-02-15 20:23:07 -08:00
907446d0ce
Merge remote-tracking branch 'apple/master' into task/tls-upgrade
2019-02-14 11:37:38 -08:00
6da5971e79
Guard all versions.h to not break old WIN32 build
2019-02-08 16:06:00 -08:00
7e26b4ef0d
Address comments from PR
2019-02-07 15:37:04 -08:00
5737349676
Fix weird bug with boost interprocess
...
Strangely, boost interprocess didn't compile with VS 2017.
However, it does compile if it is included as the first thing.
I don't quite know what is happening here, but for now this fix
makes it that I am not blocked
2019-02-07 15:37:04 -08:00
8a94d80deb
fdbservice and fdbrpc now compiling
2019-02-07 15:37:04 -08:00
1d7fec3074
Merge commit '048bfc5c368063d9e009513078dab88be0cbd5b0' into task/tls-upgrade-2
...
# Conflicts:
# .gitignore
2019-01-24 17:43:06 -08:00
58964af7e1
ctest improvements - #1058
...
- A set of CMake variables controls whether to keep
the simfdb directory and the traces and whether we
want to aggregate the traces into a single file
- Test labels now contain the directory they are in
so that one can now run `ctest -R fast/`
- A different binary can be used for restart tests. CMake
will automatically look for an installed fdb and use that
by default. If none is found, it will use the built one
but it will also print a warning
- CMake will throw an error if there are any text files in
the tests directory that are not associated with a test.
- Moved testing from fdbserver/CMakeLists.txt to
tests/CMakeLists.txt
- Moved fdb testing functions to its own cmake module
2019-01-22 14:34:51 -08:00
b096b8e3f8
First tests working with ctest
2019-01-14 19:14:25 -08:00
7eb6765698
Mention that xml is the default
2019-01-03 08:48:31 -08:00
bce5b03340
Fix whitespace
2019-01-02 15:24:11 -08:00
1bca665b29
Document --trace_format flag
2018-12-20 16:22:41 -08:00
b8df5acc15
Add --trace_format flag to fdbserver
2018-12-20 15:02:01 -08:00
3eb9b23024
Listen to multiple addresses and start using vector<NetworkAdddress> in Endpoint
...
- This patch will make FDB listen to multiple addresses given via
command line. Although, we'll still use first address in most places,
this patch starts using vector<NetworkAddress> in Endpoint at some basic
places.
- When sending packets to an endpoint, pick a random network address in
endpoints
- Renames Endpoint::address to Endpoint::addresses since it
now holds a vector of addresses.
2018-12-13 13:36:52 -08:00
42dffd4dff
Take a vector of network addresses from CLI to start FDB server
...
Extends the CLI interface to take multiple public and listen addresses.
We however do not do anything with those extra addresses and just
consider the first one for now.
2018-12-13 13:36:52 -08:00
e8e01b2406
Remove unused localAddress parameter from newNet2 and Net2 classes
2018-12-13 13:36:52 -08:00
268093a96d
Adjust all includes to be relative to the root.
...
Remove the use of relative paths. A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h". Adjust so that every include references such a header with the
latter form.
Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
0acfae1e76
fixed the windows linker error
2018-10-15 18:19:51 -07:00
4c95a5ee0f
added the basic structure for parallel restore
2018-10-09 18:47:28 -07:00
3922e477a5
Merge branch 'release-6.0'
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
# fdbclient/ManagementAPI.actor.cpp
# fdbserver/ClusterController.actor.cpp
# fdbserver/DataDistribution.actor.cpp
# fdbserver/LogSystemDiskQueueAdapter.actor.cpp
# fdbserver/SimulatedCluster.actor.cpp
# fdbserver/TLogServer.actor.cpp
2018-10-03 16:57:18 -07:00
b727f0475c
Fix warnings about ACTORs not having waits. Fix shadowing of future variable in KVFileIntegrityCheck.
2018-09-10 10:51:41 -07:00
Richard Chen