Commit Graph

143 Commits

Author SHA1 Message Date
Meng Xu 7ccaeddf05 Merge branch 'master' into mengxu/performant-restore-PR 2019-08-01 13:23:17 -07:00
A.J. Beamon 476641a087
Merge pull request #1929 from jzhou77/fix-warning
Fix compiler warnings
2019-08-01 11:15:41 -07:00
Jingyu Zhou 37450be706 Fix format usage for currentProtocolVersion
ProtocolVersion now is a class.
2019-08-01 10:19:46 -07:00
A.J. Beamon a731adeb8f --machine_id now sets locality_machineid 2019-07-30 16:11:09 -07:00
A.J. Beamon 25f93f7f1b Revert change to machine_id documentation (to be fixed in separate PR). 2019-07-30 15:20:57 -07:00
A.J. Beamon 924c51274d Move memory and locality arguments from --dev-help to --help. Also update -i/--machine_id to note that it modifies the zone identifier key (depite the name of the parameter, which I'm not changing now). 2019-07-30 14:34:27 -07:00
Evan Tschannen cfc985cdf1 re-enabled flat buffers, fixed the latencyBandConfig serialization 2019-07-27 17:48:24 -07:00
Evan Tschannen 9871045cc7 flat buffers is causing an infinite loop when serializing LatencyBandConfig::GrvConfig 2019-07-27 16:34:18 -07:00
Evan Tschannen 1a4ca05a04
Merge pull request #1889 from ajbeamon/add-cache-memory-parameter
Add cache_memory parameter to fdbserver
2019-07-26 13:34:24 -07:00
Meng Xu 1706aaf199 Merge branch 'master' into mengxu/performant-restore-PR
Fix conflict in TlogServer.actor.cpp by accepting master changes
2019-07-26 11:46:27 -07:00
sramamoorthy 9afd162e2f remove snap v1 related code 2019-07-25 17:29:31 -07:00
A.J. Beamon a92b6cd3d1 Merge branch 'master' into add-cache-memory-parameter
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2019-07-25 13:41:57 -07:00
Meng Xu 45083edf74 Merge branch 'master' into mengxu/performant-restore-PR
Fix conflicts as well.
2019-07-25 10:46:11 -07:00
sramamoorthy 869f77aef1 Few cosmetic edits and fixes 2019-07-24 15:36:28 -07:00
sramamoorthy a954cf4e06 snap v2: restore related changes for the simulator 2019-07-24 15:36:28 -07:00
A.J. Beamon 94be9560ea Add cache_memory parameter to fdbserver to control the size of the (4K) page cache. Change the default slighty from 2000 MiB to 2GiB. 2019-07-23 15:05:21 -07:00
mpilman 1ac2d01b03 Merge remote-tracking branch 'upstream/master' into flatbuffers-fixes2 2019-07-18 09:50:08 -07:00
mpilman d5caf0c1b4 Merge branch 'flatbuffers-fixes2' of github.com:mpilman/foundationdb into flatbuffers-fixes2 2019-07-16 14:47:40 -07:00
A.J. Beamon d5051b08dd Make trace event field lengths (and total event sizes) default knobified and configurable. Add a transaction option to control the field length of transaction debug logging. Make the program start command line field less likely to be truncated. 2019-07-12 16:12:35 -07:00
Alex Miller bf883d7055 Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-25 14:26:50 -07:00
Alex Miller 7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
2019-06-25 02:47:35 -07:00
Balachandar Namasivayam 5ce45a8a2d Addressed review comments. 2019-06-20 23:03:49 -07:00
Balachandar Namasivayam 4832404c85 Make public address and listen address a comma separated list 2019-06-18 18:15:15 -07:00
mpilman 6ea75713cb Overall framework and first buggify entries 2019-06-16 09:09:09 -07:00
Vishesh Yadav 6b4d30c3ae failmon: Identify client vs server when starting failure monitoring client 2019-06-09 00:43:12 -07:00
sramamoorthy 42c551a996 handle isRestoring & BackupFailed not being set
restartInfo.in->BackupFailed and isRestoring may not be
set in all cases, handle the absence of them.
2019-05-28 22:07:46 -07:00
sramamoorthy ceac68c990 restore - remove emtpy snapdir,snap loop retry fix
- remove partially snapped directories to avoid no cluster file assert
- snap create to retry max 3 times for not_fully_recovered and keep
  retrying for the other failures
2019-05-28 22:07:46 -07:00
sramamoorthy bb474dc323 if recovery < fully_recovered then fail the exec
Will do more cleanup, pushing it for a test run in CI
2019-05-28 22:07:46 -07:00
sramamoorthy 936ffc2dde rebase related changes 2019-05-28 22:07:46 -07:00
sramamoorthy ec7834e2f7 code re-orgnaization and address comments 2019-05-28 22:07:46 -07:00
sramamoorthy 61e93a9304 Address review comments and minor fixes 2019-05-28 22:07:46 -07:00
sramamoorthy 898bed66c1 Allow only whitelisted binary path for exec op 2019-05-28 22:07:46 -07:00
sramamoorthy a60145b9a1 Restore the cluster in single region configuration 2019-05-28 22:07:46 -07:00
sramamoorthy 382b246930 trace change and retain fitness file after restore 2019-05-28 22:07:46 -07:00
sramamoorthy 281c785f94 '--restoring' cmd line arg removed for fdbserver
'--restoring' command line option was introduced to indicate
simulated fdbserver to restore from snapshot and restart the cluster.
As part of this change that option is removed and restore
information is stored in the restartInfo.ini.
2019-05-28 22:07:46 -07:00
sramamoorthy 4bc4c615da exec op to all tlog, restore change in test &other
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
2019-05-28 22:07:46 -07:00
A.J. Beamon 603721e125 Merge branch 'master' into thread-safe-random-number-generation
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/genericactors.actor.cpp
#	fdbrpc/sim2.actor.cpp
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/workloads/BulkSetup.actor.h
#	flow/ActorCollection.actor.cpp
#	flow/Net2.actor.cpp
#	flow/Trace.cpp
#	flow/flow.cpp
2019-05-23 08:35:47 -07:00
Meng Xu fac63a83c4 FastRestore:Use NotifiedVersion to deduplicate requests
Add a NotifiedVersion into an applier data which represents
the smallest version the applier is at.

When a loader sends mutation vector to appliers, it sends
the request that contains prevVersion and commitVersion.

This commits also put actor into an actorCollector for
loop-choose-when situation.
2019-05-22 22:09:54 -07:00
Steve Atherton 5a8c97480a
Merge pull request #1506 from nikolas-ioannou/feature-pagecache-lru
AsyncFileCached: switch from a random to an LRU cache eviction policy
2019-05-17 13:42:21 -07:00
Paul J. Davis 53b97fe506 Extend support for parentpid
This adds support for the `--parentpid` option to non-Windows platforms.
This option is intended for testing layer implementations. When running
higher level CI chains its useful to ensure that any ephemeral instances
of fdbserver are automatically reaped.
2019-05-16 14:24:11 -10:00
mpilman 20c3f7f264 remove mixed-mode support 2019-05-13 14:15:23 -07:00
mpilman 9eeb48c43d Allow to turn on object serializer
This commit includes functionality to turn on
the object serializer for network communication.
This is done the following way:

- On incoming connections, a process will detect
  whether the client supports the object serializer
  and will only serialize responses with it, if it does
- On outgoing connections, the command line flag is used
  to determine whether the object serializer should be used
  to send data.

This way, a cluster can run in mixed mode. To upgrade one
can upgrade one process at a time and set the flag one process
at a time.

This is how this is tested on the simulator:
- The command line flag can take three options: on, off,
  and random.
- For off, the object serializer will never we used.
- For on, the object serializer will be always used.
- For random, the simulator will flip a coin for each
  process it starts up.
2019-05-13 14:15:22 -07:00
Nikolas Ioannou 067cdf9cde Simplified cache eviction policy knob arg check. 2019-05-13 08:50:04 +02:00
Meng Xu 3ecf8718f8 Fix unused variable error 2019-05-11 22:30:01 -07:00
A.J. Beamon 5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Nikolas Ioannou 5793b1a55e Validate cache eviction policy value after knob args have been set. 2019-05-07 08:32:57 +02:00
Meng Xu 25c75f4222 FastRestore: Add new empty files for restore roles
Add .h and .cpp files for RestoreLoader and RestoreApplier roles.
We will split the code for each restore role into a separate file.

This commit also fixes the bug in including RestoreCommon.actor.h, and
remove the unused code.
2019-05-06 16:59:41 -07:00
Austin Seipp bf378952cb fdbserver: fix some print/scan format warnings
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-05-06 13:35:29 -07:00
Evan Tschannen 2d5043c665 Merge branch 'release-6.1'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	versions.target
2019-04-30 18:27:04 -07:00
Evan Tschannen 1f37f82b87 invalid knob overrides do not prevent fdbserver from starting 2019-04-25 17:08:13 -07:00