Commit Graph

210 Commits

Author SHA1 Message Date
Evan Tschannen dcdbb3ec4d Merge branch 'release-6.0' of github.com:apple/foundationdb into feature-movekey-fixes 2018-09-05 10:29:13 -07:00
Evan Tschannen 40f5dbe423 fixed issues from review, added a safeguard to prevent configuring a cluster to an invalid configuration 2018-09-04 22:16:35 -07:00
Evan Tschannen 21f5cf9ce9 suppress spammy trace events 2018-09-04 17:12:26 -07:00
Evan Tschannen d8ea3dbf9a Added the ability to configure a cluster from a JSON file 2018-08-16 17:34:59 -07:00
Evan Tschannen 1c29275672 call all methods which could disable a trace event before it is initialized. In practice this means calling .error first, then .suppressFor, then all your details. 2018-08-01 14:30:57 -07:00
Evan Tschannen 2820b6e0bb data inconsistency is always an error when detected by the consistency check 2018-07-09 22:26:13 -07:00
Evan Tschannen 82cc30be62 added testing for two_satellite_fast and two_satellite_safe 2018-07-09 22:01:46 -07:00
Evan Tschannen cd4fb9285a waitForExlusion requires both regions to be healthy, which is only possible if we do not kill all logs in a region 2018-07-05 14:04:42 -07:00
Evan Tschannen da5a232d7e fix: If we have not recruited the remote logs yet and detect a configuration change, we must fail the master to update the remote recruitment request 2018-07-05 12:17:41 -07:00
Evan Tschannen e17dfea3b6 fix: desiredTLogCount was used instead of getDesiredLogs(), which caused problems with recruitment when desiredTLogCount was -1.
canKillProcess logic was wrong.
We still need to configure usable_regions because if datacenterVersionDifference is too large we cannot complete data movement.
2018-07-04 16:22:32 -04:00
Evan Tschannen 604b3bca17 increased the api correctness timeout 2018-07-02 12:51:50 -04:00
Evan Tschannen 89a4b2cd68 fix: consistency check could loop too long 2018-07-02 12:08:02 -04:00
Evan Tschannen 73e61312c6 fix: shareLogRange was never initialized 2018-07-01 22:49:24 -04:00
Evan Tschannen 4a3247da69 fixed a few problems with the consistency check 2018-06-30 10:39:28 -07:00
Evan Tschannen 02f616eb68 fix: consistency check was broken when the key server key space is sharded 2018-06-28 23:16:32 -07:00
Balachandar Namasivayam 8caa6eaecf
Merge pull request #541 from etschannen/feature-remote-logs
More multiple DC improvements
2018-06-28 11:22:08 -07:00
Evan Tschannen 45cf0067e4 fix: consistency check was not checking for data inconsistencies 2018-06-28 11:08:16 -07:00
A.J. Beamon 9f545ce002 Merge commit '892727e358c0b3f075564c60c2b7cedb64306f83' into trace-log-refactor 2018-06-26 11:37:23 -07:00
Evan Tschannen 1ccfb3a0f4 fix: log_anti_quorum was always 0 in simulation
removed durableStorageQuorum, because it is no longer a useful configuration parameter
2018-06-18 10:24:57 -07:00
Evan Tschannen 0913368651 added usable_regions to specify if we will replicate into a remote region
remote replication defaults to the primary replication
removed remote_logs, because they should be specified as an override in the regions object
2018-06-17 19:31:15 -07:00
Evan Tschannen 99e21c869c fixed a number of status calculations, and re-enabled the status workload 2018-06-14 17:58:57 -07:00
Richard Low 39894ea798 Merge remote-tracking branch 'apple/release-5.2' 2018-06-12 18:31:20 -07:00
Evan Tschannen 372ed67497 Merge branch 'master' into feature-remote-logs
# Conflicts:
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/TagPartitionedLogSystem.actor.cpp
2018-06-11 11:34:10 -07:00
A.J. Beamon f965954122 Merge commit '82be52205b95464e355c449fdf3e7d483fa06677' into trace-log-refactor
# Conflicts:
#	fdbserver/Status.actor.cpp
#	fdbserver/workloads/DDMetrics.actor.cpp
#	flow/Trace.cpp
2018-06-08 16:22:22 -07:00
Evan Tschannen b9826dc1cb fix: do not automatically reduce redundancy we move keys if the database does not have remote replicas. This is to prevent problems when dropping remote replicas from a configuration. 2018-06-08 16:17:27 -07:00
A.J. Beamon 99c9958db7 Some more trace event normalization 2018-06-08 13:57:00 -07:00
A.J. Beamon 0ca51989bb Merge branch 'master' into trace-log-refactor
# Conflicts:
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/Status.actor.cpp
#	flow/Trace.cpp
2018-06-08 13:24:30 -07:00
Balachandar Namasivayam 20febf5ef9 Address review comments. 2018-06-08 11:24:51 -07:00
A.J. Beamon e5488419cc Attempt to normalize trace events:
* Detail names now all start with an uppercase character and contain no underscores. Ideally these should be head-first camel case, though that was harder to check.
* Type names have the same rules, except they allow one underscore (to support a usage pattern Context_Type). The first character after the underscore is also uppercase.
* Use seconds instead of milliseconds in details.

Added a check when events are logged in simulation that logs a message to stderr if the first two rules above aren't followed.

This probably doesn't address every instance of the above problems, but all of the events I was able to hit in simulation pass the check.
2018-06-08 11:11:08 -07:00
Balachandar Namasivayam 514b0e3c20 Having fixed limits for getRange results in continuously getting transaction_too_old error in some scenarios.
Cutting the limits by half in such cases allows to test to progress.
2018-06-07 15:27:05 -07:00
Evan Tschannen b423d73b42 fix: do not finish a shard relocation until all of the storage servers have made the current recovery version durable. This is to prevent dropping a needed storage server as a source for a shard after dropping a remote configuration 2018-06-07 12:29:25 -07:00
A.J. Beamon 78839b20fd Merge branch 'master' into trace-log-refactor
# Conflicts:
#	flow/Trace.cpp
2018-05-31 10:46:20 -07:00
A.J. Beamon 026458baf3 Merge release-5.2 into master 2018-05-23 15:32:56 -07:00
Alec Grieser 40babc40e1
remove one unnecessary line ; fix else formatting 2018-05-15 17:20:44 -07:00
Alec Grieser 6d132717f2
add versionstamp compatibility test to VersionStampWorkload
surfaces error found in #387
2018-05-15 17:09:24 -07:00
A.J. Beamon 02df30149f Merge branch 'release-5.2' into trace-log-refactor 2018-05-11 11:22:34 -07:00
Evan Tschannen 91338fc984 Merge branch 'master' into feature-remote-logs 2018-05-10 15:33:45 -07:00
Evan Tschannen 8f984cb2c9 Merge branch 'release-5.2'
# Conflicts:
#	fdbrpc/TLSConnection.h
2018-05-10 09:13:22 -07:00
Evan Tschannen f6e55d0b74
Merge pull request #348 from etschannen/release-5.2
DR upgrade tests now test the durability of the data.
2018-05-09 15:40:03 -07:00
Evan Tschannen 8930c2e3db DR upgrade tests now test the durability of the data. 2018-05-09 15:11:05 -07:00
Alec Grieser 464e2cdbf0
change SetVersionstampedKey and SetVersionstampedValue behavior based on API version to make them consistent 2018-05-08 08:57:09 -07:00
Alec Grieser 14cca75429
server components of version of alternative versionstamp op that writes to an arbitrary place in the value 2018-05-08 08:57:08 -07:00
A.J. Beamon ce0c991e78 Refactor trace events to store a vector of fields that aren't encoded until write time. Better support for pre-network trace events. Rework how trace events are queried. Some initial work towards pluggable formatting of logs. 2018-05-02 10:44:38 -07:00
Evan Tschannen 656a817e74 fix: only reconfigure during the quiet database check, because excluding at the same time as reconfiguring causes the master to indefinitely restart recovery 2018-05-01 15:31:49 -07:00
Evan Tschannen 10d25927cd Merge branch 'master' into feature-remote-logs
# Conflicts:
#	fdbserver/DataDistribution.actor.cpp
2018-04-30 22:15:39 -07:00
Evan Tschannen 9cdabfed0e added useful trace events 2018-04-29 18:54:47 -07:00
Alec Grieser 69e831d522
Merge remote-tracking branch 'upstream/release-5.2' into merge-release-5.2 2018-04-28 17:44:52 -07:00
Yichi Chiang c721ab6854 Fix review comments 2018-04-27 13:54:34 -07:00
Yichi Chiang 6bddf8aefa Upgrade DR from 5.1 to 5.2 2018-04-26 17:24:40 -07:00
Evan Tschannen c1ccc8522c Merge branch 'release-5.2' 2018-04-17 18:38:12 -07:00