Commit Graph

911 Commits

Author SHA1 Message Date
Balachandar Namasivayam eeebf10030 Modified existing behavior of MIN and AND atomic ops. The new behavior results in a 'SET' if the atomic op is performed on a non -existing key.
Added new atomic ops ByteMin and ByteMax that does lexicographic comparison of byte strings.
2017-10-10 13:02:22 -07:00
Evan Tschannen c8525dc3e7 timekeeper is constantly changing keys in the system keyspace, so do not report errors on key mismatches on keys in the system keyspace 2017-10-10 12:04:56 -07:00
Alex Miller a21c8a820b Move cpuProfilerRequest from WorkerInterface to ClientWorkerInterface.
A way to access this stream is required if we wish to be able to toggle
profiling from fdbcli.  There's two ways to do this:

1. Use `monitorLeader()` to get a `ClusterControllerFullInterface`, and use
`getWorkers` from there to get a list of `WorkerInterface`s, from which we can
access cpuProfilerRequest.
2. Move cpuProfilerRequest to ClientWorkerInterface and use the existing code
in the client that can fetch a list of all `ClientWorkerInterface`s.

The split between WorkerInterface and ClientWorkerInterface appears to be
what a client might have a need to call versus what is fdbserver-internal (and
thus no client should even want to call). Thus, it seems to make more sense to
acknowledge that profiling is useful to be able to toggle from a client, and go
with option (2).
2017-10-05 14:08:28 -07:00
Alex Miller 11668bb359 Fixing code review comments. 2017-09-29 15:58:36 -07:00
Alex Miller b7ce9d996c Comment out verbose TraceEvents in preparation for pushing. 2017-09-29 15:58:36 -07:00
Alex Miller c40c1bb5fe Add a new workload: BackupToDBAbort, which does an ACI switchover.
This is to allower easier testing of non-durable switchovers without having to
wiggle into BackupToDBCorrectness's view of the world.
2017-09-29 15:58:36 -07:00
Alex Miller 9e9a96ae76 Make VersionStamp workload able to run with DR-style workloads.
* It is now tolerant of locked database errors, and handles them correctly.
* There is an option to specify which database to verify against.
2017-09-29 15:58:36 -07:00
Alex Miller 34630b6130 Make VersionStamp workload can handle commit_unknown_result.
Previously, if a transaction failed with commit_unknown_result, and was
actually committed, it would look like data that magically appeared in the
database and verification would fail.

Now, we explicitly re-read and check to see if the commit happened, so that we
may maintain an accurate understanding of what the database state should be.
2017-09-29 15:58:36 -07:00
Alex Miller 23945b9fea VersionStamp can co-exist with other workloads that write data to the database.
VersionStamp previously would range-read the entire database during validation.
This has the unfortunate effect of making it fail during validation if run with
any other workload that writes keys to the database.

Now, all keys written and read are done with a configurable prefix, so that it
may co-exist with a variety of other workloads.
2017-09-29 15:58:36 -07:00
Alex Miller 370a6afb80 Make VersionStamp have an option to be tolerant of data being lost. 2017-09-29 15:58:36 -07:00
Alex Miller 69523ce151 Hackish version of a test, but it does fail. 2017-09-29 15:58:36 -07:00
Alex Miller 65713b226f Fix whitespace and line endings. 2017-09-29 15:58:36 -07:00
Bhaskar Muppana 91975244fe Fixing OSX build. 2017-09-28 19:35:44 -07:00
Bhaskar Muppana 942c04e992 Merge pull request #162 from bmuppana/master
Fixing TimeKeeperCorrectness to deal with network delays.
2017-09-28 17:04:39 -07:00
Bhaskar Muppana 3d2bafc3a6 Fixing TimeKeeperCorrectness to deal with network delays. 2017-09-28 16:52:28 -07:00
Evan Tschannen ef41b07bb3 renamed past_version to transaction_too_old
implemented read_lock_aware option
2017-09-28 16:35:08 -07:00
Evan Tschannen 7b60e26660 Merge pull request #160 from cie/use-error-descriptions
Add the ability to access name and description in Error. Update error…
2017-09-28 16:00:39 -07:00
Evan Tschannen 73fca75239 added the ability to disable timeKeeper; disabled timeKeeper before consistency check in simulation 2017-09-28 13:13:24 -07:00
A.J. Beamon d30c730f75 Add the ability to access name and description in Error. Update error descriptions. 2017-09-28 12:35:03 -07:00
Bhaskar Muppana 0f8ff26029 Merge pull request #158 from bmuppana/master
<rdar://problem/34557380> Need a way to map real time to version
2017-09-27 17:56:42 -07:00
Bhaskar Muppana 6a0b1d6808 Fixing PR comments
<rdar://problem/34557380> Need a way to map real time to version
2017-09-27 17:56:01 -07:00
Evan Tschannen 2bf042a559 fix: file_corrupt was not checking for fault injection
latency threshold was too long
2017-09-25 17:22:41 -07:00
Bhaskar Muppana 0bf5bdb23a <rdar://problem/34557380> Need a way to map real time to version 2017-09-25 12:51:37 -07:00
Stephen Atherton 248dab79b6 Created “redwood” storage engine option and many changes to support that including IKeyValueStore::init() and custom DiskQueue file extensions. 2017-09-21 23:51:55 -07:00
Evan Tschannen e8b895c878 added the ability to disable connection failures for a period of time after one happens 2017-09-18 12:46:29 -07:00
Evan Tschannen 489332533c all timeouts longer than two minutes have been can be lowered to 60.0 with buggification
added a workload that tries for a 50 second maximum latency in the presence of one failure with both buggification and connection failures
2017-09-18 11:04:51 -07:00
Evan Tschannen 34f987f56d added a test in simulation which ensures that a recovery after a single failure takes less than 15 seconds 2017-09-15 17:55:01 -07:00
A.J. Beamon 4fa2415553 Merge branch 'release-5.0' 2017-09-08 17:28:12 -07:00
A.J. Beamon bb8a245bdb circus: throughput test scales latency error by the target latency 2017-09-08 17:27:54 -07:00
Bhaskar Muppana c7df951f7c Using BackupConfig from backup.actor.cpp to reduce intermediate
functions.
2017-09-07 08:36:36 -07:00
Bhaskar Muppana fe208d6adf Merge branch 'master' of github.com:apple/foundationdb into backup 2017-09-06 10:01:55 -07:00
Bhaskar Muppana 83810edabc Backup/Restore tag can be std::string instad of Key. 2017-09-05 11:38:40 -07:00
A.J. Beamon cc24072a5d Add the multi version API to the list of APIs to choose in the APICorrectness tester. Support for the multi-version client already existed. 2017-08-31 16:23:55 -07:00
Evan Tschannen d61be4c760 Merge branch 'release-5.0' 2017-08-30 12:59:24 -07:00
Evan Tschannen 963e1c3f31 fix: we need to reboot the process even if it will result in too many files, because the check will not succeed without it 2017-08-30 12:58:46 -07:00
Alvin Moore 6020d70863 Added trace event to track reboots initiated by ConsistencyCheck workload in simulation 2017-08-29 11:41:27 -07:00
Alvin Moore c95a1be5ec Add trace event for rebooting process during simulation for consistency check 2017-08-29 11:00:44 -07:00
A.J. Beamon 9a0a3b6329 Merge commit '66528becb82d826e81fa644bb378212584ab580e' 2017-08-28 16:47:59 -07:00
Alvin Moore 44e0df78c5 Added support for tracking roles for simulation workers
Fixed the exclusion and inclusion address simulation API and integration within workloads
Added more information within trace events for simulation
2017-08-28 11:25:37 -07:00
Alvin Moore 581bd6c8ed Added option to delay the displaying of the simulation workers 2017-08-28 10:53:56 -07:00
Alec Grieser 300b5a17ed Merge branch 'release-5.0' 2017-08-25 18:55:33 -07:00
Evan Tschannen 272b4b984c fix: fixed a rare bug where we do not wait for a file in the process of being deleted to shutdown before rebooting a machine 2017-08-25 10:12:58 -07:00
Evan Tschannen 26a5b5e422 rollback workload now clogs the communication between one of the proxies and the tlogs, since that is what will cause a rollback 2017-08-23 16:08:13 -07:00
Alvin Moore 8056b78414 Merge branch 'release-5.0' 2017-08-22 13:51:19 -07:00
Alvin Moore 814e471689 Added support for displaying initial workers via printf within simulation using a workload 2017-08-22 13:38:24 -07:00
Evan Tschannen 47a37f3f1e Merge pull request #135 from cie/switch-for-data-distribution
Add a switch to turn off data distribution in CLI
2017-08-07 12:54:08 -07:00
Alec Grieser ca7437ecf6 Merge branch 'release-5.0' 2017-08-02 22:07:01 -07:00
John King d0fbc41338 set LOCK_AWARE on several transactions used for getting cluster info for the consistency check 2017-07-28 18:50:32 -07:00
Yichi Chiang 6a8a5c41b0 Add a switch to turn off data distribution in CLI 2017-07-28 18:14:55 -07:00
Yichi Chiang 53e1ae9f60 shard system keyspace 2017-07-26 13:47:31 -07:00
Alvin Moore 96f40d8eb0 Added support for optionally re-including excluded servers
Removed unneeded code to protect coordinators
Ensured that simulation exclusion list is updated for excluded processes
2017-06-19 16:51:07 -07:00
Evan Tschannen 766dc23e26 fix: do not use TLS in protectedAddresses 2017-06-02 13:52:21 -07:00
Evan Tschannen 2d0dbd57e8 randomized the delays in atomic switchover workload 2017-06-01 12:08:21 -07:00
Evan Tschannen 1626e16377 Merge branch 'release-4.6' into release-5.0 2017-05-31 16:23:37 -07:00
Alvin Moore 333f2e4865 Added connection fault disabler within setup of backup submission. It should be reviewed to determine the amount of time to wait before disabling 2017-05-31 14:21:50 -07:00
Stephen Atherton 98604d33a0 Merge branch 'fix-io-timeout-handling'
# Conflicts:
#	fdbrpc/AsyncFileKAIO.actor.h
#	fdbrpc/sim2.actor.cpp
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/optimisttest.actor.cpp
#	fdbserver/worker.actor.cpp
#	fdbserver/workloads/MachineAttrition.actor.cpp
#	tests/fast/SidebandWithStatus.txt
#	tests/rare/LargeApiCorrectnessStatus.txt
#	tests/slow/DDBalanceAndRemoveStatus.txt
2017-05-26 18:43:08 -07:00
Alvin Moore ba606c2135 Removed NOT_IN_CLEAN macro from file 2017-05-26 14:52:06 -07:00
Alvin Moore b28ed397a2 Fixed printf field width specifier to reduce compilation warnings within OS X 2017-05-26 14:51:34 -07:00
Alvin Moore 0b9ed67e12 Fixed support for RemoveServers Workload
Added availability functions to simulation
2017-05-26 14:20:11 -07:00
Alvin Moore 16cc0821b1 Removed dead machine option from simulation 2017-05-25 16:29:02 -07:00
FDB Dev Team a674cb4ef4 Initial repository commit 2017-05-25 13:48:44 -07:00