Commit Graph

10523 Commits

Author SHA1 Message Date
Alex Miller 9b25c72971 Pull database config and cluster config into one struct.
This will allow us to specify custom situations to be chosen more frequently,
and in particular control machines and processes.
2017-08-21 22:35:44 -07:00
Stephen Atherton aff31b7f36 Checkpointing. Clears almost work, but two edge cases aren’t handled correctly yet involving range clears that must cross leaf page boundaries. 2017-08-21 22:29:57 -07:00
Alec Grieser 5ee07b1a9e Merge branch 'release-5.0' 2017-08-14 16:56:58 -07:00
Evan Tschannen de1b590a8a The TLog did not delete data from removed logs
The TLog continued to make data from removed logs persistent
2017-08-11 18:08:09 -07:00
Stephen Atherton 50fb44be92 Merge branch 'release-5.0'
# Conflicts:
#	versions.target
2017-08-09 23:36:12 -07:00
Evan Tschannen 2335fc73f2 fix: peek cursors were being timed out every 10 minutes, instead of 10 minutes after the last use
fix: if an interface is changed while we are not waiting in getMore, we will not reset the sequence to 0.
2017-08-09 15:58:06 -07:00
Evan Tschannen 47a37f3f1e Merge pull request #135 from cie/switch-for-data-distribution
Add a switch to turn off data distribution in CLI
2017-08-07 12:54:08 -07:00
Stephen Atherton de03491475 Switched to new commit buffer type which can support more than set, but so far only set is implemented. 2017-08-04 00:01:25 -07:00
Evan Tschannen c22708b6d6 added tag localities
fix: remote logs need to stop the master when they are stopped
2017-08-03 16:16:36 -07:00
Stephen Atherton dbcd3526d7 Memory lifetime bug fix and previously missed changes for enforcing only reading committed versions. 2017-08-03 15:07:29 -07:00
Alec Grieser ca7437ecf6 Merge branch 'release-5.0' 2017-08-02 22:07:01 -07:00
John King d0fbc41338 set LOCK_AWARE on several transactions used for getting cluster info for the consistency check 2017-07-28 18:50:32 -07:00
Yichi Chiang 6a8a5c41b0 Add a switch to turn off data distribution in CLI 2017-07-28 18:14:55 -07:00
A.J. Beamon 4243486f54 fix: TLogMetrics was being track latested with the wrong ID 2017-07-28 14:37:23 -07:00
Yichi Chiang 37e5e2acbb Fix parentheses issue in StorageMetrics.actor.h 2017-07-27 12:03:36 -07:00
Yichi Chiang cdc62e265c Merge pull request #133 from cie/shard-system-keyspace
Shard system keyspace
2017-07-26 17:09:13 -07:00
A.J. Beamon 41c90bcdea Merge commit '89ac94853c70d08289e7fb58055bc5d0cd4e494d' 2017-07-26 15:35:36 -07:00
A.J. Beamon d8e308c18f Enable use of incremental delete when deleting disk queue and sqlite KVS sqlite files. 2017-07-26 14:11:11 -07:00
Yichi Chiang 53e1ae9f60 shard system keyspace 2017-07-26 13:47:31 -07:00
Stephen Atherton 31314d06d4 Added test stats. Added enforcement of only reading committed versions, which is a currently an implementation limitation. 2017-07-25 16:10:19 -07:00
Stephen Atherton 4aaee86c2a Moved MetricLogger actor to fdbclient so applications other than fdbserver can use it. 2017-07-24 13:13:06 -07:00
Evan Tschannen 2ae445782e fix: cannot rely on the bestServer’s version because other logs may have higher versions 2017-07-21 19:21:49 -07:00
Evan Tschannen f6826f1e15 fix: log routers were popped at too high of a version
fix: make sure tlogs make everything durable
fix: make cluster controller’s temporary remote log recruitment not interfere with better master exists
2017-07-20 16:26:05 -07:00
Evan Tschannen 7fec378830 do not continue copying data from prior generations after being locked 2017-07-19 15:11:18 -07:00
Evan Tschannen 5852a6301b fixed even more bugs 2017-07-15 15:15:03 -07:00
Alec Grieser c860f09d8a Merge branch 'release-5.0' 2017-07-14 16:01:15 -07:00
Alec Grieser 660729839c moved Notified.h from flow -> fdbclient ; flow bindings package does better job when excluding testers 2017-07-14 15:49:30 -07:00
Stephen Atherton d61f74e52b Debug output changes. 2017-07-14 13:39:58 -07:00
Stephen Atherton 8b3569e27e Bug fix. 2017-07-14 11:37:08 -07:00
Stephen Atherton fc0557252b Bug fixes. Btree now uses page 0 as root and will write initial page only if necessary. Added debug printf macro. 2017-07-14 11:36:49 -07:00
Stephen Atherton d8b82ecbe4 Added use of IndirectShadowPager. Moved MemoryPager code into .cpp because of type conflicts and its implementation doesn’t need to be externally visible anyway. Added start of a performance test. Renamed tests. Correctness/set now has random reopen of disk based pager before verification. Added asserts for when keys or values that are too large to fit in a single page. 2017-07-13 22:11:48 -07:00
Evan Tschannen 5ac4de8775 fix: the same tag could be in the server tags list twice 2017-07-13 16:31:55 -07:00
Stephen Atherton b56f5643d5 Bug fixes. 2017-07-13 14:51:39 -07:00
Evan Tschannen 57ba9d36af fixed a large number of bugs 2017-07-13 12:29:21 -07:00
Stephen Atherton bea061fd75 Bug fix, building tree root (1 or more top levels) must happen inside commitSubtree() otherwise the root page logical ID (1) can be written out of version order. 2017-07-13 11:32:14 -07:00
Alec Grieser f75b6f333b Merge branch 'release-5.0' 2017-07-13 11:21:18 -07:00
Evan Tschannen 415458deef made LogSet reference counted,
fixed a few bugs
2017-07-11 15:48:10 -07:00
Evan Tschannen 81ae263ad9 implemented setPeekCursor
removed oldTLogServer
first compiling version
2017-07-10 17:41:32 -07:00
Evan Tschannen 979ebcef6c changed to using a vector of logSets instead of a duplicate set of logs for remote servers
finished porting changes to the tlog
everything but peeking is finished in the TagPartitionedLogSystem
2017-07-09 14:46:16 -07:00
Stephen Atherton c508e8bdf9 Adjacent internal btree pages can now contain the same keys at different versions. 2017-07-04 23:49:18 -07:00
Stephen Atherton 1a71df1871 Lots of bug fixes and debug output added. Unitttest for set works…pretty often. 2017-07-04 23:41:48 -07:00
Evan Tschannen aa1c903b52 fix: do not log that data distribution is initialized until readyToStart is ready 2017-06-30 16:21:59 -07:00
Evan Tschannen 0906250e78 merged everything from feature-remote-logs besides the tlog and tagpartitionedlogsystem
re-included tags in messages to the tlog
previously never committed the LogRouter
2017-06-29 15:50:19 -07:00
A.J. Beamon a2d229ff00 Merge branch 'release-5.0' 2017-06-29 08:16:22 -07:00
Evan Tschannen 69c862ed6e updated release notes for 5.0.1 2017-06-28 16:52:45 -07:00
Evan Tschannen 663535e00a add the same number for shared metrics and local metrics 2017-06-28 15:21:54 -07:00
Evan Tschannen fe37d0b056 added additional logging about bytesInput and bytesDurable 2017-06-28 13:29:40 -07:00
Alvin Moore 31d562ff7b Merge branch 'release-5.0'
# Conflicts:
#	fdbclient/ManagementAPI.actor.cpp
#	fdbserver/DatabaseConfiguration.cpp
#	versions.target
2017-06-27 11:16:08 -07:00
Evan Tschannen 9fd5955e92 Merge branch 'master' into removing-old-dc-code 2017-06-26 16:27:10 -07:00
Evan Tschannen 15cb498aa7 removed fast_recovery_double and fast_recovery_triple from the fdbcli 2017-06-23 16:18:23 -07:00
Evan Tschannen 533dca95d8 fix: bytesDurable was not correctly increased when a log was removed
re-added many TLogMetrics
added a new role for the shared tlog
2017-06-22 17:21:42 -07:00
Alvin Moore 6d19580789 Merge branch 'release-5.0' of github.com:apple/foundationdb into release-5.0
# Conflicts:
#	fdbrpc/simulator.h
2017-06-19 17:39:37 -07:00
Alvin Moore 96f40d8eb0 Added support for optionally re-including excluded servers
Removed unneeded code to protect coordinators
Ensured that simulation exclusion list is updated for excluded processes
2017-06-19 16:51:07 -07:00
Alvin Moore 9553458b78 Updated simulation to support managing exclusion and inclusion address
Added method for identifying acceptable availability process classes
Extended cluster availability function to ensure coordinators can be auto configured
Fixed availability function to allow protected processes to be considered as dead if not available
Added debug trace events for providing machine state when considering availability
Added trace event for protected coordinators
2017-06-19 16:48:15 -07:00
Stephen Atherton f405c8d88e Merge branch 'release-4.6' into release-5.0
# Conflicts:
#	fdbrpc/AsyncFileKAIO.actor.h
#	fdbrpc/sim2.actor.cpp
#	fdbserver/optimisttest.actor.cpp
#	versions.target
2017-06-15 17:40:19 -07:00
Stephen Atherton 5de6c703cf Fixed line endings which were changed to UNIX style during a recent merge. 2017-06-15 16:57:35 -07:00
Evan Tschannen 4bdcd8fc12 Merge branch 'release-4.6' into release-5.0
# Conflicts:
#	bindings/bindingtester/run_binding_tester.sh
#	fdbrpc/AsyncFileKAIO.actor.h
2017-06-14 16:43:53 -07:00
Stephen Atherton b65ad3563c Merge branch 'master' into feature-redwood
# Conflicts:
#	fdbserver/fdbserver.vcxproj
#	fdbserver/fdbserver.vcxproj.filters
2017-06-09 14:56:41 -07:00
Evan Tschannen cefaa2391d fix: tlog actor needs to be cancelled when worker shutdown 2017-06-02 17:56:47 -07:00
Evan Tschannen 766dc23e26 fix: do not use TLS in protectedAddresses 2017-06-02 13:52:21 -07:00
Evan Tschannen b108c1ea0d fix: do not delay before setting logData->version 2017-06-02 11:27:37 -07:00
Evan Tschannen 2d0dbd57e8 randomized the delays in atomic switchover workload 2017-06-01 12:08:21 -07:00
Evan Tschannen bfcbb5623f fixed build from merge error 2017-06-01 12:07:30 -07:00
Evan Tschannen 276073d91b Merge branch 'release-4.6' into release-5.0 2017-06-01 11:54:54 -07:00
Stephen Atherton fa4fdb1f1d Merge branch 'fix-io-timeout-handling' into release-5.0
# Conflicts:
#	fdbserver/optimisttest.actor.cpp
2017-05-31 17:03:15 -07:00
Evan Tschannen 1626e16377 Merge branch 'release-4.6' into release-5.0 2017-05-31 16:23:37 -07:00
Alvin Moore 333f2e4865 Added connection fault disabler within setup of backup submission. It should be reviewed to determine the amount of time to wait before disabling 2017-05-31 14:21:50 -07:00
Stephen Atherton 98604d33a0 Merge branch 'fix-io-timeout-handling'
# Conflicts:
#	fdbrpc/AsyncFileKAIO.actor.h
#	fdbrpc/sim2.actor.cpp
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/optimisttest.actor.cpp
#	fdbserver/worker.actor.cpp
#	fdbserver/workloads/MachineAttrition.actor.cpp
#	tests/fast/SidebandWithStatus.txt
#	tests/rare/LargeApiCorrectnessStatus.txt
#	tests/slow/DDBalanceAndRemoveStatus.txt
2017-05-26 18:43:08 -07:00
Alvin Moore ba606c2135 Removed NOT_IN_CLEAN macro from file 2017-05-26 14:52:06 -07:00
Alvin Moore b28ed397a2 Fixed printf field width specifier to reduce compilation warnings within OS X 2017-05-26 14:51:34 -07:00
Alvin Moore 0b9ed67e12 Fixed support for RemoveServers Workload
Added availability functions to simulation
2017-05-26 14:20:11 -07:00
Alvin Moore 16cc0821b1 Removed dead machine option from simulation 2017-05-25 16:29:02 -07:00
FDB Dev Team a674cb4ef4 Initial repository commit 2017-05-25 13:48:44 -07:00