Commit Graph

2751 Commits

Author SHA1 Message Date
Evan Tschannen 69711a107b fix: because of forced recovery, 0 log router tags does not mean we are a special tlog set 2018-10-02 17:45:11 -07:00
Evan Tschannen e7e1c634e0 fix: we need to restart the peek cursor when the known committed version becomes available 2018-10-02 17:44:14 -07:00
Evan Tschannen a92fc911ac do not spin on a failed storage server recruitment 2018-10-02 17:31:07 -07:00
Evan Tschannen dd9c6856e1 fix: forced recovery is not safe to send multiple times 2018-10-02 17:29:39 -07:00
Evan Tschannen 15ce215c1b fix: parallel peek requests leaked memory 2018-10-02 17:28:39 -07:00
Evan Tschannen bd6b743a81 fix: the storage server must always keep MAX_READ_TRANSACTION_LIFE_VERSIONS of history in memory, because forced recovery could roll back an epoch end.
fix: rollbacks were triggered unnecessarily
2018-09-28 16:04:59 -07:00
Evan Tschannen 3fdf72c626 fix: we need to force recovery if the master is still attempting to read the txs tag 2018-09-28 13:33:33 -07:00
Evan Tschannen 59335aa757 fix: the latest generation of remote transaction logs might has less data the a previous generation, because they take over at known committed version. Detect this case and end at the version that has the most data 2018-09-28 12:25:27 -07:00
Evan Tschannen c577840020 fix: forced recovery should remove all references to the old primary tlogs in all generations of logs to help the peek logic avoid attempting to read from them 2018-09-28 12:23:09 -07:00
Evan Tschannen 05e7f08b26 added a peek method which will attempt to read the txsTag from the local region as much as possible 2018-09-28 12:21:08 -07:00
Evan Tschannen a24eadd73a fix: for remote logs, their known committed version cannot be set to 1, because they can be used when their durable version is 0, leading to a known committed version being greater than a queue committed version 2018-09-28 12:17:21 -07:00
Evan Tschannen e64c55dce0 fix: data distribution would use the wrong priority sometimes when fixing an incomplete movement, this lead to the cluster thinking the data was replicated in all regions before it actually was 2018-09-28 12:15:23 -07:00
Evan Tschannen b1fe069165 fix: during forced recovery logs can be removed from the logSystemConfig. We need to avoid killing the removed logs as unneeded until we actually complete the recovery 2018-09-28 12:13:46 -07:00
Evan Tschannen 22e6afbb18 fix: the cluster controller did not pass in its own locality when creating its database object, therefore it was not using locality aware load balancing 2018-09-28 12:12:06 -07:00
Evan Tschannen b560b94ebc fix: do not force a recovery if the master was already in the other region (and therefore already recovered)
fix: reboot the remaining DC, because any storage server rejoins that were rolled back will cause that server to be unusable
2018-09-28 12:10:04 -07:00
Evan Tschannen 77e2fb787e Merge branch 'release-6.0' into feature-fix-forced-recovery 2018-09-21 14:55:37 -07:00
Evan Tschannen 3f86905ea7 fix: restore did not take into account that the end version of a log file does not exist in that file. This resulted in restores done at the same version a snapshot completes to not apply the mutations at that final version. 2018-09-21 11:48:28 -07:00
Evan Tschannen 6b6d7a087d The cluster controller should never consider itself as failed (that will be handled by the coordinators)
Simplified the check that the cluster controller is excluded
2018-09-20 17:01:11 -07:00
Evan Tschannen 31d0b0315f fix: tlog spill policy would spill everything when it wanted to spill nothing
use a flow lock to protect updatePersistData and initPersistentState from committing simultaneously
2018-09-20 15:33:38 -07:00
Evan Tschannen 03728db99b do not trigger better master exists if the cluster controller is excluded, since the master will change anyways once the cluster controller is moved 2018-09-19 18:28:24 -07:00
Evan Tschannen 42a67efb0c the cluster controller should prefer to be located on a transaction class machine over a storage server class machine 2018-09-19 18:04:59 -07:00
Evan Tschannen 861c8aa675 consider server health when building subsets of emergency teams 2018-09-19 17:57:01 -07:00
Evan Tschannen 702d018882 fix: we cannot use count on an async map, because someone waiting onChange for an item will cause it to exist in the map before it is set 2018-09-19 16:11:57 -07:00
Evan Tschannen 6d18193b3a fix: team->setHealthy was not being called correctly on initially unhealthy teams 2018-09-19 14:48:07 -07:00
Evan Tschannen 270b1b24a6 fix: we have to use durableKnownCommittedVersion, because the is the true lower bound on the recovery version of the remote logs
fixed a compiler error
2018-09-18 16:29:03 -07:00
Evan Tschannen 200e65fe61 added a workload which tests killing an entire region, and recovering from the failure with data loss.
fix: we cannot pop the txs tag from remote logs until they have a full copy of the txnStateStore
fix: we have to modify all of history, we cannot stop after finding a local remote
2018-09-17 18:32:39 -07:00
Evan Tschannen 16018ee36d
Merge pull request #770 from bnamasivayam/release-6.0
When the cluster is configured from fearless setup to usable_regions=…
2018-09-13 10:36:17 -07:00
Balachandar Namasivayam d622cb1f6e When the cluster is configured from fearless setup to usable_regions=1, master goes into a loop changing team priority . Fix this issue. 2018-09-12 18:29:49 -07:00
Bhaskar Muppana 0c82edc5d9
Merge pull request #766 from alecgrieser/post-release-cleanup-6.0.11
Post release cleanup
2018-09-11 17:15:33 -07:00
Alec Grieser 20331e32d3
update installer WIX GUID following release 2018-09-11 17:05:09 -07:00
Alec Grieser 27125d9d65
update versions target to 6.0.12 2018-09-11 17:05:09 -07:00
Evan Tschannen 700d558e5a
Merge pull request #764 from satherton/cleanup-status-refactor
Status refactor cleanup
2018-09-11 10:55:20 -07:00
Stephen Atherton b47902584b Added comparison of serialization speed of json_spirit and JsonBuilder to JsonBuilder perf test. 2018-09-10 19:54:27 -07:00
Stephen Atherton 5b8971ef18 Rename JsonString to JsonBuilder 2018-09-10 19:01:24 -07:00
Stephen Atherton 5c23f91c59 Simplified templates that were used to forward rvalue references to JsonBuilder write() functions when possible as the current design does not require or benefit from this. 2018-09-10 18:48:30 -07:00
Evan Tschannen 9e8c1941ec
Merge pull request #763 from alecgrieser/000761-getuserversion-order-of-operations
Fix issue with Versionstamp::getUserVersion on larger user versions
2018-09-10 18:21:54 -07:00
Alec Grieser 5128d9772d
cleverly use parentheses to ensure that the bitwise operation happens before the shift
This was casuing getUserVersion to return incorrect results if the user version was greater than 0x00FF.

Fixes #761
2018-09-10 18:18:13 -07:00
Evan Tschannen 79f1b8963c
Merge pull request #762 from etschannen/release-6.0
prepare for 6.0.11
2018-09-10 18:10:57 -07:00
Evan Tschannen 4c8a2df22b updated downloads for 6.0.11 2018-09-10 18:08:39 -07:00
Evan Tschannen 0dbb0244ff updated wxs UID for 6.0.11 2018-09-10 18:04:20 -07:00
Evan Tschannen 23a979a095
Merge pull request #758 from etschannen/release-6.0
fix: status would generate invalid json
2018-09-10 17:47:58 -07:00
Evan Tschannen 5fff960e5d updated release notes for 6.0.11 2018-09-10 17:45:01 -07:00
Evan Tschannen fb0d1d12a3 fix: reserve was not called correctly 2018-09-10 15:20:41 -07:00
Stephen Atherton 02d7b02036 Moved readJSONStrictly to a client cpp to avoid duplicate definitions at link time. 2018-09-10 14:35:04 -07:00
Stephen Atherton 8ed8f290c4 Merge branch 'release-6.0' of github.com:etschannen/foundationdb into evan-6 2018-09-10 13:28:49 -07:00
Stephen Atherton 4753172991 Added support for NaN and infinity in JsonBuilder in double and ascii number interfaces. 2018-09-10 13:28:31 -07:00
Evan Tschannen 63271072e8 fixed linker problem 2018-09-10 12:59:22 -07:00
Stephen Atherton 9febc01106 In simulation, strict JSON parser will print bad input to stdout (not std since whether it's an error depends on context). 2018-09-10 12:07:03 -07:00
Stephen Atherton 6a04fa0ede In simulation, deserializing a StatusReply now does a strict JSON parse. 2018-09-10 11:13:41 -07:00
Evan Tschannen ce38ddbd4b added tests with the + sign 2018-09-10 10:52:15 -07:00