Commit Graph

80 Commits

Author SHA1 Message Date
Evan Tschannen 9f11f2ec53 Merge branch 'master' of github.com:apple/foundationdb 2019-07-30 16:55:56 -07:00
Evan Tschannen aaeeb605b2 Changes to degraded can cause master recoveries, which are not supposed to happen when speedUpSimulation is true 2019-07-30 16:33:40 -07:00
Evan Tschannen 6977e7d2e8 do not return recovered version as popped for txsTags because it could cause recovery to start over
optimized how buffered peek cursor discards popped data
2019-07-30 12:21:48 -07:00
Evan Tschannen 13203da199 fix: do not set the popped version of txsTag because it could be copied over at the recoveredAt version 2019-07-27 22:36:06 -07:00
Evan Tschannen 28df2c35bb
Merge pull request #1855 from alexmiller-apple/sharded-txs-safe-upgrade
Make sharded txsTag upgradeable and downgradeable
2019-07-26 13:29:39 -07:00
sramamoorthy 9afd162e2f remove snap v1 related code 2019-07-25 17:29:31 -07:00
sramamoorthy a65c9f92ed get rid of all timeouts and other changes 2019-07-24 15:36:28 -07:00
sramamoorthy a2f2ad96ff code review comments and merge to master changes 2019-07-24 15:36:28 -07:00
sramamoorthy 869f77aef1 Few cosmetic edits and fixes 2019-07-24 15:36:28 -07:00
sramamoorthy 31c010b393 few minor fixes 2019-07-24 15:36:28 -07:00
sramamoorthy c73bdfad9f do not pop txsTag 2019-07-24 15:36:28 -07:00
sramamoorthy a335ed2011 includeCancelled for tLogSnapCreate 2019-07-24 15:36:28 -07:00
sramamoorthy 61cd690add enable/disable pop req with UID mis-match to fail 2019-07-24 15:36:28 -07:00
sramamoorthy f4e257e464 snap v2: TLog related changes 2019-07-24 15:36:28 -07:00
Evan Tschannen 6d694cc2ce
Merge pull request #1818 from alexmiller-apple/peek-cursor-timeout-bug
Fix parallel peek stalling for 10min when a TLog generation is destroyed
2019-07-19 16:39:31 -07:00
Alex Miller 9863ace96c Replace usages with intialization lists.
But C++ needs a bit of help to inference though the templates.
2019-07-18 22:27:36 -07:00
Alex Miller e9684a1f63 Fix issues configuring from sharded txs tag to not
Which is an intermingling of what should be two commits:

1. Rely on TLogVersion instead of txsTags==0

2. Copy and index sharded txsTags between KCV and RV as txsTag when
configuring log_version 4->3.
2019-07-17 01:25:09 -07:00
Alex Miller 812ce37bcd Remove buggify and unneeded safeguards.
The buggify was actually incorrect and broke an invariant, which I then
fixed on the other side, but this work was actually unneeded in total.

The real issue being fixed was returnIfBlock not sending an error, as
well as the other error cases.
2019-07-16 15:58:02 -07:00
Alex Miller 4cc60dc9b8 Merge remote-tracking branch 'upstream/master' into peek-cursor-timeout-bug 2019-07-15 17:05:39 -07:00
Alex Miller 32af11286b Copy same changes to OldTLogServer_6_0. 2019-07-15 16:43:43 -07:00
Alex Miller c8e94e601a
Merge pull request #1729 from etschannen/feature-fast-txs-recovery
Improve the recovery speed of the txnStateStore
2019-07-15 13:27:41 -07:00
Vishesh Yadav 2606794df6
Merge pull request #1812 from alexmiller-apple/improve-only-spilled
Improve the behavior of parallelPeekMore+onlySpilled.
2019-07-10 17:15:19 -07:00
Alex Miller 2c7007d662 Copy the same set of changes to OldTLogServer_6_0 2019-07-09 18:20:20 -07:00
Alex Miller 6c8f50ca66 Improve the behavior of parallelPeekMore+onlySpilled.
When onlySpilled transitions from true (don't peek memory) to false (do
peek memory) as part of a parallel peek, we'll end up wasting the rest
of the replies because we'll honor their onlySpilled=true setting and
thus not have any additional data to return.

Instead, we thread the onlySpilled back through in the same way that the
ending version of the last peek is used overrides the requested starting
version of the next peek.  This simulated the same behavior that the
client has, where the value of onlySpilled that we reply with comes back
in the next request.

I haven't actually seen it be a problem, but this should help make sure
the onlySpilled transition when catching up doesn't ever cause any ill
effects if a process starts riding the line between onlySpilled settings.
2019-07-08 22:13:09 -07:00
Evan Tschannen 15e894c724 Merge in master 2019-07-05 15:49:24 -07:00
Evan Tschannen 235697f688 fix: txsTags are not popped at the recovery version 2019-06-27 23:18:26 -07:00
Alex Miller bf883d7055 Merge remote-tracking branch 'upstream/master' into flowlock-api 2019-06-25 14:26:50 -07:00
Alex Miller 7a500cd37f A giant translation of TaskFooPriority -> TaskPriority::Foo
This is so that APIs that take priorities don't take ints, which are
common and easy to accidentally pass the wrong thing.
2019-06-25 02:47:35 -07:00
Evan Tschannen e0be631414 shard the txs tag so that more transaction logs are involved in its recovery 2019-06-19 18:15:09 -07:00
Alex Miller 51fd42a4d2 Merge remote-tracking branch 'upstream/master' into spilled-only-peek 2019-06-18 17:33:52 -07:00
sramamoorthy 1190f2f33d rebased related changes 2019-05-28 22:07:46 -07:00
sramamoorthy b43c100e57 TLog bug fixes 2019-05-28 22:07:46 -07:00
sramamoorthy 3877f87481 comment change in tLogCommit 2019-05-28 22:07:46 -07:00
sramamoorthy 31b6c86650 ignorePopDeadline to have high limit in simulator
- ignorePopDeadline to have highier limit in simulator
to accommdate for the buggify delays and make snapshot succeed.

- introduce a new knob for auto resetting the disabling of tlog pop
2019-05-28 22:07:46 -07:00
sramamoorthy b1b96946af logData->stop check right after execOpHold wait 2019-05-28 22:07:46 -07:00
sramamoorthy 5749e220bd use FlowLock for implementing critical section
Instead of using Promises and future to implement
critcal section use FlowLock
2019-05-28 22:07:46 -07:00
sramamoorthy e6c0b87a4d remove unused variable 2019-05-28 22:07:46 -07:00
sramamoorthy f27a40f118 execProcessingHelper made synchronous
tLogCommit exects no blocking between duplicate check and
setting of the new version, that constraint was broken
when synchronous execProcessingHelper was introduced.
As a fix, execProcessingHelper was made asynchronous.
2019-05-28 22:07:46 -07:00
sramamoorthy d3a179b6f9 Multiple bug fixes
- wait for snapTLogFailKeys in a loop, otherwise in some race
  condition it can cause a false assert
- in single region, there does not seem to be a guarantee of
  tagLocalityListKey for a given DC ID, avoiding that assert for now
- to find the workers that are coordinators, looking up by primary
  address is not sufficient in some cases, hence looking by both
  primary and secondary address
- test make files to reflect the location of the new test cases
2019-05-28 22:07:46 -07:00
sramamoorthy dcd2d96751 make spawnProcess predictable in the simulator 2019-05-28 22:07:46 -07:00
sramamoorthy 4083af0b01 Avoid using trackLatest for TLog pop test cases 2019-05-28 22:07:46 -07:00
sramamoorthy ec7834e2f7 code re-orgnaization and address comments 2019-05-28 22:07:46 -07:00
sramamoorthy b6e037ffbc Replace fork with boost::process::child 2019-05-28 22:07:46 -07:00
sramamoorthy e91c76834e tlog: move snap create part to indepdendent funcs 2019-05-28 22:07:46 -07:00
sramamoorthy 61e93a9304 Address review comments and minor fixes 2019-05-28 22:07:46 -07:00
sramamoorthy 9e3104c2d4 Fix: races in async exec leading to bad backup 2019-05-28 22:07:46 -07:00
sramamoorthy cfdad0c5e6 tlog to snapshot exactly at exec version 2019-05-28 22:07:46 -07:00
sramamoorthy 539e65efad Skip parsing mutations if it is tagged for TxsTag
In Tlog, if a mutation is targetted for TxsTag then skip from
parsing them.
2019-05-28 22:07:46 -07:00
sramamoorthy 17ecba8313 trace cleanup and other indentation changes 2019-05-28 22:07:46 -07:00
sramamoorthy aa79480d69 changes to make fdbfork asynchronous 2019-05-28 22:07:46 -07:00