Commit Graph

38 Commits

Author SHA1 Message Date
Jon Fu 7d8f0dfc9e Merge branch 'master' of https://github.com/apple/foundationdb into jfu-fix-snapshot-backup-agent 2020-10-15 12:44:41 -04:00
sfc-gh-tclinkenbeard c7694e7b50 Merge remote-tracking branch 'origin/master' into refactor-workloads 2020-10-14 09:21:47 -07:00
Jon Fu e096297bc1 add traces 2020-10-13 16:07:09 -04:00
Jon Fu a646df520d add option to remove snaptest retry limit 2020-10-05 13:15:43 -04:00
sfc-gh-tclinkenbeard 7f0d14c8e4 Modernize/refactor workloads directory 2020-10-04 22:29:07 -07:00
Jon Fu 57a80cfef2 added new SimpleAtomicAdd workload and adjusted SnapIncrementalRestore restarting test 2020-09-28 16:19:48 -04:00
Jon Fu 22996284c7 added changes to allow writing of last epoch end version to special keys when performing recovery due to snapshot 2020-09-11 15:00:11 -04:00
Evan Tschannen 07cc0a8d74 code cleanup 2020-04-10 17:02:11 -07:00
Andrew Noyes d4de608bb6 Fix OPEN_FOR_IDE build 2019-10-25 10:42:22 -07:00
sramamoorthy 5d87443323 improved error msgs for snapshot cmd 2019-08-27 16:43:52 -07:00
Evan Tschannen ba54508c47 code cleanup 2019-08-06 16:30:30 -07:00
sramamoorthy 9afd162e2f remove snap v1 related code 2019-07-25 17:29:31 -07:00
sramamoorthy a2f2ad96ff code review comments and merge to master changes 2019-07-24 15:36:28 -07:00
sramamoorthy 869f77aef1 Few cosmetic edits and fixes 2019-07-24 15:36:28 -07:00
sramamoorthy 671c98fa3d snap v2: test files changes 2019-07-24 15:36:28 -07:00
sramamoorthy 212136d024 SnapTest to handle retries for exec txns 2019-06-24 10:22:42 -07:00
sramamoorthy 4bcb590f12 g_random -> deterministicRandom() 2019-05-28 22:07:46 -07:00
sramamoorthy b17ad85497 exec op not supported when log_anti_quorum > 0 2019-05-28 22:07:46 -07:00
sramamoorthy 3aa848b8af minor bug in whitelist binary path testing 2019-05-28 22:07:46 -07:00
sramamoorthy 40358e1dd6 limit of getRange in snapTest reduced
With CLIENT_KNOBS->TOO_MANY in snapTest, by the time getRange
gathers all the results, the storage server's oldest version has
gone past the req->version and hence the transaction fails with
transaction_too_old
2019-05-28 22:07:46 -07:00
sramamoorthy ceac68c990 restore - remove emtpy snapdir,snap loop retry fix
- remove partially snapped directories to avoid no cluster file assert
- snap create to retry max 3 times for not_fully_recovered and keep
  retrying for the other failures
2019-05-28 22:07:46 -07:00
sramamoorthy d3a179b6f9 Multiple bug fixes
- wait for snapTLogFailKeys in a loop, otherwise in some race
  condition it can cause a false assert
- in single region, there does not seem to be a guarantee of
  tagLocalityListKey for a given DC ID, avoiding that assert for now
- to find the workers that are coordinators, looking up by primary
  address is not sufficient in some cases, hence looking by both
  primary and secondary address
- test make files to reflect the location of the new test cases
2019-05-28 22:07:46 -07:00
sramamoorthy bb474dc323 if recovery < fully_recovered then fail the exec
Will do more cleanup, pushing it for a test run in CI
2019-05-28 22:07:46 -07:00
sramamoorthy 925499954b New status cluster_not_fully_recovered 2019-05-28 22:07:46 -07:00
sramamoorthy 591ff96b93 increase retry and use eat instead of parsing 2019-05-28 22:07:46 -07:00
sramamoorthy 4083af0b01 Avoid using trackLatest for TLog pop test cases 2019-05-28 22:07:46 -07:00
sramamoorthy ec7834e2f7 code re-orgnaization and address comments 2019-05-28 22:07:46 -07:00
sramamoorthy 858604b51d minor cleanups to SnapTest 2019-05-28 22:07:46 -07:00
sramamoorthy 00ccee8a6c workaround for log giving remote log and others
logSystemConfig.allLocalLogs() sometimes returns remote TLog interface
and a workaround is implemented here. Other minor cleanup.
2019-05-28 22:07:46 -07:00
sramamoorthy 17ecba8313 trace cleanup and other indentation changes 2019-05-28 22:07:46 -07:00
sramamoorthy 898bed66c1 Allow only whitelisted binary path for exec op 2019-05-28 22:07:46 -07:00
sramamoorthy c4d27ac9d2 bug fixes in SnapTest
Earlier the test was checking for the following condition:
durable version of storage > min version of tlog, but the
check has been modified to:
durable version of storage >= min version of tlog - 1.

Ensure that the pre-snap validate keys are exactly 1000 in
the case of commit retires.
2019-05-28 22:07:46 -07:00
sramamoorthy 6431513ad0 Fail exec req until the cluster is fully_recovered 2019-05-28 22:07:46 -07:00
sramamoorthy 4016f16c76 Fix few compilation and bugs in rebase 2019-05-28 22:07:46 -07:00
sramamoorthy 3d5998e9dd tlog: when pops are disabled, store them & replay
In Tlogs, disable pop is done whlie taking snapshots. Earlier, tlogs
were ignoring the pops if it got pop requests when pops were
disabled. In this change, instead of ignoring the pop - it remembers
the list of pops in-memory and plays them once the popping is
enabled.
2019-05-28 22:07:46 -07:00
sramamoorthy 4bc4c615da exec op to all tlog, restore change in test &other
- exec operation to go to all the TLogs
- minor bug fix in tlog
- restore implementation for the simulator
- restore snap UID to be stored in restartInfo.ini
- test cases added
- indentation and trace file fixes
2019-05-28 22:07:46 -07:00
sramamoorthy 72dd067173 Trace message changes and fix few FIXMEs 2019-05-28 22:07:46 -07:00
sramamoorthy 69edefe68b Snapshot based backup and resotre implementation 2019-05-28 22:07:46 -07:00