Commit Graph

10958 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard a332dff2ec Fix class-memaccess warnings 2020-06-10 12:34:14 -07:00
Daniel Smith abf71ca822 Merge remote-tracking branch 'mpilman/features/rocksdb' into rocksdb-6.3 2020-06-10 19:30:03 +00:00
A.J. Beamon 25f59e8433 Add some more release notes for 6.3.1 2020-06-10 11:32:15 -07:00
A.J. Beamon c5e2accdaa Add a missing release note to 6.2.21 2020-06-10 11:22:02 -07:00
Xin Dong 80afcb6f30
Merge pull request #3327 from ajbeamon/cluster-controller-delay-candidacy-worst-fit
Delay cluster controller candidacy for all worst fit processes
2020-06-10 10:45:22 -07:00
Evan Tschannen f72d0a6a35
Merge pull request #3302 from sfc-gh-anoyes/anoyes/fix-watch-counting
Don't double count watches for too_many_watches
2020-06-10 10:40:11 -07:00
Evan Tschannen d3214a7740
Merge pull request #3293 from ajbeamon/fix-watch-future-version
Watches could return future_version errors unnecessarily
2020-06-10 10:39:26 -07:00
A.J. Beamon 83f53e1f86 Add PR number, match text of prior release documents 2020-06-10 10:04:21 -07:00
A.J. Beamon 739767b838 Delay cluster controller candidacy for all worst fit processes, not just storage servers. 2020-06-10 09:59:56 -07:00
Jingyu Zhou df064ac922
Merge pull request #3321 from xumengpanda/mengxu/fr-restore-ranges-PR
Fast Restore: Support restoring sub ranges in the framework
2020-06-09 21:01:39 -07:00
Evan Tschannen 2efe6fbf30
Merge pull request #3322 from satherton/feature-redwood
Added flowlock around reads done on the write path so that those read…
2020-06-09 17:03:00 -07:00
Steve Atherton c3c7db6a9d Added flowlock around reads done on the write path so that those reads cannot starve reads to support reads done by the storage server. 2020-06-09 17:00:21 -07:00
A.J. Beamon c3c1fd5a47 Add a release note. 2020-06-09 16:13:36 -07:00
Meng Xu 8533daac50 parallelRestoreFinish:fix used_during_commit error 2020-06-09 15:40:24 -07:00
A.J. Beamon 9bc7eaf55a Add missing status field to documentation 2020-06-09 14:57:14 -07:00
A.J. Beamon e10704fd76 Cherry-pick region related status changes from 6.3 2020-06-09 14:56:21 -07:00
A.J. Beamon 8a3f88a28b
Merge pull request #3317 from jleach4/3316
Fixes 3316 ByteArrayUtil.join is used heavily and creates a lot of objects
2020-06-09 14:33:16 -07:00
john_leach c122eff58a Fixes 3316 ByteArrayUtil.join is used heavily and creates a lot of objects 2020-06-09 14:06:12 -07:00
A.J. Beamon 98b96b2548
Merge pull request #3306 from AlvinMooreSr/joshua-support
Tests Packages Build Support
2020-06-09 13:17:45 -07:00
Meng Xu e5e4cb5c90 parallelRestoreFinish:separate check restore finished and clear restoreRequestDoneKey 2020-06-09 11:32:20 -07:00
Jingyu Zhou 78da69919f
Merge pull request #3310 from xumengpanda/mengxu/txn-key-not-exist-PR
FastRestore: Simplify getAndComputeStagingKeys on RestoreApplier
2020-06-09 10:52:49 -07:00
Meng Xu d85dc5a4d3 FastRestore:Only clear ranges that will be restored
Instead of clearning the entire normal key space.

This commit also removes some unnecessary tr->reset() which can invalid the txn backoff time.
2020-06-08 22:41:49 -07:00
Meng Xu 28212d397d RestoreApplier:Remove getValue actor 2020-06-08 20:32:52 -07:00
Meng Xu 1edcee4e9d RestoreApplier:Rewrite getKeys because key_not_exists error is handled by txn internally 2020-06-08 20:27:25 -07:00
Meng Xu e7fdd614fe Remove assert on error_code_key_not_found in previous commit
Correctnss shows the assert does not hold in all scenarios
2020-06-08 19:02:56 -07:00
Meng Xu 5022566b35 Validate if key_not_found error ever happens 2020-06-08 16:59:00 -07:00
Jingyu Zhou 42fc986750
Merge pull request #3283 from xumengpanda/mengxu/63-bug-for-jingyu-PR
FastRestore: Improve the getAndComputeStagingKeys() on applier
2020-06-08 15:06:02 -07:00
Daniel Smith 8d2007cd82 Delete debugging message 2020-06-08 20:20:10 +00:00
Andrew Noyes 55448a8d5a Remove call in watch actor instead of Transaction::watch 2020-06-08 18:19:18 +00:00
Alvin Moore 64a4cf287e Added support for building test packages via docker compose 2020-06-08 11:17:27 -07:00
Meng Xu f00deefd5a RestoreApplier:Remove unnecessary txn reset 2020-06-08 10:10:32 -07:00
Daniel Smith 8190329ab8 Disable the install step 2020-06-08 17:00:18 +00:00
Daniel Smith c54ad12072 Incremental improvements 2020-06-08 16:46:42 +00:00
Daniel Smith c836a9f5a9 Add RocksDB to build Dockerfile and CMake to build RocksDB 2020-06-08 16:22:28 +00:00
Meng Xu 8c81fedf11 RestoreApplier:Better handling of key not exist 2020-06-07 21:49:35 -07:00
Andrew Noyes 731b77a8c2 Don't double count watches for too_many_watches 2020-06-07 23:54:22 +00:00
Evan Tschannen a2afda9002
Merge pull request #3301 from satherton/feature-redwood
Update Redwood
2020-06-07 16:42:44 -07:00
Meng Xu 94be3afcf8 RestoreApplier:Costmic change based on review 2020-06-06 21:17:57 -07:00
Steve Atherton 8fdb81b48d Tweaked BTree test random parameter limits to avoid test runs which take too long. 2020-06-06 21:10:52 -07:00
Steve Atherton a28b5f0a8b Possible bug fix, flow locks should be taken after initializing BTree cursor in KVStoreRedwood read functions otherwise it might be possible for the BTree to be closed before the flow lock wait returns, depending on destruction order of some things. 2020-06-06 21:10:13 -07:00
Meng Xu 8044368d90 Merge branch 'release-6.3' into mengxu/63-bug-for-jingyu-PR 2020-06-06 11:22:16 -07:00
Steve Atherton 42e336baff Merge branch 'release-6.3' of github.com:apple/foundationdb into feature-redwood 2020-06-05 19:30:23 -07:00
Meng Xu f51fca0bf3 FastRestore:Sanity check actors do not throw error silently 2020-06-05 17:44:24 -07:00
Meng Xu 4a78895130
Merge pull request #3299 from ajbeamon/fuzz-test-ignore-throttling
Ignore throttling errors in fuzz tester
2020-06-05 16:47:46 -07:00
Meng Xu 96c2a164bc RestoreLoader:Wait on actorCollection error
so that we will not fail sildently
2020-06-05 16:45:08 -07:00
Meng Xu ffe949b04d Applier:getAndComputeStagingKeys:reset txn at first error
When tr->onError() is ready, the txn state has been reset.
We cannot wait on the get() future from the txn because its state has been deleted.
If we do that, it will throw txn_cancelled error, which will be throw all the way
up to the RestoreApplier main loop.

The batchData->dbApplier, which is assigned by writeMutationsToDB(self->id(), req.batchIndex, batchData, cx),
will become ready but isError(). This will make all handleApplyToDBRequest throw error silently.
2020-06-05 16:40:19 -07:00
Daniel Smith aa9d30a111 Build RocksDB as an external package 2020-06-05 22:58:45 +00:00
A.J. Beamon 9b3898db05 Ignore throttling errors in fuzz tester 2020-06-05 12:57:45 -07:00
Daniel Smith 9c9eebe9ff Improve cleanup 2020-06-05 18:30:19 +00:00
A.J. Beamon c00e6e7ad9 Reorder call to setting up watch future with waiting for data->version to advance to avoid missing potential mutations. Also add tests for and fix the case where reading the value throws a transaction_too_old error. 2020-06-05 11:24:47 -07:00