Commit Graph

7535 Commits

Author SHA1 Message Date
Stephen Atherton 0c95fef8aa Bug fix in tree clear and size check where sometimes there could still be old versions of pages in use because not enough commits have passed for them to be rolled off and freed. 2019-11-04 12:59:57 -08:00
Meng Xu e345c9061f FastRestore:Refine debug messages 2019-11-04 11:47:38 -08:00
Stephen Atherton 4e9c3164a8 COWPager can now internally remap page IDs by version and has been renamed to DWALPager. This causes the B+Tree to no longer have to rewrite all ancestors of an updated page. FIFOQueue now has a read-only cursor and a peekAll() method to read an entire queue without popping it. Fixed some valgrind false positives, made some debug logging improvements. Fixed bug in pager shutdown where it could wait on an ActorCollection containing canceled futures. 2019-11-04 03:04:03 -08:00
Meng Xu 27c7ef09a3 FastRestore:Revise code in self review
When we read the txnId from decodeRestoreApplierKey func,
we should convert the integer to little endian.
2019-11-03 20:43:17 -08:00
Meng Xu 7cf87e9ae3 FastRestore:Add ParallelRestoreCorrectnessCycle.txt test 2019-11-03 17:31:54 -08:00
Meng Xu 63359bfc8b FastRestore:handleInitVersionBatchRequest:Ensure exact once execution
Also increase the test workload for BackupAndParallelRestoreWithAtomicOp test
2019-11-03 17:26:13 -08:00
Meng Xu 58aa6711e4 FastRestore:ApplyToDB:BugFix:Serialize integer as bigEndian to ensure lexico order 2019-11-03 17:26:07 -08:00
tclinken 8f84fbc4b9 Only print 'waiting for DD to end...' if test actually waits 2019-11-03 16:13:32 -08:00
Evan Tschannen b326d26a47
Merge pull request #2308 from etschannen/master
Merge 6.2 into master
2019-11-01 15:54:32 -07:00
Evan Tschannen 4de60fc437 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/TLogServer.actor.cpp
2019-11-01 15:48:04 -07:00
Evan Tschannen c805ed11e8
Merge pull request #2307 from etschannen/feature-enable-parallel-peek
Parallel peeking was not enabled when fetching from log routers
2019-11-01 15:39:00 -07:00
Evan Tschannen f4143c4f50 updated release notes 2019-11-01 14:07:01 -07:00
Evan Tschannen 8a3521f945 Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-enable-parallel-peek 2019-11-01 14:04:22 -07:00
Evan Tschannen 85c315f684 Fix: parallelPeekMore was not enabled when peeking from log routers 2019-11-01 14:02:44 -07:00
A.J. Beamon 67dbac720f
Merge pull request #2306 from apple/revert-2300-fix-memory-leak
Revert "Clean up some memory after network thread exits"
2019-11-01 11:48:29 -07:00
Jingyu Zhou 00b3c8f48a
Revert "Clean up some memory after network thread exits" 2019-11-01 11:05:31 -07:00
A.J. Beamon 1dc5985062
Merge pull request #2305 from etschannen/release-6.2
merges crossing systemKeys.begin did not decrement systemSizeEstimate
2019-11-01 09:12:01 -07:00
Evan Tschannen 7f75eca7cb updated release notes 2019-10-31 17:06:58 -07:00
Evan Tschannen 8f0348d5e0 fix: merges which cross over systemKeys.begin did not properly decrement the systemSizeEstimate 2019-10-31 16:38:33 -07:00
Evan Tschannen 24bc9aeaf5
Merge pull request #2224 from atn34/test-buggified-delay
Replace /flow/delayOrdering with /flow/buggifiedDelay
2019-10-31 11:20:55 -07:00
Evan Tschannen 58b984b846
Merge pull request #2047 from tclinken/lock-aware-db-ping
Use lock aware transaction for pingDatabase
2019-10-31 10:24:20 -07:00
Evan Tschannen 2ce57b6735
Merge pull request #2198 from ajbeamon/increase-location-cache-size
Increase the default client shard location cache size by a factor of 2.
2019-10-31 10:23:45 -07:00
Evan Tschannen 71dfaa3f95
Merge pull request #2275 from dongxinEric/bugfix/2273/fix-read-key-sampling
Resolves #2273: Use a large value for read sampling size threshold. Also at sampling …
2019-10-31 10:21:58 -07:00
Evan Tschannen 5f6204b550
Merge pull request #2303 from ajbeamon/cleanup-cleanup-command
Cleanup the fdbbackup cleanup command output
2019-10-31 10:19:28 -07:00
A.J. Beamon f175ed30b3 Cleanup the fdbbackup cleanup command output. Add cleanup to the usage output printed for fdbbackup. 2019-10-31 09:52:21 -07:00
Xin Dong 199a34b827 Defined a minimum read cost (a penalty) for empty read or read size smaller than it. Fixed several review comments. 2019-10-30 10:04:19 -07:00
Meng Xu 988e88338b
Merge pull request #2300 from jzhou77/fix-memory-leak
Clean up some memory after network thread exits
2019-10-29 17:45:02 -07:00
Stephen Atherton 547616d5f1 Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-redwood 2019-10-29 16:33:22 -07:00
Jingyu Zhou 6c28da9093 Clean up some memory after network thread exits 2019-10-29 13:59:55 -07:00
Stephen Atherton 9c0d671d07 Two bug fixes in Redwood related to split KV pairs and one was masking the other. The first bug resulted in an incomplete erasure of fragments for a split KV pair and the second bug would generate an unnecessary explicit null record for the same key which would cause reads to correctly see the key as missing. Redwood correctness test now clears the tree and verifies expected resulting pager footprint, which succeeds due to the bug fixes. 2019-10-29 01:31:59 -07:00
Stephen Atherton 40d53e23f5 Optimization, only the first btree mutation boundary for a subtree needs to be compared to the subtree's lower bound. Also removed a check for a condition which is no longer possible due to other changes. 2019-10-28 16:05:11 -07:00
Stephen Atherton 0d993522d3 CommitSubtree() will now return an empty page set even for the tree root because commit_impl() handles this correctly. Improved commitSubtree() debug output related to which mutations are relevant to a subtree. Added random setting of range clear boundaries after clear() in Redwood correctness to make sure mutation buffer logic handles this correctly. B+Tree's dbEnd mutation is represented as a clear to prevent unnecessary rightmost subtree traversal during commit. 2019-10-28 04:00:37 -07:00
Meng Xu 529b35a886
Merge pull request #2296 from atn34/remove-nonsense-const
Remove several nonsensical const uses
2019-10-26 18:04:32 -07:00
Andrew Noyes b7b5d2ead3 Remove several nonsensical const uses
These seem to be all the ones that clang's -Wignored-qualifiers
complains about
2019-10-26 14:30:34 -07:00
Evan Tschannen 5cf0045bc0
Merge pull request #2294 from satherton/feature-redwood
Bug fixes in Redwood
2019-10-25 14:56:01 -07:00
Stephen Atherton 2ee1782c19 Bug fixes in Redwood. BTree height was not being reset when a new empty root is written. IKeyValueStore wrapper was not obeying the row limit in a reverse range query. Added yields to and delays to break up tasks and set IO priorities. 2019-10-25 14:52:06 -07:00
Evan Tschannen 1d26ca4aa7
Merge pull request #2293 from etschannen/release-6.2
fixed Javadoc headings
2019-10-25 13:40:16 -07:00
Evan Tschannen 0953bf376d fixed Javadoc headings 2019-10-25 13:38:04 -07:00
Jingyu Zhou 221e94fd01
Merge pull request #2145 from atn34/open-for-ide-debug
Open for ide debug
2019-10-25 13:32:49 -07:00
Andrew Noyes 1827e77f2e Update fdbserver/FDBExecHelper.actor.cpp
Co-Authored-By: Jingyu Zhou <jingyuzhou@gmail.com>
2019-10-25 10:42:22 -07:00
Andrew Noyes e4acd2e318 Disable TLS temporarily for OPEN_FOR_IDE build 2019-10-25 10:42:22 -07:00
Andrew Noyes a00f04eb20 Fix gcc with Make 2019-10-25 10:42:22 -07:00
Andrew Noyes b893374c68 Add -Wno-attributes for gcc 2019-10-25 10:42:22 -07:00
Andrew Noyes a3d9e549ee Remove rule from vcxproj 2019-10-25 10:42:22 -07:00
Andrew Noyes aed9dfd148 Fix flow header guard 2019-10-25 10:42:22 -07:00
Andrew Noyes de8921b660 Move RestoreWorkerInterface to fdbclient 2019-10-25 10:42:22 -07:00
Andrew Noyes 3f62d2b506 Fix actual build 2019-10-25 10:42:22 -07:00
Andrew Noyes daeb0e9ed6 Attempt to fix Makefile 2019-10-25 10:42:22 -07:00
Andrew Noyes d4de608bb6 Fix OPEN_FOR_IDE build 2019-10-25 10:42:22 -07:00
Andrew Noyes ec0789f2e7 Build in Debug mode by default for OPEN_FOR_IDE build 2019-10-25 10:41:42 -07:00