7c801513e2
Fix cases where latency band config could be discarded during recovery or process start.
2019-11-20 11:44:18 -08:00
c8aef99e87
Merge pull request #2354 from etschannen/fix-dd-recruit
...
The data distributor could still be killed and re-recruited infinitely
2019-11-13 13:26:57 -08:00
ffc89d1182
fix: dd test recruitment should prefer the location of ratekeeper over other used processes
2019-11-13 12:58:55 -08:00
11525f6922
added comments
2019-11-13 12:53:23 -08:00
8f725db92e
serialization of logRangeMutation->second caused long slow tasks
2019-11-12 23:06:58 -08:00
b1b5f88cb1
Merge pull request #2344 from bnamasivayam/release-6.2
...
Fix bug where DD or RK could be halted and re-recruited in a loop for…
2019-11-12 21:47:28 -08:00
5e463f7290
Merge pull request #2342 from ajbeamon/packet-size-event-rename
...
Rename LargePacket warnings to distinguish between sent and received packets.
2019-11-12 20:42:55 -08:00
be303cad7a
Merge pull request #2339 from etschannen/feature-increase-reboot-priority
...
Increase the priority of reboot and recruitment requests
2019-11-12 20:42:22 -08:00
7ebbb4d9cf
Merge pull request #2337 from etschannen/feature-logrouter-peek
...
Do not limit log router peeking from satellite logs
2019-11-12 20:41:22 -08:00
c26bb52979
Enable Consistency Checks for DD and RK.
2019-11-12 20:11:08 -08:00
2e41497580
This commit tries to distribute RK and DD among other empty available processes.
2019-11-12 17:52:42 -08:00
17059596e9
Merge pull request #2346 from satherton/feature-redwood
...
Update Redwood
2019-11-12 16:25:10 -08:00
f5282f2c7e
Fix bug where DD or RK could be halted and re-recruited in a loop for certain valid process class configurations. Specifically, recruitment of DD or RK takes into account that master process is preferred over proxy, resolver or cc.
...
But check for better DD only looks for better machine class ignoring that the new recruit could share a proxy or resolver or CC. Also try to balance the distribution of the DD and RK role if there are enough processes to do so.
2019-11-12 14:22:36 -08:00
ef801a6432
Rename LargePacket warnings to distinguish between sent and received packets. Also remove Net2_ prefix from packet size trace events.
2019-11-12 09:23:46 -08:00
f841d14141
Bumped format versions, also simplified version scheme to a pager version and a btree version, removing per-page versions for queue and btree pages.
2019-11-11 16:28:21 -08:00
1e5677b55a
increase the priority of reboot and recruitment requests
2019-11-11 15:17:11 -08:00
9227de5c20
Redwood correctness unit test was using wallclock based time limit which breaks determinism.
2019-11-11 15:13:58 -08:00
61558eea04
Implemented page preloading on BTree cursor seeks to enable hiding latency on soon-to-be-read sibling pages. Added random scans with various preload sizes to the set performance unit test. ObjectCache now tracks hits, misses, and pages which were preloaded but then never used prior to eviction. BTree pages no longer store flags because height is sufficient. Removed virtual specifier in classes not designed to be further inherited. Removed old prototype code (PrefixTree, IndirectShadowPager, MemoryPager) as some interface changes are incompatible and they are no longer worth maintaining.
2019-11-11 09:54:22 -08:00
396dccbc98
when peeking from satellites we do not need to limit the amount of peeking on log router tags, because that is the only thing that can be peeked from a satellite log
2019-11-08 18:34:05 -08:00
d76070e37d
Merge pull request #2331 from satherton/feature-redwood
...
Update redwood
2019-11-08 10:38:16 -08:00
2aa672cb59
When bulk building pages, make most of them full.
2019-11-07 15:52:23 -08:00
5d00d93665
Fixed errors found by valgrind involving incorrect page memory lifetimes for IO operations plus some false positives for partially used pages.
2019-11-07 13:57:09 -08:00
a11db961cd
Merge pull request #2323 from etschannen/feature-efficient-buffered-cursor
...
remote logs use bufferedCursor when peeking from log routers
2019-11-05 20:45:15 -08:00
dbc5a2393c
combineMessages still did not serialize tags correctly
2019-11-05 18:44:30 -08:00
1c873591be
fixed a compiler error
2019-11-05 18:32:15 -08:00
86560fe727
fix: tempTags was not used correctly
2019-11-05 18:22:25 -08:00
a8ca47beff
optimized memory allocations by using VectorRef<Tag> instead of std::vector<Tag>
2019-11-05 18:07:30 -08:00
4a597fdcce
increase the task priority of popping
2019-11-05 15:03:41 -08:00
ea428b72f5
Bug fix, DWALPager must flush its page ID queues in order to get an accurate user page count.
2019-11-05 01:11:34 -08:00
daac8a2c22
Knobified a few variables
2019-11-04 20:21:38 -08:00
457896b80d
remote logs use bufferedCursor when peeking from log routers to improve performance
...
bufferedCursor performance has been improved
2019-11-04 19:47:45 -08:00
3728ef996e
Merge pull request #2318 from satherton/feature-redwood
...
Redwood update
2019-11-04 15:14:17 -08:00
0c95fef8aa
Bug fix in tree clear and size check where sometimes there could still be old versions of pages in use because not enough commits have passed for them to be rolled off and freed.
2019-11-04 12:59:57 -08:00
4e9c3164a8
COWPager can now internally remap page IDs by version and has been renamed to DWALPager. This causes the B+Tree to no longer have to rewrite all ancestors of an updated page. FIFOQueue now has a read-only cursor and a peekAll() method to read an entire queue without popping it. Fixed some valgrind false positives, made some debug logging improvements. Fixed bug in pager shutdown where it could wait on an ActorCollection containing canceled futures.
2019-11-04 03:04:03 -08:00
8a3521f945
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-enable-parallel-peek
2019-11-01 14:04:22 -07:00
85c315f684
Fix: parallelPeekMore was not enabled when peeking from log routers
2019-11-01 14:02:44 -07:00
1dc5985062
Merge pull request #2305 from etschannen/release-6.2
...
merges crossing systemKeys.begin did not decrement systemSizeEstimate
2019-11-01 09:12:01 -07:00
8f0348d5e0
fix: merges which cross over systemKeys.begin did not properly decrement the systemSizeEstimate
2019-10-31 16:38:33 -07:00
547616d5f1
Merge branch 'release-6.2' of github.com:apple/foundationdb into feature-redwood
2019-10-29 16:33:22 -07:00
9c0d671d07
Two bug fixes in Redwood related to split KV pairs and one was masking the other. The first bug resulted in an incomplete erasure of fragments for a split KV pair and the second bug would generate an unnecessary explicit null record for the same key which would cause reads to correctly see the key as missing. Redwood correctness test now clears the tree and verifies expected resulting pager footprint, which succeeds due to the bug fixes.
2019-10-29 01:31:59 -07:00
40d53e23f5
Optimization, only the first btree mutation boundary for a subtree needs to be compared to the subtree's lower bound. Also removed a check for a condition which is no longer possible due to other changes.
2019-10-28 16:05:11 -07:00
0d993522d3
CommitSubtree() will now return an empty page set even for the tree root because commit_impl() handles this correctly. Improved commitSubtree() debug output related to which mutations are relevant to a subtree. Added random setting of range clear boundaries after clear() in Redwood correctness to make sure mutation buffer logic handles this correctly. B+Tree's dbEnd mutation is represented as a clear to prevent unnecessary rightmost subtree traversal during commit.
2019-10-28 04:00:37 -07:00
5cf0045bc0
Merge pull request #2294 from satherton/feature-redwood
...
Bug fixes in Redwood
2019-10-25 14:56:01 -07:00
2ee1782c19
Bug fixes in Redwood. BTree height was not being reset when a new empty root is written. IKeyValueStore wrapper was not obeying the row limit in a reverse range query. Added yields to and delays to break up tasks and set IO priorities.
2019-10-25 14:52:06 -07:00
a7492aab0a
fix: poppedVersion can update during a yield, so all work must be done immediately after getMore returns
2019-10-23 23:06:02 -07:00
f8e44d2f71
fix: If a storage server was offline, it would not be checked for being in an undesired dc
2019-10-23 23:04:39 -07:00
eb910b850b
fixed a window build error
2019-10-23 13:48:24 -07:00
2722c8b188
avoid starting a new startSpillingActor with every TLog recruitment
2019-10-23 11:15:54 -07:00
ae3f8132a7
Merge pull request #2280 from satherton/feature-redwood
...
Update redwood
2019-10-23 10:57:38 -07:00
9197b03122
Merge pull request #2279 from ajbeamon/latency-band-ignore-batch
...
Ignore batch priority GRVs for latency band tracking
2019-10-23 10:52:44 -07:00
A.J. Beamon