470896bdc4
Redwood inline same-size value updates ( #5925 )
...
* Refactored mutation application in leaf nodes to do fewer comparisons and do in place value updates if the new value is the same size as the old value.
* Renamed updatingInPlace to updatingDeltaTree for clarity. Inlined switchToLinearMerge() since it is only used in one place.
2021-11-10 08:22:57 -08:00
d0c9cf4fb0
Enable mismatched-tags clang warning
2021-11-01 14:18:31 -07:00
25257f6f87
Enable unused-function warning for clang
2021-11-01 14:18:31 -07:00
b9a22a61ef
Fix many -Wreorder-ctor warnings
2021-07-23 17:33:18 -07:00
8cbc26d436
Added documentation for DeltaTree2.
2021-06-10 02:29:17 -07:00
69f7c7cba2
Make deltatree debug toggle easier to use.
2021-06-09 21:19:52 -07:00
f2904dadf3
Reading BTree pages no longer requires boundary records, as they are not needed if the page is already cached.
2021-06-08 16:58:35 -07:00
f95d592db8
Optimized record delta decoding / applying a bit, changed substring order to (value, keySuffix) since value is needed more frequently. Added DeltaTree2 item type requirement to create T from base's cached partial item instead of full record.
2021-06-08 16:58:35 -07:00
3451c2242f
DeltaTree2::Cursor now reconstructs current item on-demand, caches it in an Optional member, and does not initialize it when a cursor is copied.
2021-06-08 16:58:35 -07:00
751bac2271
Write path no longer uses non-caching reads because it is no longer necessary to avoid a page copy. Page copies are only done just before an actual change is made.
2021-06-08 16:46:09 -07:00
a6f7d37a25
Bug fixes related to DeltaTree2::Cursor contract being different from DeltaTree::Cursor.
2021-06-08 16:46:09 -07:00
8ef516ead2
Bug fixes from bad search/replace. DeltaTree2::Cursor now keeps current decoded item as a member instead of calculating it on demand in get().
2021-06-08 16:46:09 -07:00
d208d3f3ec
Bug fixes, moveFirst/Last didn't handle tree size of 1 correctly.
2021-06-08 16:46:09 -07:00
caf4b3c345
DeltaTree2 refactor. Nodes no longer contain parent offsets. DecodedCache no longer uses a hash but rather a vector of DecodedNodes, which Cursors reference by vector index. DecodedNodes contain parent node indexes which are populated on-demand, making storage in the serialized form no longer necessary.
2021-06-08 16:46:04 -07:00
b0ec76d401
Test output improvements.
2021-06-08 15:32:54 -07:00
9ab69b5cb1
RedwoodRecordRef support for DeltaTree2.
2021-06-08 15:32:54 -07:00
701f05e513
Bug fix, recursive call to get() could cause rehashing so hash lookup must be redone afterward.
2021-06-08 15:32:54 -07:00
b2c7d957e2
Added DeltaTree2, which can be shared between updated versions of the same tree, but so far it is 50% slower.
2021-06-08 15:32:26 -07:00
1ee25e9b91
Avoid casting NaN to uint8_t
2021-06-04 09:43:50 -07:00
390b026f08
Merge branch 'master' into arena-page
2021-05-16 05:06:50 -07:00
799e7cc213
BTreeCursor now exposes the current Page and Mirror arenas so that users extracting Standalone Keys and Values can just reference those instead of copying.
2021-05-05 21:05:32 -07:00
63d82b9b63
Merge pull request #4616 from sfc-gh-satherton/redwood-improvements
...
Redwood page splitting/building refactor
2021-05-03 13:07:41 -07:00
4b5bca6761
Minor Redwood comparison optimizations
2021-04-23 18:49:43 +00:00
5c93e684f8
Added comments.
2021-04-04 19:23:08 -07:00
90ebf90c8b
Refactored page rebuild logic to bulk build pages full and split pages more evenly.
2021-04-03 19:54:49 -07:00
df90cc89de
apply clang-format to *.c, *.cpp, *.h, *.hpp files
2021-03-10 10:18:07 -08:00
8766bb44b1
Address review comment
2021-03-09 01:40:59 +00:00
5ecb1dfcba
Move ASSERT to right before dereference
2021-03-08 22:37:30 +00:00
7188c7ddfb
Add ASSERT to help clang-tidy
...
If n == nullptr here, then we'll dereference a nullptr. Add an assert.
2021-03-08 18:50:28 +00:00
f306666e21
Added commonPrefixLength for StringRefs that takes a prefix skip length.
2020-05-10 04:24:11 -07:00
1966dbb576
Checkpointing large refactor of how commitSubtree() works regarding internal pages such that now they can be incrementally modified and detecting changes involves far fewer comparison and staging of records.
2020-05-08 07:11:51 -07:00
dda0993d16
Apply clang-format to Redwood source.
2020-04-24 14:12:40 -07:00
d4334256f1
Bug fix: BTree::writePages() was not accounting for the small amount of unpredictability in node overhead sizes which could lead to serializing a tree that doesn't fit into its destination page buffer. WritePages() now skips the prefix common to all records in the input set when calling deltaSize() to estimate resulting DeltaTree serialized sizes.
2020-04-08 20:18:41 -07:00
9bf0b5e71f
RedwoodRecord simplified to remove chunked key-value pair concept. Cursor and InternalCursor updated to be correct (but not optimal) for new record definition. DeltaTree::seek* refactored to support more seek types and with minor code duplication. Lots of debug output improvements. Refactored how bulk pages are written and introduced fill factor percentage (though since internal pages are not yet incrementally modified this is temporarily worse for those).
2020-04-06 20:27:30 -07:00
1ed425233a
DeltaTree now has two size modes, 2 byte integer offers for <= 65k and 4 byte integer offsets for larger.
2020-04-01 04:24:40 -07:00
5a1884d054
Leaf page modifications are now made incrementally until the DeltaTree is out of space or the tree is too imbalanced, at which point it falls back to a linear merge and page(s) rebuild.
2020-02-21 16:26:44 -08:00
986e533c8c
Bug fix, DeltaTree was not updating numItems after erase / insert operations which only change the deleted flag of an existing equivalent entry.
2020-02-20 22:49:36 -08:00
9f4e39f1ef
DeltaTree now supports inserting a deleted item that is still present as a deleted node by simply clearing the node's deleted flag. Insertion of duplicates (determined by T::compare() == 0) is no longer allowed.
2020-02-07 01:00:11 -08:00
9cf9c46668
DeltaTree now tracks internal used/free/deleted space, and insert will fail safely if a new item does not fit.
2020-02-05 15:30:04 -08:00
9dbb9e4cec
DeltaTree now supports node deletion by flagging nodes as deleted (a state which the T::Delta must be able to store and return). DeltaTree::Cursor will hide deleted nodes from view.
2020-02-04 01:22:27 -08:00
460c9b78d8
DeltaTree improvements. DecodedNodes store non-parent ancestor pointer which enables moveNext() and movePrev() operations to be O(1) from any leaf node. This also enables seekLessThanOrEqual to accept a hint in the form of a cursor with a position close to the target to avoid traversal from the root when the destination is close to the hint. DeltaTree T::compare() now must take a skipLen argument which is used by insert() and seek operations to skip over bytes known to be shared between the new item or query and every node in the tree.
2020-01-31 00:32:48 -08:00
887acae74a
DeltaTree cursor equality only needs to check the DecodedNode pointer.
2019-12-01 22:28:50 -08:00
5f1644f293
DeltaTree::Reader is now DeltaTree::Mirror and supports insertion into a DeltaTree. DeltaTrees now support an item count, so BTreePage no longer has an item count, so the VersionedBTree format version has been bumped.
2019-11-23 00:09:11 -08:00
61558eea04
Implemented page preloading on BTree cursor seeks to enable hiding latency on soon-to-be-read sibling pages. Added random scans with various preload sizes to the set performance unit test. ObjectCache now tracks hits, misses, and pages which were preloaded but then never used prior to eviction. BTree pages no longer store flags because height is sufficient. Removed virtual specifier in classes not designed to be further inherited. Removed old prototype code (PrefixTree, IndirectShadowPager, MemoryPager) as some interface changes are incompatible and they are no longer worth maintaining.
2019-11-11 09:54:22 -08:00
848a344aa7
DeltaTree building now passes the prev/next common prefix length, which is effectively a subtree shared prefix, to recursive calls, which enables each new prev/next common prefix comparison to start from the position at which the previous call on the stack left off.
2019-09-27 22:56:33 -07:00
b19ef86ab9
Pager2 interface now supports getting a read snapshot at a version and setting the oldest readable version. FIFOQueue now supports pushFront() which is needed for the BTree's incremental tree deletion process.
2019-09-27 21:46:24 -07:00
71ba490cf8
Removed use of the C "struct hack" as it is not valid C++. Replaced zero-length members with functions returning a pointer for arrays or a reference for single members.
2019-07-02 16:02:58 -07:00
3e155a2563
Bug fixes.
2019-05-29 17:38:55 -07:00
02882dbf00
Checkpointing progress, RedwoodRecordRef and DeltaTree tests pass but BTree test does not. RedwoodRecordRef::Delta rewritten to actually do prefix compression on key and integer fields. Added related unit tests and benchmarks. Some improvements to DeltaTree and requirements on its T and Delta types to avoid repeated common prefix discovery.
2019-05-29 06:23:32 -07:00
6660ccd936
Large rewrite/refactor from the page format outward. Replaced the in-page format, PrefixTree, with more general and templated implementation of the same concept, DeltaTree, which will be incrementally modifiable soon. New implementation for in-page cursors to use a per-page arena and share decoded binary tree nodes between cursors in the same page. New implementation for internal record B+Tree cursor which uses reference counted path hops which makes cloning a cursor very cheap. New implementation of external facing B+Tree cursor which makes efficient use of two internal cursors to find user-visible KV pairs and return them without copying keys and values except in the case of a large sharded value.
2019-02-21 02:46:30 -08:00
Steve Atherton