b4b7f382a7
Fix issues that a newer compiler warned about.
2018-12-14 14:43:50 -08:00
b08497b7ea
Bug fix, at least some users of IKeyValueStore expect the read actors to make their own copies of key arguments.
2018-10-25 19:48:31 -07:00
f17cc1e20f
StorageServer will no longer send io_error or other inappropriate errors to a client (this would never happen on SQLite). Many bug fixes around error handling, initialization, and shutdown in Redwood. StorageServer now calls init() on its underlying storage engine.
2018-10-24 15:57:06 -07:00
6bb1f4093d
Merge pull request #856 from dropbox/pr/include-fix
...
Adjust all includes to be relative to the root.
2018-10-22 09:51:55 -07:00
1ef29cbf0d
more windows build fixes
2018-10-19 17:00:24 -07:00
268093a96d
Adjust all includes to be relative to the root.
...
Remove the use of relative paths. A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h". Adjust so that every include references such a header with the
latter form.
Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
8dd900a337
fixed the windows build
2018-10-18 20:26:45 -07:00
5bc45958d8
Finished VersionedBTree's IClosable implementation. Added deletion of existing unit test pager state.
2018-10-15 03:43:43 -07:00
3c36722ef8
Renamed redwood test cases to match references in test spec files.
2018-10-05 22:13:22 -07:00
3ea9193fa7
Renamed redwood to redwood-experimental. UnitTest names can now be hidden using # as the first character so that random correctness tests will not run them. Excluded redwood tests from correctness testing. Reverted default storage engine to ssd.
2018-10-05 14:43:54 -07:00
5d9cd9acdc
Correctness test now has additional random reader which doesn't do verification but isn't stopped when the btree is closed. Fixed bug exposed by this where pager snapshots will still try to read pages after the pager has been shut down or even destroyed. Added new error type, shutdown_in_progress.
2018-10-04 23:46:37 -07:00
15c1f87681
Rewrite superpage writes to fix bugs involving logical ID reuse and to make the logic cleaner. Correctness test is stronger, with pipelined writes/commits during verification.
2018-10-01 16:51:57 -07:00
a9f467c502
Btree correctness verification of contents is now asynchronous.
2018-09-28 00:35:03 -07:00
5c3a93e41f
Bug fixes involving PrefixTree length encodings and page size limits, correctness test improvements such as better key size selection and random key reuse, reduced memory footprint of control set.
2018-09-27 16:07:29 -07:00
d7d2b58fef
Increased several testing parameters which exposed new bugs. Bug fix, PrefixTree's current format does not support trees larger than 64k. Cleaned up some debug output.
2018-09-24 02:42:23 -07:00
2fc86c5ff3
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbrpc/AsyncFileCached.actor.h
# fdbserver/IKeyValueStore.h
# fdbserver/KeyValueStoreMemory.actor.cpp
# fdbserver/workloads/StatusWorkload.actor.cpp
# tests/fast/SidebandWithStatus.txt
# tests/rare/LargeApiCorrectnessStatus.txt
# tests/slow/DDBalanceAndRemoveStatus.txt
2018-09-20 03:39:55 -07:00
90d9ef87d7
Changed simplePagerTest prints to use debug_printf() for output which is not reporting an error.
2018-09-19 19:16:18 -07:00
6f84b6cee9
Bug fix, allocated superpage size was wrong. Also added initialization for valgrind build to prevent uninitialized read errors during page checksum calculation.
2018-09-19 18:54:50 -07:00
ca9eb228bc
Bug fix, page write at version based on a reference page's latest didn't work if latest version of the page being written was already at that version.
2018-09-19 01:34:19 -07:00
4df093f1ea
Implemented large key support by adding a superpage concept which is (currently) a BTree-level construct that treats multiple pages from the pager as a single contiguous page, splitting it up on write and reconstituting it on read. Refactored how pages are written along the way. The superpage construct will later be replaced by variable sized page support in IPager.
2018-09-19 00:32:39 -07:00
383cac4991
This checkin is an attempt to support long keys by placing oversize key suffixes (as much as will fit) into a page and forcing the prefix bytes that do not fit to be part of the parent's page boundary leading to this page. This strategy is being abandoned but there are some valuable refactorings and debugging enhancements that are still in the right direction so they are being committed.
2018-08-28 13:46:14 -07:00
2e651f02d3
Crash fixes related to initialization and memory ownership in cursor classes.
2018-07-25 02:29:17 -07:00
4edcd8b0f0
Bug fixes.
2018-07-23 03:49:40 -07:00
9d391498e8
Refactored how key/value memory is held in PrefixTree and VersionedBtree, eliminated many unnecessarily copies of large strings. PrefixTree tests pass but btree is still broken, just committing this because it's a large change set.
2018-07-23 03:09:13 -07:00
7397ea4a79
Bug fixes. Memory corruption was happening due to memory ownership problems with using PrefixTree cursors. Now that internal page boundaries are truncated to the minimal necessary length they can be incomplete tuples of (key, version, offset) which was not being handled correctly (causing incorrect mutation buffer to page mappings during commit) for some partial tuple values. Tuple was modified to allow optional safe decoding of partial integer values and to assert if partial integer values are found but not allowed. Lots of debug output changes.
2018-07-18 03:19:35 -07:00
540d88e85f
Minimal page boundary keys are now being used, though there are still bugs. Many debug output changes. Bug fix regarding common prefix length calculation which was reducing compression and causing incorrect boundary keys.
2018-07-17 00:41:42 -07:00
bbf5ece5af
Btree pages can now borrow prefix bytes from their upper bound key (the next key after their lower bound key in their parent page) as well as their lower bound key.
2018-07-14 13:37:52 -07:00
af9aa43f39
Tree-level prefix compression partly done. Btree page writing and reading now passes parent key lower bound to the PrefixTree builder / cursor so it can be used as a prefix.
2018-07-10 02:24:01 -07:00
2cb0362102
AsyncFileCached now allows writing and truncation of whole pages previously read using readZeroCopy and not yet released without prior readers seeing the effects of the write.
2018-07-05 02:59:13 -07:00
5936113cf4
Bug fix in shutdown. Normalized pager test filenames.
2018-07-04 21:12:09 -07:00
f55e952307
Disabled memory pager unit test as it is not currently being maintained. Bug fix in unversioned IKeyValueStore shutdown.
2018-07-04 05:51:01 -07:00
09e68a4335
Lots of bug fixes around page reads and concurrency.
2018-07-03 15:39:32 -07:00
e4f347d3fd
Bug fix: PrefixTree::PathEntry was not safely copyable or swappable which caused a subtle bug at the higher level causing a Cursor state to appear valid but not make sense. More debug output improvements.
2018-06-17 06:48:41 -07:00
ee17854713
Bug fixes in internal page boundaries and prefix tree overhead calculations. Improved performance test output and increased amount of work it can do. Removed temporary restrictions on correctness test parameter ranges.
2018-06-14 17:52:25 -07:00
77df55d9e1
Bug fixes in page building causing memory corruption, improved debugging output, added memory initialization to new pages when valgrind is enabled to avoid unconcerning errors.
2018-06-14 04:15:14 -07:00
d291cc9ec3
Bug fixes in cursor logic related to database and page ends. Rewrote cursor related toString() methods and debug output to make debugging easier.
2018-06-12 01:43:19 -07:00
69b713918b
VersionedBTree now uses PrefixTree based pages (with bugs). This required significant changes to both classes because the interface and semantics for building, seeking in, and iterating through pages is very different from the previous trivial approach which was based on serialized vectors. PrefixTree node format rewritten to support optional values without increasing overhead for common node scenarios. PrefixTree::Cursor rewritten to reuse path prefix memory instead of allocating new memory on each movement which is then 'leaked' until destruction. PrefixTree::Cursor movement modified to work better with VersionedBTree::InternalCursor, which was also heavily modified. Added knobs related to key arrangement in PrefixTree nodes. Added StringRef::toHexString() as an alternative to printable() to make reading raw PrefixTree data easier. PrefixTree performance is temporarily worse with this update and VersionedBtree fails its unit test.
2018-06-08 03:32:34 -07:00
4a58ed3b11
Debug output improvements, removed unused function.
2017-10-09 13:31:54 -07:00
b15d29dd6b
VersionedBTrees now have names for logging purposes (trees do not deal with files directly but rather just an IPager). Bug fix in pager vacuumer which was not updating checksums. KeyValueStoreRedwood now relies on VersionedBTree to handle commit ordering, as there’s no reason to do this in both classes.
2017-10-09 13:24:16 -07:00
0635a31604
Better error and dispose/close handling, closer to correct behavior for an IKeyValueStore.
2017-10-02 03:32:22 -07:00
ed226e2e1c
VersionedBtree now allows writes while a commit is in progress. Also, lots of bug fixes, like the one where writes during commits weren’t prevented but also didn’t work.
2017-09-22 17:18:28 -07:00
248dab79b6
Created “redwood” storage engine option and many changes to support that including IKeyValueStore::init() and custom DiskQueue file extensions.
2017-09-21 23:51:55 -07:00
d880569d52
Checkpointing progress on KeyValueStoreMVBTree. All methods are implemented to a usable point, and everything compiles, but Worker does not yet try to use it.
2017-09-21 04:43:49 -07:00
e16ce63db0
Bug fix, find was being done with a key ref that wasn’t living as long as the actor.
2017-09-21 00:58:56 -07:00
eee8116fa4
Cleaned up how the root page is written and page write debug output.
2017-09-20 17:50:02 -07:00
9b4cbe94aa
Bug fix, values exactly the size of the maximum part size were still split into a part set but of size 1, which is invalid. Also some debug output changes.
2017-09-19 13:03:30 -07:00
25f958ac3b
Bug fixes. Finished backward iteration on Cursor. Correctness range verification now also reads the range in reverse and verifies the result.
2017-09-17 04:38:01 -07:00
1aae32dc16
Some cleanup. InternalCursor does not need to be reference counted.
2017-09-16 02:09:09 -07:00
c89b45158a
Removed InternalCursor::seekEqualOrGreaterThan because it isn’t needed, it’s always better to do a <= search so if the target key is found the initial cursor position will be at the last version prior to the target version.
2017-09-16 01:59:16 -07:00
cea7903b46
Bug fixes involving edges cases with find >= and range read endpoints.
2017-09-16 01:45:39 -07:00
Alex Miller