wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
20,016 Commits 32 Branches 308 Tags 272 MiB
Commit Graph

128 Commits

Author SHA1 Message Date
Steve Atherton 6af2b83bc6 Updated BTree write-only forensic fields for last known logical page ID and last known logical parent page ID. 
Also simplified multipage update loops which were using an unnecessary temp variable left from a prior implementation involving remapping.
 2022-04-08 01:17:50 -07:00
Steve Atherton 702bd59ded Clear unused memory regions for structures written to disk. 2022-04-07 23:02:33 -07:00
Steve Atherton 3ef18bc173 Rename Page and VersionHeader to PageHeader for consistency. Check for known encrypted types explicitly in isEncodingTypeEncrypted(). Make writePhysicalBlock() hold a Page reference which is simpler and more robust. Remove unused argument in switchTree(). 2022-04-07 20:59:34 -07:00
Steve Atherton 1033f64da2 Removed BTree header parse from cursor creation, which recently became necessary, by storing the extracted root link in the PagerSnapshot. Refactored ArenaPage::userData and userDataDestructor to a generic ArbitraryObject class which can own arbitrary heap allocated objects or references and has a cleaner interface. Added ArbitraryObject to IPagerSnapshot. 2022-04-04 02:45:08 -07:00
Steve Atherton d7838982f9 Code simplification and minor overhead reduction - don't check multi-page BTree nodes for remapped pages because they no longer can contain any. 2022-04-03 23:48:54 -07:00
Steve Atherton dbccd47650 Added pageFormat to ArenaPage header. 2022-04-03 03:06:53 -07:00
Steve Atherton 39fb0a44d7 Merge commit 'f09bdc840c00d712487500b9e752d87cedb1964a' into redwood-header-changes 
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
 2022-04-03 00:37:01 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
Steve Atherton 30957c3a43 Fix unclear comment. 2022-03-10 14:22:45 -08:00
Steve Atherton a32c3a2891 Redwood now uses one global page cache eviction order and size limit for all page sizes, for real or virtual processes, and DecodeCache memory is counted against the global page cache budget. 2022-02-25 03:20:53 -08:00
Steve Atherton bd12aacee4 Refactored main headers for Pager and BTree to use serialization framework to enable future evolution. 2022-01-11 22:04:27 -08:00
Steve Atherton 722465f7cd Improvements to encrypted related APIs and error handling. 2022-01-09 05:17:32 -08:00
Steve Atherton 7a08d59c22 Addressing PR discussions. Many renames. Changed encryption API to use new EncryptionKeyRef struct. Added a tree offset byte to BTreePage to allow for adding an additional field in the future. Added dummy definition for valgrind defined check. Added check that BTree pages are encrypted if encryption is enabled to prevent attack by replacing encrypted page with unencrypted page. 2022-01-06 22:27:22 -08:00
Steve Atherton d4bb27c05a Added static template ::construct(args...) on both ReferenceCounter<> classes as a more convenient way to construct references. 2022-01-06 03:01:29 -08:00
Steve Atherton 1ec8dd38b7 Refactored ArenaPage to include sub-header and payload offsets in the universal header to make them easier to work with and more future proof. Updated all ArenaPage comments. 2022-01-06 02:38:41 -08:00
Steve Atherton fec872e11e Reorganized header/footer versioned structs, reworked preWrite() to make room for supporting update of older header versions, and added comments. 2022-01-02 04:42:39 -08:00
Steve Atherton 3ae2ee194c Bug fixes with encryption key provider initialization. 2021-12-27 16:21:18 -08:00
Steve Atherton 642541ba16 Added IEncryptionKeyProvider and first implementation to provide keys for dummy XOR encryption mode. Renamed some things for clarity. 2021-12-27 05:53:49 -08:00
Steve Atherton f5610e5831 Fixed a valgrind-only errors caused by writing structures with intentionally-unused parts to disk and basing checksums on them. Added a valgrind check of ArenaPage payload to make finding issues like this more direct. 2021-12-26 22:05:52 -08:00
Steve Atherton 4f8e2a8e03 Bug fix: ArenaPage::subPage() wasn't decoding the page. Debug-only crash fix in FIFOQueue::Cursor::toString(). 2021-12-25 02:53:38 -08:00
Steve Atherton 7896c2979f Finished Redwood page API refactor and implemented dummy page encryption scheme with XOR, used randomly in simulation, to prove that pluggable encryption schemes fit in the design. 2021-12-25 00:22:36 -08:00
Steve Atherton 592006a537 Several bug fixes in ArenaPage pre and post read/write initialization. Several bug fixes in Queue and Page usage of ArenaPage. Bug fixes in page splitting logic involving incorrect space calculations within the new available page space. 2021-12-11 19:41:14 -08:00
Steve Atherton fc86ee147b Merge branch 'page-api-bug-fixes' into redwood-header-changes 2021-12-09 02:22:26 -08:00
Steve Atherton 4bc0cfe4af Many bug fixes in new page API and usages of it. 2021-12-09 02:21:50 -08:00
Steve Atherton 4f94515bf4 Merge branch 'master' of https://github.com/apple/foundationdb into redwood-header-changes 
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
#	fdbserver/art_impl.h
 2021-12-05 14:45:15 -08:00
Steve Atherton d1ede0225b Initial commit of Redwood page header format and API to enable different page encodings to support encryption or different checksum types. 2021-12-05 13:39:35 -08:00
Steve Atherton 0f5535fce1 Remove explicit tryEvict(x) as it is unused and it is functionally replaced by prioritizeEviction(x). 2021-11-27 03:26:34 -08:00
Steve Atherton 3901d60548 Test-only bug fixes in Redwood along with debug logging detail improvements. Added clearRemapQueue() to Pager to more cleanly and reliably expire all old data and process the remap queue, fixing a bug where with certain configuration parameters and a lot of data the DestructiveSanityCheck would fail because it would not run cleanup long enough. Added more parameters to performance/set unit test. 2021-11-19 01:00:14 -08:00
Steve Atherton 84854761cb Change Redwood to use xxhash for checksums. 2021-11-02 21:47:31 -07:00
Steve Atherton b75edbda31 Merge branch 'master' of github.com:apple/foundationdb into RedwoodSuperpage 
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
 2021-10-25 00:55:52 -07:00
Fuheng Zhao de2afc6d9a conver to standalone in earlier stage to improve the throughput 2021-09-17 15:57:59 -07:00
Fuheng Zhao 0829230e66 format IPager.h 2021-09-10 11:32:43 -07:00
Steve Atherton b06cf7a328 Redwood commit versions are now FDB commit versions for storage instances. Redwood no longer makes internal commits during Pager or BTree recovery so that commit version is always set by the user. 2021-09-10 03:28:46 -07:00
Fuheng Zhao 19e3132632 remove the extra space in these two files 2021-09-07 16:17:32 -07:00
Fuheng Zhao 9aab561f30 format 2021-09-07 16:13:13 -07:00
Fuheng Zhao feab3c4cf8 revert back changes in tracking cache size 2021-09-07 09:08:26 -07:00
Fuheng Zhao 82929fe40e check the thoroughput without the keeping the size 2021-09-03 10:24:21 -07:00
Fuheng Zhao 3bb9ba0687 check the single page vs multi page in upper readPage function call 2021-09-02 15:42:39 -07:00
Fuheng Zhao a94cb058e2 readPage and readMultiPage 2021-09-02 11:46:39 -07:00
Fuheng Zhao 16f864f2de need to debug segfault 2021-09-01 16:15:20 -07:00
Fuheng Zhao 17f2ff4c1b avoid unnecessary function calls 2021-08-31 10:41:25 -07:00
Fuheng Zhao e82706c86b change Standalone<VectorRef> to VectorRef 2021-08-09 14:49:37 -07:00
Fuheng Zhao 2aadffdb3d debugging segfault 2021-08-05 14:48:25 -07:00
Fuheng Zhao 0205f37d53 format 2021-08-03 14:19:46 -07:00
Fuheng Zhao aec0d044de support superpage withou modifying atomic update function 2021-08-03 11:17:14 -07:00
Steve Atherton c16b73bb2f Bug fix, BTreeCursor pager event reason was not being initialized. Changed metric column order. 2021-07-21 01:28:25 -07:00
Steve Atherton 01af706228 Refactor metric logging to be shorter and changed text format to be more condensed. 2021-07-20 21:23:50 -07:00
Steve Atherton 39eb71f54a Corrections in BTreeCursor event reasons, fixed by BTreeCursor now storing reason as a member because it's much simpler. Some other clang-format changes and a couple of renames. 2021-07-18 03:02:10 -07:00
Steve Atherton 09a8561606 Merge branch 'master' of github.com:apple/foundationdb into redwood-io-priority 
# Conflicts:
#	fdbserver/IPager.h
#	fdbserver/VersionedBTree.actor.cpp
 2021-07-18 01:49:36 -07:00
Steve Atherton 89140b0281 Fixed PagerEventReason for BTree writePages(). Renamed enums to start with capital letters to match one of the two styles used throughput the code base (CamelCase or ALLCAPS). 2021-07-17 03:44:00 -07:00