be37a7c01d
Added format versioning to COWPager page, header, BTreePage, BTree meta record. Added height to BTree pages.
2019-09-27 21:46:24 -07:00
b19ef86ab9
Pager2 interface now supports getting a read snapshot at a version and setting the oldest readable version. FIFOQueue now supports pushFront() which is needed for the BTree's incremental tree deletion process.
2019-09-27 21:46:24 -07:00
5384cf8f9c
Bug fixes in FIFOQueue. Read cursor would not start loading pages again after its end was pushed forward. Queue flushing of the free list queue would leave tail cursor in a bad state.
2019-09-27 21:46:24 -07:00
1bb323fa8c
Bug fix in FIFOQueue pop() when freeing an exhausted page causes a recursive pop() from the same queue, which happens when the queue is the freelist itself and the write cursor is also at the end of its page.
2019-09-27 21:46:24 -07:00
1882b58d21
COWPager dispose() was not deleting the page file.
2019-09-27 21:46:24 -07:00
8d2d1f4f24
Bug fix, COWPager recovery can't simulate header read failure using buggify anymore because the backup header is now a previous version and it is invalid to not recover with an fsync'd latest header. Debug output improvements.
2019-09-27 21:46:24 -07:00
61054492b6
Bug fix in the design of the COWPager commit sequence. Page 1 is now used to store a copy of the previous committed header rather than the new one, as recovering to an unsync'd new header from Page 1 is incorrect behavior since other pager writes may not have made it to disk. Also fixed header page size handling which would write unusable backup headers when using >4k pages.
2019-09-27 21:46:24 -07:00
ca11845934
Debug output tweaks.
2019-09-27 21:46:24 -07:00
65ddae1373
Bug fix, ObjectCache could evict the object it just added and then return an invalid reference to it.
2019-09-27 21:46:24 -07:00
95c8004049
Bug fixes. COWPager header recovery was using the wrong checksum input and did not work for physical page sizes other than 4k.
2019-09-27 21:46:24 -07:00
8c0b9b5111
COWPager now uses Page 1 as a write-ahead copy of the header which is written and sync'd before modifying Page 0.
2019-09-27 21:46:24 -07:00
af14bfc255
Changed COWPager page cache size argument to bytes instead of pages and changed initialization to use appropriate knobs in simulation.
2019-09-27 21:46:24 -07:00
537b8dc7ac
Bug fix, COWPager failed to reopen a created but unsync'd pager file. Added proper checksum error handling.
2019-09-27 21:46:24 -07:00
57f55c1e99
Bug fix - FIFOQueue design changed to not rely on the durability of unchanging bytes in modified pages that are not fsync'd.
2019-09-27 21:46:23 -07:00
e0873e2ba0
Removed COWPager snapshot lifetime management for now as it's the wrong strategy and causes crashes when snapshot references outlive the pager.
2019-09-27 21:46:23 -07:00
abc22d2610
COWPager bug fixes involving shut down while operations are in progress.
2019-09-27 21:46:23 -07:00
046dd76d73
Cleaned up the Redwood BTree correctness test, tweaked the parameters to favor shorter tests and added a time limit.
2019-08-08 23:08:08 -07:00
f81eeea495
Bug fixes. COWPager initialization was not flushing non-header pages with fsync() before writing and syncing the header. FIFOQueue was writing the initial page of a new queue multiple times. FIFOQueue::writePage() would unnecessarily (and invalidly) attempt to write if the page is not yet loaded.
2019-08-08 02:57:23 -07:00
bd8ed07f4d
Missing header.
2019-08-07 04:31:11 -07:00
10be4b8127
Merge branch 'master' of github.com:apple/foundationdb into feature-redwood
...
# Conflicts:
# fdbserver/VersionedBTree.actor.cpp
2019-08-07 04:20:14 -07:00
a04e3cce23
Bug fixes in COWPager cache. Write while a read is outstanding is supported, the new write will wait on the prior read. New writes also wait on old writes. Cache entries no longer cancel in-progress operations when evicted.
2019-08-07 04:11:33 -07:00
da9c4e97d3
Added new pager interface, IPager2, whose write interface enables forcing the user to handle a page update causing a copy to a new Page ID. Implemented FIFOQueue<T> which uses pages of T stored in a Pager2 instance to implement a FIFO queue. Implemented COWPager, a copy-on-write Pager2 in which all page writes cause a change of Page ID. VersionedBTree, still only operating in single-version mode, now uses Pager2.
2019-08-07 02:36:33 -07:00
46dc9db879
Merge pull request #1963 from etschannen/master
...
Code cleanup and bug fixes
2019-08-06 18:43:09 -07:00
c9fa7237f1
Merge branch 'master' of github.com:apple/foundationdb
2019-08-06 16:40:14 -07:00
f08704557a
Merge pull request #1959 from alexmiller-apple/fix-fdbcli-help
...
Reword consistencycheck and remove snapshot.
2019-08-06 16:38:52 -07:00
7022dabcdd
Merge pull request #1960 from mpilman/remove-objser-flag
...
Remove --object-serializer flag from executables
2019-08-06 16:37:38 -07:00
9382a58390
fix: after a forced recovery it is possible to not have logs from all generations, so only wait at most a second for getting a popped txs version
2019-08-06 16:32:28 -07:00
b0480edd15
fix: messageVersion could be larger than poppedVersion, and we will discard messages that are needed
2019-08-06 16:31:05 -07:00
ba54508c47
code cleanup
2019-08-06 16:30:30 -07:00
f453c12f0f
remove object serialization flag from TestRunner
2019-08-06 10:20:01 -07:00
370ba8b841
Remove --object-serializer flag from executables
2019-08-06 09:25:40 -07:00
aaf72c903d
Reword consistencycheck and remove snapshot.
...
`consistencycheck` help text made it sound like it would cause
consistency checking to be done, which was not the case. Consistency
checking still requires dedicated `-r consistencycheck` processes to be
running.
Snapshotting requires documentation and a bit more work, cleanup, and
polish before it could be used by users without great confusion. Thus,
it's being shuffled over to a hidden command, until that work is done.
2019-08-05 19:10:00 -07:00
cf95a7b2d7
Merge pull request #1957 from etschannen/feature-txs-popped
...
the master checks the popped version of the txsTag before recovering the txnStateStore
2019-08-05 17:23:38 -07:00
4c9a392f05
the master checks the popped version of the txsTag before recovering the txnStateStore, to avoid restoring data that is later found to be popped
2019-08-05 17:01:48 -07:00
1c730baedc
Merge pull request #1956 from etschannen/master
...
Minor bug fixes
2019-08-05 17:00:51 -07:00
5dc4c80d44
fix: the machineAttrition workload did not ensure that healthyZone was always cleared
...
fix: an assert could trigger spuriously
2019-08-05 15:00:17 -07:00
9923472636
Added missing quote for correctness command string
2019-08-01 13:27:06 -07:00
476641a087
Merge pull request #1929 from jzhou77/fix-warning
...
Fix compiler warnings
2019-08-01 11:15:41 -07:00
79b4b1fff1
Merge pull request #1952 from ajbeamon/fix-spacing-and-rename
...
Rename fdbrpc/Stats.h and fix spacing issue within it
2019-08-01 11:13:53 -07:00
37450be706
Fix format usage for currentProtocolVersion
...
ProtocolVersion now is a class.
2019-08-01 10:19:46 -07:00
2e0e5a27bb
Use invoke_result_t for all
2019-08-01 09:56:49 -07:00
863204a29d
Update names in CMakeLists, vcxproj
2019-08-01 08:48:25 -07:00
e0736232d4
Rename file in comment header
2019-08-01 08:40:45 -07:00
e61cac4ed4
Fix spacing issue; rename fdbrpc/Stats.h to fdbrpc/TimedRequest.h
2019-08-01 08:39:52 -07:00
fd124236b2
Merge branch 'master' of github.com:apple/foundationdb
2019-07-31 19:57:15 -07:00
18194172e7
Merge pull request #1951 from mpilman/bugfix/request-time-fix
...
Two minor bug fixes from recent optimizations
2019-07-31 19:56:41 -07:00
0e474ed47e
Update fdbrpc/Stats.h
...
Co-Authored-By: Evan Tschannen <36455792+etschannen@users.noreply.github.com>
2019-07-31 19:56:21 -07:00
7d247af500
Two minor bug fixes from recent optimizations
2019-07-31 19:14:11 -07:00
653d9be6e2
we cannot pop old generations because it breaks forced recoveries
2019-07-31 18:27:36 -07:00
1ea3ce8f9c
txs pops also go to the old generations of tlogs to reduce the chance we have to restart txnStateStore recovery
2019-07-31 18:06:39 -07:00
Stephen Atherton