Commit Graph

17 Commits

Author SHA1 Message Date
mpilman ea67b742c7 Implemented Traceable for printable types 2019-04-05 13:12:19 -07:00
Alex Miller 37ea71b117 Implement limiting how many bytes recovery will read.
This time, track what location in the DiskQueue has been spilled in
persistent state, and then feed it back into the disk queue before
recovery.

This also introduces an ASSERT that recovery only reads exactly the
bytes that it needs to have in memory.
2019-03-18 15:09:43 -07:00
Alex Miller 29ab7370cd Clear versionLocation when spilling, and pop DQ separately.
Popping the disk queue now requires potentially recovering the location
to which we can pop from the spilled data itself, and for each tag we
must maintain the first location with relevant data.

The previous queue we had to represent the ordering, queueOrder, was
used by spilling, and popped when a TLog had been spilled.  This means
that as soon as a TLog has been fully spilled, we have no idea how it
relates in order to other fully spilled TLogs.

Instead, use queueOrder to keep track of all the TLog UIDs until they're
removed, and use spillOrder to keep track of the order only for
spilling.
2019-03-18 15:09:22 -07:00
Alex Miller ee4721a63f Make checking or ignoring checksums part of the IDiskQueue::read API. 2019-03-15 21:01:18 -07:00
Alex Miller bf247eeed0 If TLogVersion >= 3, use crc32c for the DiskQueue hash for TLogs.
We don't have a forward compatibility story for the memory storage
engine, so its DiskQueue will still be hashlittle2 until one exists.
2019-03-15 21:01:16 -07:00
Alex Miller aff9ebe21a Spill (start,length) instead of (begin,end) to save a few bytes. 2019-03-04 01:42:38 -08:00
Alex Miller 232f2434a5 Allow IDiskQueue::location to be serialized. 2019-02-26 18:00:55 -08:00
Alex Miller ee8964c8ec Plumb through getNext{Commit,Push}Location 2019-02-26 18:00:55 -08:00
Alex Miller 12123f41d6 Plumb a read function up the stack to IDiskQueue 2019-02-12 23:44:13 -08:00
Alex Miller 8b21d1ac8f Add a standalone recovery initialization function. 2019-02-12 23:44:13 -08:00
Robert Escriva 268093a96d Adjust all includes to be relative to the root.
Remove the use of relative paths.  A header at foo/bar.h could be included by
files under foo/ with "bar.h", but would be included everywhere else as
"foo/bar.h".  Adjust so that every include references such a header with the
latter form.

Signed-off-by: Robert Escriva <rescriva@dropbox.com>
2018-10-19 17:35:33 +00:00
Stephen Atherton 2752a28611 Merge branch 'release-5.2' of github.com:apple/foundationdb into feature-redwood 2018-04-06 16:29:37 -07:00
Alec Grieser 0bae9880f1 remove trailing whitespace from our copyright headers ; fixed formatting of python setup.py 2018-02-21 10:25:11 -08:00
Stephen Atherton 0a35f167e4 Merge branch 'master' into feature-redwood
# Conflicts:
#	fdbserver/DiskQueue.actor.cpp
#	fdbserver/IDiskQueue.h
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/fdbserver.vcxproj
#	fdbserver/fdbserver.vcxproj.filters
#	fdbserver/worker.actor.cpp
2018-02-12 01:30:02 -08:00
Evan Tschannen 49dac11a5f added a SevWarnAlways for when a disk queue file grows larger than 20GB 2017-12-01 15:05:17 -08:00
Stephen Atherton 248dab79b6 Created “redwood” storage engine option and many changes to support that including IKeyValueStore::init() and custom DiskQueue file extensions. 2017-09-21 23:51:55 -07:00
FDB Dev Team a674cb4ef4 Initial repository commit 2017-05-25 13:48:44 -07:00