c193bc8cb5
Revert parseTime() back to using strptime() for Linux and MacOS.
2019-03-20 02:39:49 -07:00
827107469f
Fixed typo in attempt to get tm.tm_gmtoff to exist on Windows.
2019-03-20 02:19:36 -07:00
bb58114ab1
Attempt to get tm.tm_gmtoff to be defined on Windows.
2019-03-20 01:53:50 -07:00
3eadcf835b
Added missing header.
2019-03-20 01:36:25 -07:00
d5e50e6963
Rewrote BackupAgentBase::parseTime() to use std::get_time() so it compiles on all supported platforms. Added unit test for parseTime() and formatTime().
2019-03-20 01:18:37 -07:00
2d7b48dadc
Merge pull request #1311 from etschannen/feature-increase-grv-batch
...
Increased the GRV client batch size
2019-03-19 08:23:05 -07:00
7f4adcc338
Merge pull request #1314 from etschannen/feature-ssd-memory-spill
...
configure memory now selects the ssd engine for transaction log spilling
2019-03-19 08:22:22 -07:00
10a30fed42
Merge pull request #1316 from senthil-ram/atomicreplace
...
atomicReplace to preserve owner & permission info
2019-03-19 08:21:35 -07:00
fea18e7be0
fix: fdbserver segfault when started with wrong arguments
...
Public address is required for roles FDBD, NetworkTestServer and
Restore only. Therefore, check those cases, and for others follow the
earlier behaviour of using default ip address 0.
FIXES #1305
2019-03-19 02:05:11 -07:00
ddf8e86730
Merge pull request #1275 from alexmiller-apple/tstlog9
...
Spill-By-Reference TLog Part 5: TLogs That Can Spill Large Amounts Of Data
2019-03-18 17:58:21 -07:00
2554fed965
reduce max transaction to start
2019-03-18 16:16:03 -07:00
87e2a1a029
The proxy budget is implemented to let one request over its limit through, and then pay back what was over the limit in the next update
2019-03-18 16:09:57 -07:00
33a211c834
Merge pull request #1315 from etschannen/feature-memory-logging
...
Occasionally log a backtrace when the fast allocators allocates a new magazine
2019-03-18 16:05:20 -07:00
b11ecb3210
Remove random bits of code that were either unneeded or leftover from debugging.
2019-03-18 15:47:20 -07:00
fe18596c01
Add a comment about ReplaceFile API
2019-03-18 15:24:08 -07:00
eb54a700ba
changed the old memory configuration to memory-1
2019-03-18 15:10:04 -07:00
37ea71b117
Implement limiting how many bytes recovery will read.
...
This time, track what location in the DiskQueue has been spilled in
persistent state, and then feed it back into the disk queue before
recovery.
This also introduces an ASSERT that recovery only reads exactly the
bytes that it needs to have in memory.
2019-03-18 15:09:43 -07:00
29ab7370cd
Clear versionLocation when spilling, and pop DQ separately.
...
Popping the disk queue now requires potentially recovering the location
to which we can pop from the spilled data itself, and for each tag we
must maintain the first location with relevant data.
The previous queue we had to represent the ordering, queueOrder, was
used by spilling, and popped when a TLog had been spilled. This means
that as soon as a TLog has been fully spilled, we have no idea how it
relates in order to other fully spilled TLogs.
Instead, use queueOrder to keep track of all the TLog UIDs until they're
removed, and use spillOrder to keep track of the order only for
spilling.
2019-03-18 15:09:22 -07:00
c8122edb6b
avoid relying on 128<<10 in knobs.cpp
2019-03-18 12:40:15 -07:00
7711542b55
Update flow/FastAlloc.cpp
...
Co-Authored-By: etschannen <36455792+etschannen@users.noreply.github.com>
2019-03-18 12:34:46 -07:00
c0094830c3
addressing minor format related comments
2019-03-18 11:59:51 -07:00
ce03651bea
Merge pull request #1313 from ravilution/patch-1
...
Fixing the Tutorials Link
2019-03-18 08:54:35 -07:00
51a655d9c7
atomicReplace to preserve owner & permission info
...
atomicReplace currently does not preserve the
ownership and permission info, this fix will
preserves the ownership and permission info - it
fails the call if it is not able to do so.
2019-03-18 00:18:05 -07:00
fa5d929106
switch to g_nondeterministic_random to make simulation deterministic
2019-03-17 22:48:43 -07:00
d2eb7578fd
Occasionally log a backtrace when the fast allocators allocates a new magazine to help track down memory leaks
2019-03-17 21:59:56 -07:00
6161144788
Fixing the Tutorials Link
...
Fixing the Tutorials link to point to GO version of the tutorials
2019-03-17 12:21:03 -05:00
44e25e219c
do not suppress KeyValueStoreMemory_OutOfSpace in simulation
2019-03-17 00:35:48 -07:00
a372c7cf18
configure memory now selects the ssd engine for transaction log spilling. Transaction log spilling is only used when the transaction logs cannot keep all of the unpopped mutations it has in memory. If we are only using this data structure because we do not have enough memory, it is much less safe to use the memory storage engine for this purpose.
2019-03-16 22:48:24 -07:00
ec6c843124
increased the GRV client batch size, similarly increased the proxy limits related to the number of transactions started in a batch
2019-03-16 16:18:58 -07:00
471ac206a3
Add new boost preprocessor definitions to windows build in release mode, still just for flow project to see if it's sufficient.
2019-03-15 21:03:03 -07:00
2c8224f0b7
Add new boost preprocessor definitions to windows build.
2019-03-15 21:03:03 -07:00
913588d6f1
Update cmake/ConfigureCompiler.cmake
...
Co-Authored-By: satherton <stevea@apple.com>
2019-03-15 21:03:03 -07:00
7f8475cf27
Removed flow/boost.cpp and the copied boost files it was using, added boost macros to makefile and cmake stuff to hopefully enable building without them.
2019-03-15 21:03:03 -07:00
9f713a6b3d
Testing to see if builds work without the contents of the boost source copies.
2019-03-15 21:03:03 -07:00
5407c87179
Updated two files copied from boost 1.52.0 to their 1.67.0 equivalents since the default boost version to compile against is now 1.67.0 and compilation (on MacOS at least) fails against boost 1.52.0 because some definitions have moved since.
2019-03-15 21:03:03 -07:00
7f5bc2981f
Checksum DiskQueue pages on read, but at a lower priority.
...
If a server has its data spilled, then it's behind the 5s window.
Feeding it data is less important than committing, so we can hide the
extra CPU usage from checksumming the read amplified disk queue pages.
2019-03-15 21:01:19 -07:00
ee4721a63f
Make checking or ignoring checksums part of the IDiskQueue::read API.
2019-03-15 21:01:18 -07:00
81c59e88a8
Persist the protocol version of a TLog instance when it is created.
...
This allows us to do easy upgrades of SpilledData in the future, if the
need arises, because we then have a protocol version to compare against.
2019-03-15 21:01:17 -07:00
bf247eeed0
If TLogVersion >= 3, use crc32c for the DiskQueue hash for TLogs.
...
We don't have a forward compatibility story for the memory storage
engine, so its DiskQueue will still be hashlittle2 until one exists.
2019-03-15 21:01:16 -07:00
686b097397
Remove verification code from DiskQueue and TLogServer.
2019-03-15 21:01:15 -07:00
bdd7d5d3df
Initialize firstPages with 0xFF.
...
There's various ASSERT()'s that assume firstPages is empty, and enforces
things about `seq`. Some of these asserts have spuriously passed, since
uninitialized pages look like they have a `seq` of 0, which would be the
beginning of the disk queue.
Now they'll look like the end of the disk queue, which is far easier to
fail on.
2019-03-15 21:01:14 -07:00
77f596743f
Bump persistFormat in TLogServer to differ from OldTLogServer*
...
Though this format is being deprecated in favor of an eventual plumbing
through of TLogVersion, we should probably bump it anyway.
And also remove the fallback to OldTLogServer code. It should never be
executed, as OldTLogServer_6_0 is entirely relied upon to execute
OldTLogServer_4_6.
2019-03-15 21:01:13 -07:00
4f98634f59
Add LogId to all TLog TraceEvents that have it.
2019-03-15 21:01:12 -07:00
0010d500c5
Merge pull request #1303 from ajbeamon/fix-compiler-warnings
...
Fix various compiler warnings
2019-03-15 17:58:49 -07:00
f5ba5bf4ca
Merge pull request #1304 from amouehsan/amouehsan-python3.7-documentation
...
Update python3.7 documentation
2019-03-15 11:12:35 -07:00
40293d6fae
Update documentation/sphinx/source/api-python.rst
...
Co-Authored-By: amouehsan <mehsan@apple.com>
2019-03-15 11:05:37 -07:00
b5d2debda7
Merge pull request #1298 from ajbeamon/fix-ryow-onerror-reset
...
RYOW onError no longer resets a transaction when it gets a non-retrya…
2019-03-15 10:46:28 -07:00
19d27b6271
Merge branch 'master' into fix-compiler-warnings
2019-03-15 10:35:36 -07:00
85b3f11e71
Fix various compiler warnings
2019-03-15 10:34:57 -07:00
9ed41a4932
Update python version to 3.7 in the docs
...
This change updates the documentation to reflect that we support python 3.7.
2019-03-15 10:24:33 -07:00
Stephen Atherton