f7956cfbfc
Clear backup UID from backupStartedKey when finish/abort backups
...
Clearing this key signals backup workers that backup is no longer needed. When
no backup is going on, the backup workers switch to the NOOP state.
2020-01-31 19:29:09 -08:00
19ef7f6bdb
Skip watch of backup task's started key if it's already set
...
The backup task may be restarted multiple times so the started key for the
backup task may already be set. In this case, the wait on watch should be
skipped.
2020-01-31 19:29:09 -08:00
7cf2881fe8
Fix backup worker ID and remove some fields
...
The backup worker ID was changed to be the interface ID, not the request ID.
The lastSeenVersion is replaced with minKnownCommittedVersion, which is
incremented even when there is no mutations. This also avoid the problem of
setting popVersion to be higher than the actual committed version (no harm
here though, as there are no mutations).
The "backupStartedKey" handling is also fixed to correctly handle cases when
we wait for the stopping of backups.
2020-01-31 19:29:09 -08:00
7544ff88d9
Comment out frequent TLogPop trace event
2020-01-31 19:29:09 -08:00
f8342f0884
Add keepRunning for start backup transaction
...
TaskBucket::keepRunning() needs to be called in backup transactions to be sure
that the task has not been cancelled. If so, the task is cancelled. Otherwise,
the task can continue run, causing multiple runs of the same task.
Another subtle issue is that the beginVersion is persisted on backupStartedKey.
So while reading it back from that key, we should set task's beginVersion with
the value persisted earlier.
2020-01-31 19:29:09 -08:00
5a602f58e8
Start backup with a wait on all backup workers running
...
This wait is to make sure that backup workers are already saving mutations so
that no mutations are missed. The idea is that the CLI sets a "backupStartedKey"
in the database and waits for allWorkerStarted() key of the backup to be set.
Backup workers monitor the changes to the "backupStartedKey" and start logging
mutations. Additionally, backup worker for Tag(-2,0) monitors all other workers
have started (checking their saved progress version is larger than the backup's
start version), and then sets the allWorkerStarted() key for the backup.
2020-01-31 19:29:09 -08:00
e9c7ad82cc
Comment out pseudo tag pop trace event
2020-01-31 19:29:09 -08:00
83907bd453
Backup worker allows on and off of backups
...
The monitoring loop of system key "backupStartedKey" and decides to be in one
of two modes: NOOP and backup. In the NOOP mode, the worker just pop TLogs. In
the backup mode, the worker pulls mutations from TLogs and save the mutations
into logs.
2020-01-31 19:29:09 -08:00
3ddadf0481
Merge pull request #2628 from ajbeamon/add-transaction-profiling-analyzer-tests
...
Add transaction_profiling_analyzer tests
2020-01-31 14:07:19 -08:00
ee6490c9d1
Merge pull request #2314 from mengranwo/memory-engine
...
New Radix-Tree based Memory Storage Engine
2020-01-30 16:20:13 -08:00
b58f185a14
Add transaction_profiling_analyzer tests
2020-01-30 12:02:18 -08:00
10d274b680
Merge pull request #2624 from ajbeamon/master
...
Add section to the version upgrade guide for API version 700.
2020-01-30 09:33:34 -08:00
8ffaf1d09b
Add section to the version upgrade guide for API version 700.
2020-01-30 09:30:32 -08:00
ed768944c6
Merge pull request #2565 from schedutron/optionstypo
...
Fix typo in Class Scheduling Python Example
2020-01-30 08:20:20 -08:00
761e07b58d
Merge pull request #2621 from 1u0/documentation-fix-some-typos
...
Documentation: fix some typos
2020-01-30 08:15:20 -08:00
e23c4e8b9c
Documentation: fix some typos
2020-01-30 15:01:03 +01:00
d1c823b3d7
Fix transaction retry typo for other lang tutorials
2020-01-30 19:10:08 +05:30
daa5ac9a56
Merge branch 'master' of https://github.com/apple/foundationdb into optionstypo
2020-01-30 19:03:22 +05:30
99a1a3694b
Merge pull request #2610 from Daniel-B-Smith/clang-link
...
Add link option specific to Clang
2020-01-29 18:19:04 -08:00
9f1fda554c
Formatting
2020-01-28 21:32:56 -05:00
4ecf58adff
Apply suggestions from code review
...
Co-Authored-By: Markus Pilman <markus@pilman.ch>
2020-01-28 21:32:13 -05:00
125fe0ac27
Merge pull request #2611 from alexmiller-apple/fix-the-build
...
Force setuptools to be upgraded during sphinx build
2020-01-28 15:17:05 -08:00
714850bfa4
Set an actual minimal version for setuptools
...
It looks like PEP 508 was merged into setuptools 20?
2020-01-28 15:11:55 -08:00
ec87850bfa
45.0.0 not pushed to pypi but 44.0.0 is
2020-01-28 15:06:38 -08:00
e43ee302b7
Force setuptools to be upgraded during sphinx build
...
So that setuptools understands PEP 508, which is now required for a
transitive watchdog dependency.
2020-01-28 15:02:52 -08:00
71d41d0981
Merge pull request #2609 from ajbeamon/merge-release-6.2-into-master
...
Merge release 6.2 into master
2020-01-28 14:22:13 -08:00
adc72dde43
Merge branch 'release-6.2' into merge-release-6.2-into-master
...
# Conflicts:
# documentation/sphinx/source/release-notes.rst
2020-01-28 12:16:24 -08:00
b9de844ee7
Add link option specific to Clang
2020-01-28 12:51:05 -05:00
fa51a1abc5
Merge pull request #2604 from xumengpanda/mengxu/fast-restore-valgrind-fix-PR14
...
Performant restore [14/XX Add-on]: Fix initialized field in VersionBatch struct
2020-01-27 15:22:31 -08:00
bfcdf6bc88
Merge pull request #2243 from canardleteer/go-pack-with-versionstamp
...
Add PackWithVersionstamp to Go Subpace & Directory bindings.
2020-01-27 08:34:20 -08:00
b566a0bcdf
Merge pull request #2602 from AlvinMooreSr/crc32c_update
...
Updated crc32c File Location
2020-01-27 08:09:52 -08:00
76f30e71dc
FastRestore:Init VersionBatch explicitly
...
Built-in variable may not be zero initialized by
compiler provided default constructor.
2020-01-26 13:15:45 -08:00
d03e49b4a1
Fixed the location of crc32c.h from fdbrpc to flow
2020-01-26 07:01:25 -08:00
5c325940ab
Merge pull request #2599 from etschannen/release-6.2
...
establishing 20 TLS connections in parallel is too expensive
2020-01-25 10:59:56 -08:00
1ed3ba7170
establishing 20 TLS connections in parallel is too expensive
2020-01-25 10:59:20 -08:00
8972027f43
Merge pull request #2456 from ajbeamon/document-api-version-upgrade-guide
...
Add an API version upgrade guide
2020-01-24 18:30:53 -08:00
792e9ed85d
Merge pull request #2574 from ajbeamon/use-python3-in-docpreview
...
Use python3 in docpreview make target
2020-01-24 18:20:39 -08:00
6945a6ea01
Merge pull request #2345 from zjuLcg/add-consistency-verification-in-mako-workload
...
Add consistency verification in mako workload
2020-01-24 17:07:49 -08:00
499ffdb572
add memory-radixtree-beta into database configuration test
2020-01-24 16:40:11 -08:00
d06d664ed7
Merge pull request #2149 from tapaswenipathak/ticket-2135
...
Add comments to explain functions in ReplicationUtils.cpp
2020-01-24 16:35:11 -08:00
2bc5b2cf8a
Merge pull request #2585 from Ma27/fix-glibc230-build
...
Fix build with glibc 2.30
2020-01-23 20:21:32 -08:00
3453d5563d
Merge pull request #2591 from etschannen/master
...
fix: backupWorker would crash when run outside of simulation
2020-01-23 19:08:08 -08:00
8f599e9d15
fix: backupWorker would crash when run outside of simulation
2020-01-23 19:06:39 -08:00
76e192d490
Merge pull request #2538 from alexmiller-apple/hashlittle2-to-crc32c
...
Convert more hashlittle{,2} uses to crc32c_append
2020-01-23 17:54:38 -08:00
6c0b934dda
Merge pull request #2242 from alexmiller-apple/fix-10min-stall-again
...
Fix the 10min multi-region recovery stall again
2020-01-23 17:53:02 -08:00
7557ec8ed1
revert changes in SimulatedCluster and add radixtree memory storage engine as third option
2020-01-23 17:09:35 -08:00
b2c8a4a34c
Merge pull request #2519 from xumengpanda/mengxu/fast-restore-versionBatch-fixSize-PR
...
Performant restore [14/XX]: Ensure each version-batch not exceed a configured size
2020-01-23 16:49:01 -08:00
8a065b9da4
Merge pull request #2557 from alexmiller-apple/reduce-versionstamp-conflictranges
...
Narrow the unreadable range of keys after a versionstamped key operation
2020-01-23 11:14:47 -08:00
fe0e10c312
Merge pull request #1625 from jzhou77/backup-worker
...
Create a backup role in the cluster
2020-01-23 10:06:35 -08:00
b5c1c8cdd0
Merge pull request #2560 from ajbeamon/java-bindingtester-error-retry
...
Add retry loop to Java binding tester instruction reader
2020-01-23 09:54:43 -08:00
Jingyu Zhou