Commit Graph

1763 Commits

Author SHA1 Message Date
Xin Dong 03287a0214 Fix build error. 2020-02-12 14:57:40 -08:00
Xin Dong 57f0c11712 Address Evan's review comments 2020-02-12 14:57:40 -08:00
Xin Dong d20ce99774 Resolved the review comment and renamed the functions 2020-02-12 14:57:40 -08:00
Xin Dong d934aed1d7 Because when the user issue 'getStorageByteSample' on a large key range, which can be as large as the whole DB, we need to change the behavior of 'waitStorageMetricsMultipleLocation' to avoid the case where a target key range got moved/splited by DD and thus the call to 'waitMetircs' on the corresponding storage server will return 'wrong_shard_server' error and thus the whole 'waitStorageMetricsMultipleLocation' will be retried on the large key range. What we want is to do the retry only for the key range that caused the error. 2020-02-12 14:57:40 -08:00
Xin Dong 807204e676 Update fdbclient/MultiVersionTransaction.actor.cpp
Apply A.J's suggestion.

Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-02-12 14:57:40 -08:00
Xin Dong d5c3f821e2 Added missing pieces. 2020-02-12 14:57:40 -08:00
Xin Dong 70f89042fd Remove comment that does not apply anymore 2020-02-12 14:57:40 -08:00
Xin Dong 0c16d43c2f Added necessary plumbings to expose byte sample collected by storage servers to fdb_c library 2020-02-12 14:57:40 -08:00
Andrew Noyes 1248d2b8b4 Remove USE_OBJECT_SERIALIZER knob 2020-02-12 10:41:52 -08:00
Evan Tschannen 38a5511b96 additional simulation fixes 2020-02-11 15:52:06 -08:00
Andrew Noyes 86089fdc1b
Merge branch 'release-6.2' into atn34/configure-locked 2020-02-11 13:51:41 -08:00
Evan Tschannen fd5eb5946e
Merge pull request #2606 from ajbeamon/options-documentation-fix
Fix database default retry limit documentation.
2020-02-11 13:29:30 -08:00
Meng Xu cda8fc189e FastRestore:AtomicOp:Intro weighted size for atomicOp
atomicOp has an amplified performance overhead to the cluster,
for example, an ADD operation can be small, but SS has to load
the value to do the operation and the value can be large.
2020-02-11 12:48:05 -08:00
Andrew Noyes 7b5de42d43 Address review comments 2020-02-11 10:40:09 -08:00
mpilman 5a9d420cb7 Merge remote-tracking branch 'upstream/release-6.2' into release-merges/20200210 2020-02-10 10:02:05 -08:00
A.J. Beamon ff44bd2b33
Merge pull request #2639 from atn34/atn34/include-port-in-address-default
Enable include_port_in_address by default for api version 700
2020-02-10 09:50:59 -08:00
Markus Pilman e71fe44ee3
Merge branch 'master' into features/icc 2020-02-08 21:33:02 -08:00
Alex Miller e390dbd36c Add a non-FDBLibTLS verify peers framework to new TLS impl 2020-02-06 21:06:52 -08:00
Evan Tschannen 38d8d0d675 fixed simulation 2020-02-06 19:29:31 -08:00
Evan Tschannen 844c8511c4
Merge pull request #2588 from jzhou77/backup-worker
Integrate new backup worker with existing backup command
2020-02-05 14:14:43 -08:00
Jingyu Zhou d5849af5c0 Address review comments 2020-02-05 10:33:51 -08:00
Andrew Noyes 90c1b2df88 Don't include header 2020-02-05 09:57:18 -08:00
Evan Tschannen 53d0867a17 limit the number of connections a process can attempt to establish in parallel 2020-02-04 18:15:10 -08:00
Meng Xu 08443ed18d FastRestore:Remove debug trace for debugging connection errors 2020-02-04 17:06:02 -08:00
Evan Tschannen c8c34333c1 increased connect parallelism 2020-02-04 14:59:20 -08:00
Evan Tschannen 84853dd1fd switched SSL implementation to use boost ssl 2020-02-04 14:56:40 -08:00
Evan Tschannen 8449badb3e
Merge pull request #1868 from dongxinEric/fix/1827/error_instead_of_timeout
Send error back before put the GRV request with PRIORITY_BATCH into t…
2020-02-04 14:32:47 -08:00
mpilman 100402aadf Don't call operator explicitely 2020-02-04 11:03:43 -08:00
mpilman 52ca752dd3 Merge remote-tracking branch 'origin/features/icc' into features/icc 2020-02-04 10:29:49 -08:00
mpilman d09e07f1f5 Merge remote-tracking branch 'upstream/master' into features/icc 2020-02-04 10:26:18 -08:00
Jingyu Zhou 52c6737411 Rename backupLoggingEnabled as backupWorkerEnabled
To highlight the changes for 7.0 backup changes. By default,
backup_worker_enabled flag is set for 7.0 version.
2020-02-04 10:09:16 -08:00
Jingyu Zhou 7c10683c77 Backup workers save logs into right containers
The mutation logs of backup workers are saved into "mlogs" directory under the
container directory. The backup worker has been restructured to handle multiple
backups, where each one is stored in a separate backup container.

In the backup worker, mutations pulled from TLogs are buffered in a message
queue. When writing out to different containers, their corresponding mutation
ranges are used to check if a mutation should be written. When a new backup
is submitted by the client, "backupStartedKey" is updated. The worker monitors
this key, updates its internal map of backups, and then next pull from TLog
needs to wait for the readiness of the new backup. This is to ensure when
worker 0 sets the backup is started, all workers have already been logging
mutations for the backup.
2020-02-03 20:27:14 -08:00
Jingyu Zhou 0db03f1d3c Use backup_logging_enabled flag
The default is to enable new backup workers. Users can disable this flag to
turn off the backup worker feature.
2020-02-03 20:03:22 -08:00
Meng Xu 3b57bf1781 Merge branch 'master' into mengxu/fast-restore-agent-PR 2020-02-03 17:23:54 -08:00
Evan Tschannen 4524831456
Merge pull request #2518 from vishesh/task/failmon-remove-server
FailureMonitoring: Server processes no longer need to talk to ClusterController
2020-02-03 17:22:50 -08:00
Meng Xu ca3b6135d0 FastRestore:Add debug to see why restore role is not connected
Reason: restore is a fdbserver who does not register with CC.
The new failure monitor changes how connection works for client and server.
For client, it does not connect to CC to get connected.
For server, it has to connect to CC to get connected.
Restore worker becomes the special role that behaves like a client but is a server.
2020-02-03 17:19:52 -08:00
Andrew Noyes a043646d1d Don't read in init database transaction 2020-02-03 15:32:01 -08:00
Andrew Noyes 61a727d701 Allow new databases to be configured as locked 2020-02-03 15:32:01 -08:00
Andrew Noyes 2ce887012c Respect api version for include_port_in_address 2020-02-03 15:25:30 -08:00
Andrew Noyes 07a3051f0e Enable include_port_in_address by default for api version 700
Resolves #2607
2020-02-03 15:10:00 -08:00
Meng Xu 9c2046b11b FastRestore:Minic fdbd to monitor coordintors
Before we start a fdb restore process.
2020-02-03 14:48:31 -08:00
Jingyu Zhou 297f22726c Add backup_type database configuration option
Update simulation tests to randomly set backup types to be one of: old backup
(default), new backup (tagged), or both (default+tagged).
2020-01-31 19:29:09 -08:00
Jingyu Zhou 38aa1903fd Add a DB configuration option for backup workers
Right now, the default is to keep the old backup behavior, i.e., do NOT use
backup workers. Specifically, if BackupType is not set (or is set to default),
the master will not recruit backup workers and will not add pseudo locality for
backup workers.

The StartFullBackupTaskFunc is updated to check if backup worker is enabled.
Only when it is not enabled, starting a backup will wait on all backup workers
to be started.
2020-01-31 19:29:09 -08:00
Jingyu Zhou f7956cfbfc Clear backup UID from backupStartedKey when finish/abort backups
Clearing this key signals backup workers that backup is no longer needed. When
no backup is going on, the backup workers switch to the NOOP state.
2020-01-31 19:29:09 -08:00
Jingyu Zhou 19ef7f6bdb Skip watch of backup task's started key if it's already set
The backup task may be restarted multiple times so the started key for the
backup task may already be set. In this case, the wait on watch should be
skipped.
2020-01-31 19:29:09 -08:00
Jingyu Zhou f8342f0884 Add keepRunning for start backup transaction
TaskBucket::keepRunning() needs to be called in backup transactions to be sure
that the task has not been cancelled. If so, the task is cancelled. Otherwise,
the task can continue run, causing multiple runs of the same task.

Another subtle issue is that the beginVersion is persisted on backupStartedKey.
So while reading it back from that key, we should set task's beginVersion with
the value persisted earlier.
2020-01-31 19:29:09 -08:00
Jingyu Zhou 5a602f58e8 Start backup with a wait on all backup workers running
This wait is to make sure that backup workers are already saving mutations so
that no mutations are missed. The idea is that the CLI sets a "backupStartedKey"
in the database and waits for allWorkerStarted() key of the backup to be set.

Backup workers monitor the changes to the "backupStartedKey" and start logging
mutations. Additionally, backup worker for Tag(-2,0) monitors all other workers
have started (checking their saved progress version is larger than the backup's
start version), and then sets the allWorkerStarted() key for the backup.
2020-01-31 19:29:09 -08:00
Jingyu Zhou e9c7ad82cc Comment out pseudo tag pop trace event 2020-01-31 19:29:09 -08:00
Xin Dong 7016f7903b Fixed another build error. Do not use timeReplyIgnoreError since we don not want the logging inside that function and thus that's unnecessary anymore. Change to use ready() which basically ignores the error. 2020-01-31 15:48:29 -08:00
Xin Dong c1f992667b Fix build failure 2020-01-31 14:27:47 -08:00
Xin Dong 8d28c2a7f0 Added two new counters for transaction throttled error and remove the verbose trace event logging. Also changed a chain of 'if' statements into 'if-else' statements since they are mutal exclusive 2020-01-31 14:16:39 -08:00
Alex Miller ee6490c9d1
Merge pull request #2314 from mengranwo/memory-engine
New Radix-Tree based Memory Storage Engine
2020-01-30 16:20:13 -08:00
Xin Dong 1b313a4f7e Address review comments. Rebased with latest master 2020-01-30 14:13:56 -08:00
Xin Dong e21426d12a Send error back to the GRV requests with batch priority when the cluster is saturated, instead of blindly enqueue the requests and let the client timeout. 2020-01-30 14:13:56 -08:00
A.J. Beamon f46090d081 Fix typo 2020-01-29 14:25:24 -08:00
Xin Dong 68f6e7be97 Added some java docs and changed some texts in fdboptions for further comparision and clarification around .asList() and .iterator() 2020-01-29 14:01:54 -08:00
A.J. Beamon adc72dde43 Merge branch 'release-6.2' into merge-release-6.2-into-master
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2020-01-28 12:16:24 -08:00
Meng Xu cab9d51e06 Merge branch 'master' into mengxu/fast-restore-pipeline-PR 2020-01-27 18:16:26 -08:00
Meng Xu 141609e80a FastRestore:Improve code style and fix typos 2020-01-27 18:13:14 -08:00
A.J. Beamon aa7acaf02c Fix transaction retry limit description. Fix typo. 2020-01-27 10:33:31 -08:00
Meng Xu 76f30e71dc FastRestore:Init VersionBatch explicitly
Built-in variable may not be zero initialized by
compiler provided default constructor.
2020-01-26 13:15:45 -08:00
Evan Tschannen 1ed3ba7170 establishing 20 TLS connections in parallel is too expensive 2020-01-25 10:59:20 -08:00
Meng Xu 16f9ec45bd Merge branch 'master' into mengxu/fast-restore-pipeline-PR 2020-01-23 20:15:21 -08:00
A.J. Beamon b2c8a4a34c
Merge pull request #2519 from xumengpanda/mengxu/fast-restore-versionBatch-fixSize-PR
Performant restore [14/XX]: Ensure each version-batch not exceed a configured size
2020-01-23 16:49:01 -08:00
A.J. Beamon 8a065b9da4
Merge pull request #2557 from alexmiller-apple/reduce-versionstamp-conflictranges
Narrow the unreadable range of keys after a versionstamped key operation
2020-01-23 11:14:47 -08:00
Jingyu Zhou 1eaea91cb3 Address review comments 2020-01-22 19:42:13 -08:00
Jingyu Zhou 1311fec45a Add an option to get minKnownCommittedVersion from Proxies
The backup worker needs to use this version for popping when running in a NOOP
mode. This option is added to GetReadVersionRequest and proxies will send back
minKnownCommittedVersion if the option is set.

Also add a couple of knobs for backup workers.
2020-01-22 19:42:13 -08:00
Jingyu Zhou c08a192c75 Add a backup start key
If the backup key is not set, do not recruit backup workers for old epoches.
2020-01-22 19:42:13 -08:00
Jingyu Zhou 1123157ae0 Ignore mutations large than the end version 2020-01-22 19:38:46 -08:00
Jingyu Zhou d8c74e7e1a Extend BackupContainer to support tagged log files
That is, the file name contains the log router tag ID as the last component,
e.g., "log,39638169,42718056,016f52a4d16ef36fd3335db9c68abfc1,1048576,1".
2020-01-22 19:38:46 -08:00
Jingyu Zhou 7f7ec99170 Serialize and deserialize new backup files
The BackupWorker writes files that can be read by FileConverter. Move
StringRefReader to the header file for reuse in FileConverter.
2020-01-22 19:38:46 -08:00
Jingyu Zhou f21d7ca44c Add tag ID to backup log file names 2020-01-22 19:38:46 -08:00
Jingyu Zhou 485d3d0feb Use Version instead of int64_t 2020-01-22 19:38:45 -08:00
Jingyu Zhou dafcaee844 Fix compiler errors. 2020-01-22 19:38:45 -08:00
Jingyu Zhou c7f51782b8 Use override for virtual functions. 2020-01-22 19:38:45 -08:00
Jingyu Zhou 19d6a889ff Recruit backup workers for old epochs
If there are unfinished ranges in the old epochs, the new master will recruit
backup workers responsible for finishing these ranges. These workers remains in
the cluster until the next epoch, when it will remove itself.
2020-01-22 19:38:45 -08:00
Jingyu Zhou 11964733b7 WIP: should be divided into smaller commits. 2020-01-22 19:38:45 -08:00
Jingyu Zhou 41f0cf2bb5 Add decode function for backup progress 2020-01-22 19:38:45 -08:00
Jingyu Zhou 7da9f47f26 Enable pop from backup workers
This is still WIP as some edge cases can trigger test failure, most likely due
to not popping mutations by backup workers when epoch ends.
2020-01-22 19:38:45 -08:00
Jingyu Zhou de8d953865 Add backup role, class, and worker skeleton 2020-01-22 19:35:30 -08:00
Evan Tschannen 73ad702d14 Clients which fetch status should not disconnect from the coordinators and cluster controller between each retrieval 2020-01-22 15:41:22 -08:00
Meng Xu 009fcdeb16 FastRestore:Sanity check each restore asset is processed exactly once 2020-01-21 17:17:45 -08:00
chaoguang 7224807d1d fix typo in var name 2020-01-21 15:38:48 -08:00
Alex Miller 914a693f3a Copy the key size check over also.
It doesn't matter in this case, as getVersionstampKeyRange would have
already passed the check, but it should be done if anyone ever calls
this function in the future.
2020-01-21 15:35:27 -08:00
Alex Miller 18e6e36a8e Fill in the versionstamp, but still leave the position.
Using range.begin() stripped off the last 4 bytes that specified where
the versionstamp is, so when the mutation got to the proxy, it wouldn't
get versionstamped correctly.
2020-01-21 15:24:57 -08:00
chaoguang 5386b4ecdf Adding prefix to returned keys, keeping consistency with query range 2020-01-21 15:18:16 -08:00
Meng Xu 4ac92d223b Cleanup batch buffer for each restore request 2020-01-21 14:49:36 -08:00
Vishesh Yadav daef5f011a Merge remote-tracking branch 'apple/master' into task/failmon-remove-server 2020-01-21 13:20:15 -08:00
Meng Xu d69bd2f661 FastRestore:Loader buffer data for multiple batches 2020-01-17 17:01:06 -08:00
Meng Xu bfbf2164c4 FastRestore:Applier buffer data for multiple batches 2020-01-17 17:01:01 -08:00
chaoguang 5b62d672dc update comments 2020-01-17 15:18:07 -08:00
chaoguang 17721a00f1 change vars names 2020-01-17 14:26:45 -08:00
chaoguang 670c3e629f Fix arena issue and change std::vector back to VectorRef 2020-01-17 14:24:13 -08:00
chaoguang d17d1c88cd add const to local KeyRef 2020-01-17 11:31:43 -08:00
Alex Miller eb64eede8d Make a smaller range inaccessable after writing a versionstamped key
A transaction's read version is the lower bound of what a transaction's
commit version could be.  Thus, we can narrow the conflict range of a
versionstamped key, and thus reduce the amount of the keyspace that is
rendered inaccessable, by filling in the read version on the
versionstamped key and using that as the lower bound of the conflict
range.

This allows reads to still be done to versionstampled keys lower than
the read version of the current transaction.
2020-01-16 21:41:59 -08:00
Evan Tschannen 3f9d9d8b84 Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	cmake/FlowCommands.cmake
#	documentation/sphinx/source/release-notes.rst
#	fdbclient/StorageServerInterface.h
#	fdbserver/DataDistributionTracker.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/Knobs.h
#	flow/Platform.cpp
#	versions.target
2020-01-16 18:37:47 -08:00
chaoguang db901d626a Update some var names and make returned list in ResolveTransactionBatchReply as Optional 2020-01-16 14:41:06 -08:00
chaoguang 1fa934837d Update some var names 2020-01-16 13:14:13 -08:00
chaoguang 30b9858cf5 using std::vector instead of VectorRef to avoid unknown arena change behavior 2020-01-16 13:06:36 -08:00
mengranwo 227edd4248 change memory storage engine name from memory-radixtree to memory-radixtree-beta 2020-01-15 13:49:45 -08:00
mengranwo f597aa7e18 WIP : deployable/stable version since Nov 3. Start rebase to master branch 2020-01-15 13:49:45 -08:00
chaoguang 72d39a31f1 Change to send back index of read_conflict_range not keys, which speeds up but fail to give narrowed keyrange 2020-01-15 10:21:51 -08:00
Evan Tschannen c93ca04ea6 Do not merge more than 100 shards together to avoid creating untrackable shards 2020-01-15 09:33:27 -08:00
chaoguang 1cf3b5b807 Change to general prefix '\xff\xff/transaction/conflicting_keys/' 2020-01-13 20:54:16 -08:00
chaoguang 1c2b116688 Push sorting and union of overlapped keyranges to clients 2020-01-13 18:08:54 -08:00
Meng Xu f436ea806e FastRestore:Resolve review comment
1) Sort logfiles by endVersion

2) Exit program early when restore will not succeed

3) Do not increase nextVersion unncessarily when
calculate version batches.

4) Change assert condition that ensures progress in
calculating version batches.
2020-01-13 14:08:27 -08:00
Evan Tschannen 8475da359c
Merge pull request #2527 from etschannen/feature-region-fixes
A database could perform poorly while a remote region catches up to the primary
2020-01-10 17:26:43 -08:00
Evan Tschannen 855f03a41f ratekeeper needed to check remoteDC in another location
the storage server scoped a transaction incorrectly
2020-01-10 15:58:36 -08:00
chaoguang 1f633d3e5f change local var from Key to KeyRef 2020-01-10 15:53:06 -08:00
Evan Tschannen 16bf3dbba3
Merge pull request #2512 from etschannen/feature-kill-ping
Improve the reliability of the kills from fdbcli
2020-01-10 13:03:49 -08:00
Evan Tschannen c6087add51
Update fdbclient/Knobs.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2020-01-10 12:51:37 -08:00
Evan Tschannen 4aab9b7bc8 fix: clients would waste time attempting to read from a remote region when it was in the process of catching up 2020-01-10 12:23:59 -08:00
Evan Tschannen da1be272cb fix: servers which opened the database would use the full list of proxies 2020-01-10 12:20:30 -08:00
chaoguang 1cdb22c4a4 refine code according to comments 2020-01-09 15:42:21 -08:00
Alvin Moore 7628d04fb9 Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2020-01-09 07:21:16 -08:00
Meng Xu a2b26906e8 FastRestore:Filter out empty files before distributing workload
and clean up unused code
2020-01-07 17:01:53 -08:00
Vishesh Yadav 6e6cfaff16 Cleanup old Failure Monitoring code 2020-01-07 15:53:32 -08:00
Vishesh Yadav 85c24dc074 Active Failure Monitoring no longer needed at server processes
This patch removes active failure monitoring at server processes.
Hence like client processes, servers no longer require continuously
publishing their membership to cluster controller.

When a process is marked as failed, we still need to know if it back
up at certain point, particularly when the reference count is
incremented. In that case, loadBalance may see AllAlternativesFailed
as failed. To overcome this problem, whenever peer references is
incremented and and the address is marked as failed, connectionKeeper
will bypass waiting for data and connects immediately to check if the
process is back up.
2020-01-07 15:53:32 -08:00
chaoguang 941ee6d9f9 update comments 2020-01-07 15:39:50 -08:00
Meng Xu c29e380076 FastRestore:Remove prevVersion from LoadingParam 2020-01-07 14:59:17 -08:00
chaoguang b8ffc72cca Use a RYW in transactionInfo to track conflicting keys 2020-01-07 13:55:05 -08:00
chaoguang 57fb28af2c Explicitly set read-version and clear the whole keyspace to make sure the getRange happens locally 2020-01-07 13:54:33 -08:00
Meng Xu 9df02512ab FastRestore:Apply clang-format 2020-01-07 11:50:32 -08:00
Meng Xu 67e913c3d5 Change LoadingParam struct and endVersion definition
1) Remove endVersion field because it has been included in RestoreAsset;

2) Ensure endVersion in VersionBatch and RestoreAsset is always exclusive;

3) Revise ASSERT in laoder and applier in situations when the dummy commit version
is endVersion, to avoid false positive ASSERT failure.
2020-01-07 11:48:03 -08:00
Meng Xu c3f8f3b445 FastRestore:Build VersionBatch less than threshold size 2020-01-07 11:46:56 -08:00
chaoguang 10719200c3 A hack way to call API through getRange("\xff\xff/conflicting_keys\<start_key>", "\xff\xff/conflicting_keys\<end_key>"). 2020-01-06 11:22:11 -08:00
Evan Tschannen 9a3dfec7c5 open a connection with processes before attempting to kill them to improve the reliability of the kill process
secondaryAddresses are included in the list of processes which can be killed
2020-01-03 16:10:44 -08:00
Meng Xu 8d6f511816 FastRestore:Resolve review comment
Filter out range mutations that do not overlap with the restore range.
Small changes on format.
2019-12-22 20:09:10 -08:00
Meng Xu 61b29de3ce FastRestore:Self code review
Clean up commented code;
Add sanity check.
2019-12-20 22:24:34 -08:00
Meng Xu ddcf3fdd80 FastRestore:Apply clang format 2019-12-20 22:00:36 -08:00
Meng Xu e98b2a0d1c FastRestore:Introduce RestoreAsset 2019-12-20 18:00:10 -08:00
Evan Tschannen fca0e74655
Merge pull request #2418 from satherton/blobstore-api-enhancements
BlobStoreEndpoint enhancements
2019-12-19 11:23:37 -08:00
Steve Atherton 4ff058e86b Backup and DR layer status document generation now uses snapshot reads for all keys read to avoid unnecessary conflicts when read during a status update or cleanup transaction. Since many of the keys read use wrapper functions, all of the underlying functions in BackupAgentBase and its two implementations also required a snapshot mode argument. All snapshot arguments default to false to match the underlying FDB API get/getrange methods. 2019-12-19 00:29:35 -08:00
Andrew Noyes cc4e57f72c Use consistent "> 0" style for guarding memcpy 2019-12-16 08:40:35 -08:00
Jingyu Zhou ded2a301e0
Merge pull request #2443 from xumengpanda/mengxu/fast-restore-fix-valgrind-PR
Performant restore [12/XX]: Code clean up
2019-12-13 14:35:20 -08:00
Meng Xu 97030d9168 FastRestore:Revise and test SevFRMutationInfo
Enabled SevFRMutationInfo for valgrind test, no error found, and disable it again.
Revise debug trace message a bit.
2019-12-13 13:51:21 -08:00
Alvin Moore 3bf971ba8b Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/storageserver.actor.cpp
2019-12-12 07:13:12 -08:00
Meng Xu 1371db4cdc FastRestore:Self code review and cleanup
1. Review memory use cases and improve:
Ensure state varialble is initialized and
change unnecessary  state variable to variable.

2. Remove debug code that is no longer useful;

3. Mute verbose debug.
2019-12-11 16:37:33 -08:00
Steve Atherton 9a031bfc47 Function was renamed. 2019-12-11 11:00:12 -08:00
Meng Xu 9a6dabe47e Merge branch 'mengxu/fastrestore-code-cleanup-PR' into mengxu/fast-restore-fix-valgrind-PR 2019-12-10 20:05:35 -08:00
Meng Xu feb2a8c70c FastRestore Change RestoreSendMutationVectorVersionedRequest name
Change RestoreSendMutationVectorVersionedRequest to
RestoreSendVersionedMutationsRequest for better naming
2019-12-10 17:23:40 -08:00
Meng Xu 20a19978f9 FastRestore:LoadingParam cleanup 2019-12-10 17:20:44 -08:00
A.J. Beamon 290cbb7628 Fix an internal error that occurred when using client buggify and a successful transaction was failed with commit_unknown_result.
Move some buggify errors into a try catch block so that the normal error handling path can run.
Initialize startTime so that it's valid in the case of errors thrown early.
2019-12-09 09:44:48 -08:00
Meng Xu e8dfc1c187 Replace pop_front(size) with new empty standalone obj 2019-12-06 23:16:49 -08:00
chaoguang b376b947d5 Encode conflicting keys using base64. 2019-12-06 16:21:19 -08:00
chaoguang eacfdf6da3 Add a simple workload, ReportConflictingKeysWorkload, to test correctness of the API and performance overhead added to the resovler. 2019-12-06 16:21:03 -08:00
Stephen Atherton 09e8d804e8 Added BlobStoreEndpoint::listBuckets(), renamed listBucket() and several related functions with similar names to listObjects() to avoid confusion and closer match what it actually does. Added a bytesDeleted output statistic to BlobStoreEndpoint::deleteRecursively. 2019-12-06 00:14:13 -08:00
Meng Xu c6b36dbffb FastRestore:Sampling:Resolve review comments 2019-12-04 17:35:11 -08:00
Meng Xu 2b987d1945 FastRestore:typedef Standalone<VectorRef<MutationRef>> MutationsVec 2019-12-04 11:39:55 -08:00
Andrew Noyes c8e8d0c8ab Fix UBSAN report
/home/anoyes/workspace/foundationdb/fdbclient/SnapshotCache.h:36:9: runtime error: null pointer passed as argument 2, which is declared to never be null
2019-12-04 09:42:35 -08:00