Commit Graph

190 Commits

Author SHA1 Message Date
Oleg Samarin 4c9df78076 dstonly 2020-07-10 15:13:42 +03:00
Meng Xu c13ba7aeb4 FastRestore:Cosmetic code style improvement
Also mute some trace events
2020-06-24 21:25:37 -07:00
Meng Xu 2fcef90cbc FastRestore:Add addPrefix and removePrefix feature 2020-06-18 11:58:42 -07:00
Jingyu Zhou dc70591859 Fix fdbbackup help message 2020-06-11 17:26:21 -07:00
Evan Tschannen 3b9e9a9257
Merge pull request #3210 from ajbeamon/remove-include-version-h
Remove IncludeVersion.h
2020-05-20 14:12:23 -07:00
A.J. Beamon 4b5e94cf27
Merge pull request #3208 from apple/release-6.2
Merge release-6.2 into release-6.3
2020-05-20 13:47:48 -07:00
A.J. Beamon b589c3c940 Remove IncludeVersions.h, which is no longer needed for cmake-only builds 2020-05-20 13:23:02 -07:00
A.J. Beamon 54c97fe2bc Backup's ProgramStart command line string would be incomplete if arguments were specified using '=' 2020-05-20 09:38:33 -07:00
A.J. Beamon 59e33968c6 Remove erroneous restore options. 2020-05-20 08:57:56 -07:00
Evan Tschannen f17f00fdd5 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2020-05-10 22:33:38 -07:00
Meng Xu c237ca9708
Merge pull request #3081 from jzhou77/backup-cmd
Update fdbbackup documentation on new "-p" option
2020-05-04 14:17:34 -07:00
Jingyu Zhou 2c5da1c116 Remove "-p" option and use "--partitioned_log_experimental" option 2020-05-04 13:45:54 -07:00
Alex Miller 78d644f673
Merge branch 'master' into bugfixes/cmake-includes 2020-05-04 02:11:46 -07:00
Meng Xu 37a537c2a3 Fix MacOS compilation error 2020-04-30 18:30:15 -07:00
Steve Atherton 02aeab1edd Re-add improperly named options in order to not break compatibility with anything depending on them in a patch release. 2020-04-30 16:23:30 -07:00
Meng Xu 07a9a05683 FastRestore:Agent:Fix restore requests 2020-04-30 16:20:20 -07:00
Steve Atherton 697beb3ebb Fix fdbrestore parameters with one leading hyphen instead of two, which did not match documentation. 2020-04-30 12:13:35 -07:00
Meng Xu 8742cc0ab6 FastRestore:Fix windows build 2020-04-28 19:12:27 -07:00
Meng Xu f5e8345496 FastRestoreAgent:Use atomicParallelRestore to kick off restore
Replace the handcrafted version with atomicParallelRestore actor
which is simulation tested
2020-04-27 22:15:00 -07:00
tclinken 52860043c9 Merge remote-tracking branch 'origin' into atomically-update-dependent-knobs 2020-04-08 12:26:21 -07:00
Jingyu Zhou 9fb3fb9d82 Add pause/resume for new backups
To pause/resume the backup workers, the fdbbackup command will write to the
backupPausedKey. Then backup workers noticed the value of the key has been
changed and stops/resumes pulling from TLog.
2020-04-06 14:29:46 -07:00
Markus Pilman e4611e8ae4 fix versions.h stupidity 2020-04-06 10:28:55 -07:00
Markus Pilman 8b5780c36c don't include source and binary dir
This forces users to use include paths from the sources root.

So `#include "Arena.h"` won't work anymore, only
`#include "flow/Arena.h"` will.
2020-04-06 10:13:49 -07:00
tclinken 884e92bb49 Atomically update dependent knobs 2020-04-01 15:18:49 -07:00
Jingyu Zhou 906174e3e8 Add an option for fdbbackup to use new backup system
I.e., "-p", or "--partitioned_log" to enable it. By default, old backup system
is used.
2020-03-31 14:54:32 -07:00
Meng Xu a93f13cfd7 Remove redundant restoreRequestDone break in backup.actor 2020-03-25 15:19:46 -07:00
Meng Xu 495afe2e0b Improve how to wati for restore to finish
Remove default parameter for atomicRestore as suggested in review.
2020-03-25 13:54:21 -07:00
Evan Tschannen 303df197cf Merge branch 'release-6.2'
# Conflicts:
#	CMakeLists.txt
#	bindings/c/test/mako/mako.c
#	documentation/sphinx/source/release-notes.rst
#	fdbbackup/backup.actor.cpp
#	fdbclient/NativeAPI.actor.cpp
#	fdbclient/NativeAPI.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogRouter.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/fdbserver.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/flow.vcxproj
#	flow/flow.vcxproj.filters
#	versions.target
2020-03-06 18:22:46 -08:00
Alex Miller 595dd77ed1 Merge remote-tracking branch 'upstream/release-6.2' into certificate-refresh 2020-03-04 20:25:42 -08:00
Alex Miller 9b5ef3416e Refactor TLSParams into TLSConfig + LoadedTLSConfig
The idea being that we keep around a TLSConfig that the configuration
that the user has provided, and then when we want to intialize an SSL
context, we ask the TLSConfig to load all certificates and return us a
LoadedTLSConfig that is a concrete set of certificate bytes in memory.

initTLS now just takes the in-memory bytes and applies them to the ssl
context.

This is a large refactor to lead up into certificate refeshing, where we
will periodically check for changes to the certificates, and then
re-load them and apply them to a new SSL context.
2020-03-04 20:14:47 -08:00
Evan Tschannen cdcb816866
Update fdbbackup/backup.actor.cpp 2020-03-04 16:08:45 -08:00
A.J. Beamon 58e621eca1 Invalid knobs or knob values are treated as warnings rather than errors. Apply this change to backup as well. 2020-03-04 15:50:04 -08:00
Evan Tschannen c11c24b79d removed the fdbrpc version of platform.h 2020-02-28 14:56:10 -08:00
Evan Tschannen 96258b9809 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbcli/fdbcli.actor.cpp
#	fdbclient/ManagementAPI.actor.cpp
#	fdbrpc/FlowTransport.actor.cpp
#	fdbserver/ClusterController.actor.cpp
#	fdbserver/DataDistribution.actor.cpp
#	fdbserver/DataDistribution.actor.h
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/QuietDatabase.actor.cpp
#	fdbserver/SkipList.cpp
#	fdbserver/StorageMetrics.actor.h
#	fdbserver/TLogServer.actor.cpp
#	fdbserver/fdbserver.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/KVStoreTest.actor.cpp
#	flow/CMakeLists.txt
#	flow/Knobs.cpp
#	flow/Knobs.h
#	flow/genericactors.actor.cpp
#	flow/serialize.h
2020-02-21 19:09:16 -08:00
Alex Miller 927cff3317 Report errors on TLS misconfigurations ... or at least try to. 2020-02-20 16:57:29 -08:00
Evan Tschannen 761da5a059 code cleanup 2020-02-19 17:59:45 -08:00
Meng Xu 132f5aa9ba FastRestore:Improve trace name and cosmetic change 2020-02-18 16:41:19 -08:00
Meng Xu 72110de7e2 FastRestore:Add trace for quick perf. measurement 2020-02-06 19:48:26 -08:00
Alvin Moore 7628d04fb9 Merge branch 'release-6.2' of github.com:apple/foundationdb into release_6.2_merge
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
2020-01-09 07:21:16 -08:00
Steve Atherton 4ff058e86b Backup and DR layer status document generation now uses snapshot reads for all keys read to avoid unnecessary conflicts when read during a status update or cleanup transaction. Since many of the keys read use wrapper functions, all of the underlying functions in BackupAgentBase and its two implementations also required a snapshot mode argument. All snapshot arguments default to false to match the underlying FDB API get/getrange methods. 2019-12-19 00:29:35 -08:00
A.J. Beamon ed8d3f163c Rename hgVersion to sourceVersion. 2019-11-15 12:26:51 -08:00
Evan Tschannen 4de60fc437 Merge branch 'release-6.2'
# Conflicts:
#	documentation/sphinx/source/release-notes.rst
#	fdbserver/TLogServer.actor.cpp
2019-11-01 15:48:04 -07:00
A.J. Beamon f175ed30b3 Cleanup the fdbbackup cleanup command output. Add cleanup to the usage output printed for fdbbackup. 2019-10-31 09:52:21 -07:00
Andrew Noyes d4de608bb6 Fix OPEN_FOR_IDE build 2019-10-25 10:42:22 -07:00
Meng Xu 7599dd0093
Merge pull request #2205 from jzhou77/backup-fix
Fix format string with more portable code
2019-10-02 20:03:35 -07:00
Jingyu Zhou ceb39c0279 Fix format string with more portable code 2019-10-02 15:25:14 -07:00
Meng Xu d0147e5e5d Merge branch 'release-6.2' into mengxu/merge-release620-to-master-v3
Resolved Conflicts:
	documentation/sphinx/source/release-notes.rst
	fdbserver/DataDistribution.actor.cpp
	versions.target
2019-10-02 13:22:56 -07:00
Evan Tschannen 9463b91594
Update fdbbackup/backup.actor.cpp
Co-Authored-By: A.J. Beamon <ajbeamon@users.noreply.github.com>
2019-09-30 13:16:55 -07:00
Evan Tschannen 4d659662b8 made cleanup handle retries better 2019-09-30 12:44:20 -07:00
Evan Tschannen ef01ad2ed8 optimized log range clearing to clear everything for each possible hash (256 clears) if that would be more efficient than one clear per second that has elapsed
aborting a DR without the —cleanup flag will still attempt to cleanup for 30 seconds before giving up
added a cleanup command to fdbbackup which can remove mutations from orphaned DRs which were stopped without the —cleanup flag
2019-09-27 18:32:27 -07:00