Commit Graph

97 Commits

Author SHA1 Message Date
leonliao bc2c3df61f #6231 Fix wrong json path to check DB lock status while switching over dr clusters. 2022-01-24 13:37:09 -08:00
A.J. Beamon d8e161f89e Refactor NativeAPI transactions to create and pass around a reference counted state object. Watches no longer use the tranasction info object but instead use their own state. 2021-12-17 11:57:39 -08:00
FDB Formatster 2c788c233d apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-08-27 17:07:47 -07:00
Steve Atherton 2bf7b9b7a9 Remove many calls to .toString(), .printable(), and printable(...) from TraceEvent .detail() values to avoid double-escaping characters such that the same values look different in different events and can't be easily searched for. Also fixed a few warnings found by clang. 2021-08-08 00:04:54 -07:00
sfc-gh-tclinkenbeard c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Steve Atherton 507c1f11e3 Add .log() to bare TraceEvent() invocations without any .detail()s to avoid clang-tidy warning about immediate destruction of object without use. 2021-07-26 19:55:10 -07:00
sfc-gh-tclinkenbeard 3442ebd3b7 Fix more -Wreorder-ctor warnings across many files 2021-07-24 11:20:51 -07:00
Steve Atherton f596a81073 Rename ::TRUE and ::FALSE in BooleanParams to ::True and ::False so as to not conflict with the TRUE and FALSE macros provided by the Windows and MacOS SDKs. 2021-07-17 00:11:40 -07:00
Oleg Samarin 47f13ad1b0 Formatting status if the log version has not been set
Update fdbclient/DatabaseBackupAgent.actor.cpp

Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
Provide log version information to fdbdr status
2021-07-10 20:54:10 +03:00
sfc-gh-tclinkenbeard 8370d92c20 Expand use of BOOLEAN_PARAM in backup code 2021-07-04 18:30:55 -07:00
sfc-gh-tclinkenbeard cadfdc551e Use BOOLEAN_PARAM for backup agents 2021-07-04 16:27:26 -07:00
sfc-gh-tclinkenbeard 49c680b252 Use BOOLEAN_PARAM for TaskBucket 2021-07-04 13:14:25 -07:00
sfc-gh-tclinkenbeard 8cc40e3a2b Expand use of BOOLEAN_PARAM 2021-07-02 21:41:50 -07:00
sfc-gh-tclinkenbeard 5c2d7b6080 Create RangeResult type alias 2021-05-03 13:14:16 -07:00
Lukas Joswiak 66d88e5f12
Merge branch 'master' into fixes/backup-delete 2021-04-20 14:49:26 -07:00
Lukas Joswiak 8b2a72fea2 Add option to clear destination range before backup 2021-04-20 12:24:17 -07:00
Lukas Joswiak 5d0d837268 Fix version cutoff 2021-04-20 09:35:40 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes cec050a1f8 Apply clang-tidy's modernize-use-override fix 2021-01-26 02:04:12 +00:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
sfc-gh-tclinkenbeard 035708f43a Post-merge cleanup 2020-10-29 22:35:27 -07:00
sfc-gh-tclinkenbeard cf4c8e375f Merge remote-tracking branch 'origin/release-6.3' into merge 2020-10-29 22:15:41 -07:00
Jingyu Zhou e183a10c8c Add "waitForDestUID" flag for abortBackup 2020-10-20 22:55:31 -07:00
Jingyu Zhou 47a9bc9d3d Fix commit_unknown_result issue 2020-10-18 23:11:19 -07:00
Jingyu Zhou bfd3328448 Fix a race between submit and abort backup
After submit a backup, immediately abort the backup may cause a rare race
condition, which results in BackupCorrectnessLeftoverVersionKey error.

Specifically, in the StartFullBackupTaskFunc:
1st Txn sets the destUid at the source database and the 2nd Txn writes the dest
DB.

An abort can come after the 1st Txn succeeds, and clears the config rage so
that the 2nd Txn above would fail. Because 2nd Txn didn't write destUid, the
3rd Txn of abort can't read the correct source DB for latestVersionKey, which
contains the destUid value.

The fix is to let the 1st Txn of abort to wait until destUid becomes valid.
2020-10-18 23:11:15 -07:00
sfc-gh-tclinkenbeard 8ae0679112 Remove unnecessary separation of declaration and definition of version field 2020-10-04 23:27:05 -07:00
sfc-gh-tclinkenbeard d13a9d3c91 Make EBackupState an enum class 2020-09-23 15:14:09 -07:00
Jingyu Zhou 90654628fb
Merge pull request #3490 from apple/release-6.3
Merge Release 6.3
2020-07-10 12:54:37 -07:00
Oleg Samarin 4c9df78076 dstonly 2020-07-10 15:13:42 +03:00
Oleg Samarin 6d70887d68
Update fdbclient/DatabaseBackupAgent.actor.cpp
Speed up sync between Primary and DR: Event Name Correction

Co-authored-by: Jingyu Zhou <jingyuzhou@gmail.com>
2020-07-04 21:37:28 +03:00
Oleg Samarin 6c23901e61 Speed up sync between Primary and DR 2020-07-03 11:38:54 +03:00
Oleg Samarin 1832b88e3f Removed debug messages 2020-06-26 13:34:52 +03:00
Oleg Samarin d887d5dd70 Increased CopyLogRange performance 2020-06-26 13:34:35 +03:00
Evan Tschannen ced65cd30b finished explicitly versioning everything stored in the database 2020-05-22 17:14:21 -07:00
Evan Tschannen 0420b3e786 fix compile error 2020-04-29 14:05:53 -07:00
Evan Tschannen 7cebe743f9 A number of bug fixes of rare correctness errors 2020-04-29 13:50:13 -07:00
Evan Tschannen a486ec2de0 pipelined fdbdr status 2020-02-25 15:48:00 -08:00
Evan Tschannen daee15cbb5 fix: starting a DR should do the commit on the first proxy to ensure all mutations from previous backups have been flushed 2020-02-25 12:35:24 -08:00
Evan Tschannen 73ad702d14 Clients which fetch status should not disconnect from the coordinators and cluster controller between each retrieval 2020-01-22 15:41:22 -08:00
Steve Atherton 4ff058e86b Backup and DR layer status document generation now uses snapshot reads for all keys read to avoid unnecessary conflicts when read during a status update or cleanup transaction. Since many of the keys read use wrapper functions, all of the underlying functions in BackupAgentBase and its two implementations also required a snapshot mode argument. All snapshot arguments default to false to match the underlying FDB API get/getrange methods. 2019-12-19 00:29:35 -08:00
Evan Tschannen 628b4e0220 added a warning if multiple log ranges exist for the same range 2019-10-02 17:06:19 -07:00
Evan Tschannen 9ec9f41d34 backup and DR would not share mutations if they were started on different versions of FDB 2019-10-01 18:52:07 -07:00
Evan Tschannen ef01ad2ed8 optimized log range clearing to clear everything for each possible hash (256 clears) if that would be more efficient than one clear per second that has elapsed
aborting a DR without the —cleanup flag will still attempt to cleanup for 30 seconds before giving up
added a cleanup command to fdbbackup which can remove mutations from orphaned DRs which were stopped without the —cleanup flag
2019-09-27 18:32:27 -07:00
Jingyu Zhou 3a63d053e9 Address review comments for PR#1725 2019-06-20 14:06:32 -07:00
A.J. Beamon 5f55f3f613 Replace g_random and g_nondeterministic_random with functions deterministicRandom() and nondeterministicRandom() that return thread_local random number generators. Delete g_debug_random and trace_random. Allow only deterministicRandom() to be seeded, and require it to be seeded from each thread on which it is used. 2019-05-10 14:01:52 -07:00
Andrew Noyes 781b6ece77 Fix OPEN_FOR_IDE -Wunused-variable warnings
CC #1255, #1173
2019-04-16 15:28:01 -07:00
mpilman 1c16f87a4e Remove trace-calls to printable (in non-workloads) 2019-04-05 13:12:19 -07:00
Steve Atherton 8aab719c22
Merge branch 'master' into feature-backup-json 2019-03-12 18:23:16 -07:00
Stephen Atherton 023bbb566f Renamed backup state enums for clarity, added backup state names. Changed Epochs to EpochSeconds in backup JSON along with some other renaming/moving of fields, and added information about snapshot dispatch. Changed timestamp format for input/output in all backup/restore contexts to be a fully qualified time with timezone offset. Added information about the last snapshot dispatch to backup config and status (not yet populated). 2019-03-10 16:00:01 -07:00
Evan Tschannen f1897f3eb6 Merge branch 'master' into feature-metadata-version
# Conflicts:
#	fdbclient/NativeAPI.actor.cpp
2019-03-04 21:06:16 -08:00