Commit Graph

3342 Commits

Author SHA1 Message Date
RenxuanW 41ca11c3e5 Implement restoring an inconsistent snapshot as a real feature. 2021-04-22 13:53:37 -07:00
Steve Atherton 4ff714efbd
Merge pull request #4691 from RenxuanW/first-consistent-version
Add first consistent version in restore status.
2021-04-21 17:46:13 -07:00
RenxuanW bc43fa99ac Move commit to its own try loop. 2021-04-21 17:37:58 -07:00
Andrew Noyes 8a00c6cdf8 Add -Wshift-sign-overflow
This catches the bug fixed in #4656 at compile time
2021-04-21 23:49:26 +00:00
RenxuanW b90f61d740 Move commit to its own try loop. 2021-04-21 15:50:25 -07:00
RenxuanW 7c4b5b0337 Add first consistent version in restore status.
First consistent version is the max of versions in RestoreFileSet.
2021-04-21 14:32:13 -07:00
A.J. Beamon b9d8302ac7
Merge pull request #4540 from dlambrig/batch
Log latency metrics for batch GRV requests
2021-04-21 10:23:37 -07:00
RenxuanW ba60f18ebf Merge remote-tracking branch 'upstream/master' into backup-agent 2021-04-20 22:54:47 -07:00
Lukas Joswiak 66d88e5f12
Merge branch 'master' into fixes/backup-delete 2021-04-20 14:49:26 -07:00
Lukas Joswiak 8b2a72fea2 Add option to clear destination range before backup 2021-04-20 12:24:17 -07:00
Trevor Clinkenbeard 81fbe9ceaa
Merge pull request #4684 from sfc-gh-satherton/restore-range-fixes
Restore target range handling bug fixes
2021-04-20 11:57:13 -07:00
Evan Tschannen 83097c92cf
Merge pull request #4602 from sfc-gh-ljoswiak/fixes/applied-version
Fix early cutoff when DR copies logs
2021-04-20 11:10:30 -07:00
Lukas Joswiak 38d780e847 Add buggify 2021-04-20 09:35:40 -07:00
Lukas Joswiak 5d0d837268 Fix version cutoff 2021-04-20 09:35:40 -07:00
Steve Atherton 75425b5a24
Merge pull request #4620 from RenxuanW/renxuan/first-pr
Control backup's initial snapshot interval via backup cmd argument.
2021-04-19 23:59:02 -07:00
Steve Atherton 3f54a4a6dc Throw an error if an empty range set is passed to restore(). 2021-04-19 21:52:38 -07:00
Steve Atherton c2c9ca4362 Assert was incorrect. Restore ranges must begin with the restore prefix to remove. 2021-04-19 17:01:20 -07:00
RenxuanW 491fc6d69e Merge remote-tracking branch 'upstream/master' into backup-agent 2021-04-19 14:34:27 -07:00
RenxuanW ab4c5ff90e For better readability 2021-04-19 14:06:50 -07:00
RenxuanW 03c031a09d Update getCurrentVersion_impl
- If the restore is in the running state, then the current version is the getApplyBeginVersion()
- If the restore is in the completed state, the current version is the restore target version which comes from the restoreVersion() property.
- If the restore is in any other state, the current version can be reported as -1 as you have done.
2021-04-19 13:43:51 -07:00
Markus Pilman 863e262302
Merge pull request #4581 from sfc-gh-anoyes/anoyes/improve-ryw-disable-doc
Document that ryw disable can only be set at beginning of transaction
2021-04-19 14:38:06 -06:00
Lukas Joswiak bb5539bb70 Initialize version field 2021-04-16 14:08:36 -07:00
RenxuanW 0378dc0a50 Report the current version in the restore status. 2021-04-14 22:19:39 -07:00
Markus Pilman d51f94f16e
Merge pull request #4330 from sfc-gh-ljoswiak/features/global-configuration
Add global configuration framework implementation
2021-04-14 16:10:08 -06:00
RenxuanW 97b995fb4f
Update fdbclient/FileBackupAgent.actor.cpp
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2021-04-14 13:49:59 -07:00
RenxuanW a285d6019e We cannot put 2 Future functions in the same wait if the second one uses the first's result.
Before this change:
20210414-180825-renxuan-7451fad7aed4f0c7           compressed=True data_size=22960315 duration=732 ended=146 fail=10 fail_fast=10 max_runs=100000 pass=46 priority=100 remaining=0 runtime=0:01:12 sanity=False started=147 stopped=20210414-180937 submitted=20210414-180825 timeout=5400 username=renxuan

After this change:
20210414-192849-renxuan-cbe0f71ad5c48286           compressed=True data_size=22959419 duration=4261266 ended=106778 fail=1 fail_fast=10 max_runs=100000 pass=99999 priority=100 remaining=0 runtime=0:24:49 sanity=False started=106963 stopped=20210414-195338 submitted=20210414-192849 timeout=5400 username=renxuan
2021-04-14 13:10:56 -07:00
A.J. Beamon 3ed0d614d2 Move fdb_get_server_protocol to be a function on the database object. Add an argument for expected_version that can be used to signal that the function shouldn't return unless the protocol version is different. 2021-04-14 12:50:30 -07:00
RenxuanW 9737212e51 The default value of the first snapshot interval should be 0 rather than -1. 2021-04-14 10:56:42 -07:00
Lukas Joswiak 51e4c19675 Add migration for client profiling keys 2021-04-14 10:56:33 -07:00
Lukas Joswiak 2594d91f11 Update casing 2021-04-14 10:56:33 -07:00
Lukas Joswiak 7de23918c0 Add comments, fix erase bug, make optimizations 2021-04-14 10:56:33 -07:00
Lukas Joswiak c38ddf5eb7 Add comments 2021-04-14 10:56:33 -07:00
Lukas Joswiak 7ba7257cd2 Store global config data on heap 2021-04-14 10:56:33 -07:00
Lukas Joswiak 1c60653c2a Add fix to conditionally set global config history 2021-04-14 10:56:33 -07:00
Lukas Joswiak aa0014ab6e Fix version serialization 2021-04-14 10:56:33 -07:00
Lukas Joswiak 6de28dd916 clang-format 2021-04-14 10:56:33 -07:00
Lukas Joswiak 1260385965 Use object to wrap global configuration history 2021-04-14 10:56:32 -07:00
Lukas Joswiak 1c84c04ffc Add global configuration prefix function 2021-04-14 10:56:32 -07:00
Lukas Joswiak 388344c31e Better estimation for arena size 2021-04-14 10:56:32 -07:00
Lukas Joswiak b7cd8175be Add arena per object in global config 2021-04-14 10:56:32 -07:00
Lukas Joswiak e5e48da5ce Revert removal of history size check 2021-04-14 10:56:32 -07:00
Lukas Joswiak e9e2ca54d6 Assert history contains data 2021-04-14 10:56:32 -07:00
Lukas Joswiak 70c4bbe119 Fix clear range persistence issue 2021-04-14 10:56:32 -07:00
Lukas Joswiak 4a799baa1d Add clear range for global configuration 2021-04-14 10:56:32 -07:00
Lukas Joswiak c3f68831af Move existing ClientDBInfo variables to global configuration 2021-04-14 10:56:32 -07:00
Lukas Joswiak 80c6048a01 Naming fixes 2021-04-14 10:56:32 -07:00
Lukas Joswiak 9587318696 Fix crash when history size is 0
This shouldn't happen in normal operation (if ClientDBInfo has been
updated, that means at least one item should have been added to the
history). But there is old functionality that uses other ClientDBInfo
fields to send updates to all nodes, and until this functionality is
removed this check needs to be here.
2021-04-14 10:56:32 -07:00
Lukas Joswiak 2acefa2c82 Add double and float support to tuples
Note that this functionality is copied from bindings/flow/Tuple.cpp.
These classes should eventually be combined (see #4351).
2021-04-14 10:56:32 -07:00
Lukas Joswiak 96732810ff Move actor implementation 2021-04-14 10:56:32 -07:00
Lukas Joswiak c9b0d3dd4e Fix memory leak
The map containing global configuration data had keys of type StringRef,
referencing data allocated in history arenas. When the old history
was deleted, this memory was no longer valid and some keys would point
to garbage memory.
2021-04-14 10:56:32 -07:00