wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,792 Commits 35 Branches 312 Tags 238 MiB
Commit Graph

360 Commits

Author SHA1 Message Date
RenxuanW ba60f18ebf Merge remote-tracking branch 'upstream/master' into backup-agent 2021-04-20 22:54:47 -07:00
Trevor Clinkenbeard 81fbe9ceaa
Merge pull request #4684 from sfc-gh-satherton/restore-range-fixes 
Restore target range handling bug fixes
 2021-04-20 11:57:13 -07:00
Steve Atherton 75425b5a24
Merge pull request #4620 from RenxuanW/renxuan/first-pr 
Control backup's initial snapshot interval via backup cmd argument.
 2021-04-19 23:59:02 -07:00
Steve Atherton 3f54a4a6dc Throw an error if an empty range set is passed to restore(). 2021-04-19 21:52:38 -07:00
Steve Atherton c2c9ca4362 Assert was incorrect. Restore ranges must begin with the restore prefix to remove. 2021-04-19 17:01:20 -07:00
RenxuanW ab4c5ff90e For better readability 2021-04-19 14:06:50 -07:00
RenxuanW 03c031a09d Update getCurrentVersion_impl 
- If the restore is in the running state, then the current version is the getApplyBeginVersion()
- If the restore is in the completed state, the current version is the restore target version which comes from the restoreVersion() property.
- If the restore is in any other state, the current version can be reported as -1 as you have done.
 2021-04-19 13:43:51 -07:00
RenxuanW 0378dc0a50 Report the current version in the restore status. 2021-04-14 22:19:39 -07:00
RenxuanW 97b995fb4f
Update fdbclient/FileBackupAgent.actor.cpp 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2021-04-14 13:49:59 -07:00
RenxuanW a285d6019e We cannot put 2 Future functions in the same wait if the second one uses the first's result. 
Before this change:
20210414-180825-renxuan-7451fad7aed4f0c7           compressed=True data_size=22960315 duration=732 ended=146 fail=10 fail_fast=10 max_runs=100000 pass=46 priority=100 remaining=0 runtime=0:01:12 sanity=False started=147 stopped=20210414-180937 submitted=20210414-180825 timeout=5400 username=renxuan

After this change:
20210414-192849-renxuan-cbe0f71ad5c48286           compressed=True data_size=22959419 duration=4261266 ended=106778 fail=1 fail_fast=10 max_runs=100000 pass=99999 priority=100 remaining=0 runtime=0:24:49 sanity=False started=106963 stopped=20210414-195338 submitted=20210414-192849 timeout=5400 username=renxuan
 2021-04-14 13:10:56 -07:00
RenxuanW 9737212e51 The default value of the first snapshot interval should be 0 rather than -1. 2021-04-14 10:56:42 -07:00
RenxuanW a0430536f1 Remove knob BACKUP_INIT_SNAPSHOT_INTERVAL_SEC. 2021-04-14 10:41:41 -07:00
RenxuanW ebf37594f7 Change initialSnapshotIntervalSeconds from knob to a backup argument. 2021-04-13 19:22:13 -07:00
RenxuanW c8b27e71c5 Revert TraceEvent 
We've found the problem (issue #4640), so we no longer need the TraceEvent.
 2021-04-12 15:03:47 -07:00
RenxuanW dc00d99626 Log FileBackupLogRangeStart before calling getLogRanges() in . 
It will tell us if or why this function is legitimately trying to use too much ram. getLogRange() should normally return about 20 items in the result. If the inputs are trash, it could return far more.

If it isn’t the case, then there’s something else wrong that has corrupted something such that when we try to allocate memory.
 2021-04-09 15:02:45 -07:00
RenxuanW fadc9cccee Use knob RESTORE_IGNORE_LOG_FILES in restore. 
Rename IGNORE_LOG_FILES to RESTORE_IGNORE_LOG_FILES. Also, this knob should be used in regular restore, not parallel restore.
 2021-04-07 10:05:56 -07:00
RenxuanW 1b6ad42db8 Use a knob to completely ignore log files 2021-04-06 19:07:01 -07:00
RenxuanW edb3dd4414 Control backup's initial snapshot interval via knob. 2021-04-05 18:25:18 -07:00
Jingyu Zhou 2b4744ad8a Reword comments. 2021-03-29 13:43:43 -07:00
Jingyu Zhou 55ef40864f Add comments for touched functions 2021-03-29 10:09:07 -07:00
Jingyu Zhou 1a0ef39e5d Fix keyRangesFilter with logsOnly flag 
Remove the Sev 40 trace event.
 2021-03-27 21:48:27 -07:00
Jingyu Zhou a9d8806828 Use the restored range in the actual restore 2021-03-25 10:07:25 -07:00
Steve Atherton 720c0b3a4f
Added const. 
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
 2021-03-24 14:36:55 -07:00
Steve Atherton a6136ffc62 Applied clang-format to recent changes. 2021-03-24 06:51:08 -07:00
Steve Atherton 17a19021fb Added mutation batch filtering during restore, to the extent possible where each restore mutation log task only sees one block of data (usually 1MB) which can contain incomplete and therefore unfilterable mutation sets. 2021-03-24 06:38:50 -07:00
A.J. Beamon 3751ecd11f
Merge pull request #4503 from sfc-gh-ajbeamon/merge-release-6.3-into-master 
Final merge of release-6.3 into master
 2021-03-16 09:29:05 -07:00
A.J. Beamon 25c4880ebe Merge branch 'release-6.3' into merge-release-6.3-into-master (temporarily discard all changes to BackupContainer.actor.cpp) 
# Conflicts:
#	fdbclient/BackupContainer.actor.cpp
#	fdbserver/Knobs.h
 2021-03-15 16:41:04 -07:00
Trevor Clinkenbeard 766b9d24d6
Merge branch 'master' into remove-invalid-backup-status-fields 2021-03-14 18:57:03 -07:00
Markus Pilman cfbaaa6001 Merge remote-tracking branch 'origin/release-6.2' into release-6.3 2021-03-10 14:30:47 -07:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
sfc-gh-tclinkenbeard f81d4eb6d3 Remove invalid backup status fields 2021-03-08 15:48:46 -08:00
Steve Atherton 318b862aa3 Apply clang-format to backup changes. 2021-03-08 01:49:29 -08:00
Steve Atherton 8a4aca3f47 Bug fix: In simulation only, RangeFileWriter was not waiting on a write future in one location. 2021-03-07 22:19:04 -08:00
FDB Formatster 8a8c488ede apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-05 18:13:38 -06:00
A.J. Beamon aaf0a9aa7b Merge branch 'release-6.3' into merge-release-6.3-into-master 
# Conflicts:
#	build/docker-compose.yaml
#	cmake/ConfigureCompiler.cmake
#	fdbclient/FileBackupAgent.actor.cpp
#	fdbrpc/AsyncFileCached.actor.h
#	fdbrpc/IAsyncFile.h
#	fdbrpc/IRateControl.h
#	fdbrpc/simulator.h
#	fdbserver/KeyValueStoreSQLite.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbservice/ServiceBase.cpp
 2021-02-08 12:58:34 -08:00
Christophe Chevalier 3fc8870ce3 winbuild: fix duplicate definition of Codec<ERestoreState>::pack and unpack in Debug build 
- defined twice in FileBackupAgent.actor.cpp and BackupAgent.actor.h
- only fails when building in Debug ?
 2021-01-30 19:03:25 +01:00
Andrew Noyes cec050a1f8 Apply clang-tidy's modernize-use-override fix 2021-01-26 02:04:12 +00:00
sfc-gh-tclinkenbeard 4669f837fa Add uses of makeReference 2020-11-07 22:10:18 -08:00
Jon Fu 51db9a7e0a add static method to access backup pause key instead of constructing it manually 2020-11-06 14:03:29 -05:00
sfc-gh-tclinkenbeard c1d88b39ab Merge remote-tracking branch 'origin/master' into add-upgrade-and-backup-restore-test 2020-10-17 11:35:12 -07:00
Andrew Noyes a1e868a569
Merge pull request #3862 from sfc-gh-tclinkenbeard/use-override-more 
Add uses of override keyword, remove unnecessary uses of virtual
 2020-10-14 15:06:45 -07:00
sfc-gh-tclinkenbeard ccf2ca73f3 Fix Windows CMake build 2020-10-08 23:58:58 -07:00
sfc-gh-tclinkenbeard a9607bdcec Explicitly seal classes that inherit but aren't inherited from 2020-10-07 21:58:24 -07:00
sfc-gh-tclinkenbeard a3de8c333b Use override where applicable in fdbclient 2020-10-07 16:57:38 -07:00
Jon Fu 7218586f2f adjust test spec and backup describe 2020-10-06 17:03:09 -04:00
sfc-gh-tclinkenbeard 43ac7f21a4 Mark toString methods const 2020-10-04 23:27:10 -07:00
sfc-gh-tclinkenbeard 8ae0679112 Remove unnecessary separation of declaration and definition of version field 2020-10-04 23:27:05 -07:00
sfc-gh-tclinkenbeard 6326de052b Remove unused LogFileWriter::FFs 2020-10-02 17:03:08 -07:00
Jon Fu 8da8c6ff0d Merge branch 'master' of https://github.com/apple/foundationdb into jfu-snapshot-record-version 2020-09-30 14:25:29 -04:00
Jon Fu 44cd3b0999 add stopBackup to incrementalBackup workload 2020-09-30 14:24:52 -04:00