wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,393 Commits 37 Branches 304 Tags 268 MiB
Commit Graph

9393 Commits

Author SHA1 Message Date
Xin Dong 8e489e7031
Merge pull request #2870 from dongxinEric/feature/2399/provide-comprehensive-info-about-exclude-in-fdbcli 
Instead of providing partial information about exclude results in dif…
 2020-03-31 11:58:41 -07:00
Xin Dong 18356ec4e6
Apply suggestions from code review 
Co-Authored-By: John Brownlee <mail@johnbrownlee.com>
 2020-03-31 11:11:24 -07:00
A.J. Beamon 09a5629114
Merge pull request #2883 from tclinken/add-getversion-command 
Added fdbcli getversion command
 2020-03-31 07:56:15 -07:00
Meng Xu 42df1e7792
Merge pull request #2879 from jzhou77/backup-progress 
Update mutation bytes written for new backups
 2020-03-30 21:42:45 -07:00
Meng Xu a85652375c
Merge pull request #2872 from jzhou77/backup-fix 
Switch off old mutation logging on proxies for new backups
 2020-03-30 21:42:10 -07:00
Meng Xu 60f6edc3b5
Merge pull request #2860 from zjuLcg/report-conflicting-key-roll-forward 
Report conflicting key roll forward
 2020-03-30 17:33:56 -07:00
tclinken bee6acb717 Added fdbcli getversion command 2020-03-30 17:12:52 -07:00
Jingyu Zhou 3c76722504 Fix valgrind error of invalid memory access 2020-03-30 16:38:22 -07:00
Jingyu Zhou 3c32835cce Fix decoder for unfinished version batch in a log 
A mutation log's version batch data can be split into multiple blocks, and some
of the blocks can be spread across two mutation logs. Thus, the decoder needs
copy unfinished version batch data from previous file progress to the next one.
 2020-03-30 11:34:51 -07:00
Alvin Moore f92ef730a4
Merge pull request #2876 from mpilman/bugfixes/cmake-env-set 
Introduce `set_env` function
 2020-03-30 02:13:53 -07:00
Jingyu Zhou 411b4c28ac Update mutation bytes written for new backups 
This will make the log bytes written available to backup status and describe
backup calls.
 2020-03-29 21:23:34 -07:00
Jingyu Zhou 65e3b9192e Add an assert for probably dead code 2020-03-28 21:19:47 -07:00
Jingyu Zhou 241f9c123e Try to find continuous log ranges that cover snapshot begin version 
Backup container can have mutation log files that are not continuous overall,
but contain a continuous range that cover the snapshots. So when determine the
continuous log ranges, try to find one that cover the first snapshot's begin
version.
 2020-03-28 21:19:47 -07:00
Jingyu Zhou 280bc94738 Do not recruit backup workers with wrong tags 
In a rare scenario, the master can recruit backup workers with more tags than
the number of log router tags for an epoch. This can be caused by an
unsuccessful recovery, which uses more tags than the next epoch. When
recruiting for the next epoch, if no progress has been made yet, the recruiting
logic will look back at the previous epoch. If previous epoch has saved past
this epoch's begin version, current epoch's progress is updated with that
information and can result in more tags being inserted to this epoch's
recruitment.
 2020-03-28 21:19:41 -07:00
Jingyu Zhou 3261eae8c1 Initialize contiguousLogEnd to the lowest possible version 
If set to the first file's end version, then we may erroneously set to an
unrestorable version.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou 34745d71fa Fix continuous end version 
When scan from a known version, stop if logs are not continuous. However, if
scan from 0, we should reset minLogBegin to the next file and continue scan
from that file's begin version.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou abcb458a44 Fix contiguousLogEnd calculation 
The contiguous version should scan from the max of scanBegin version and the
minLogBegin version. Once we found a version that's larger, set it as the
contiguousLogEnd version.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou efbd01894d Really fix the stuck of backup 2020-03-28 17:38:02 -07:00
Jingyu Zhou f0d0c863e6 Attempt to fix 2020-03-28 17:38:02 -07:00
Jingyu Zhou 78f74ed6fe Skip copying inside BackupLogsDispatchTask 2020-03-28 17:38:02 -07:00
Jingyu Zhou a2b8be5dc6 Switch off old mutation logging on proxies for new backups 
Specifically, stop writing LogRange mutation for new backup jobs.
 2020-03-28 17:38:02 -07:00
Jingyu Zhou cf7404a771
Merge pull request #2871 from xumengpanda/mengxu/parallel-range-log-file-loading-PR 
Performant restore [24/xx]: Process range and log files in parallel for old backup format
 2020-03-28 17:33:50 -07:00
Meng Xu 92998d3df9 Add restore with old backup tests into CMakeLists 2020-03-28 16:08:22 -07:00
Meng Xu 13f343ec96 Resolve minor review comment 2020-03-28 16:03:01 -07:00
Meng Xu 6e60c62d7b Add ParallelRestoreCorrectnessAtomicOp.txt 2020-03-28 16:00:51 -07:00
Meng Xu 8a30526336 FastRestore:Remove commented assertion 2020-03-28 13:11:32 -07:00
Meng Xu 404a3e2619 FastRestore:Loader:Remove sanity chech for the order of sending log and range mutations 2020-03-27 23:36:13 -07:00
Meng Xu 21a5c67f9a FastRestore:Remove assertion on mutation sending order 2020-03-27 17:09:31 -07:00
Meng Xu 75fc9af5c8 Apply clang format 2020-03-27 16:55:52 -07:00
Meng Xu 0222e8096c FastRestore:Send log mutations and range mutations in parallel 
With the subversion extension, appliers can order log and range mutations
based on LogMessageVersion instead of sending order.
 2020-03-27 16:54:19 -07:00
Markus Pilman 95fc5ae4ec Introduce `set_env` function 
This makes it easier to handle environment variables in cmake.
It also reduces the code needed to check whether variables are correct
 2020-03-27 15:44:55 -07:00
Alvin Moore 137ea71203
Merge pull request #2865 from mpilman/features/build-test-harness 
Build all joshua packages with cmake
 2020-03-27 13:11:39 -07:00
Meng Xu f7233bade7 Rename ParallelRestoreCorrectnessAtomicOpTinyData.txt by removing TinyData 2020-03-27 13:08:59 -07:00
Meng Xu 5c6752fb24 Add tests for new restore with old backup 2020-03-27 13:07:38 -07:00
Meng Xu 97f8e46388 Sanity check subversion for log mutations 2020-03-27 13:07:08 -07:00
Meng Xu 32b0ba1822 Merge branch 'master' into mengxu/parallel-range-log-file-loading-PR 2020-03-27 12:13:47 -07:00
Meng Xu 113d0fb48b Remove incorrect assertion 2020-03-27 12:13:30 -07:00
Markus Pilman 644e0ff30b Use full version string in package names 2020-03-27 09:47:37 -07:00
Markus Pilman 0f224ead7a fixed formatting 2020-03-27 09:38:20 -07:00
Markus Pilman c5191f5c51
Update cmake/AddFdbTest.cmake 
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
 2020-03-27 09:36:57 -07:00
Markus Pilman ce318c18fa
Update cmake/AddFdbTest.cmake 
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
 2020-03-27 09:36:46 -07:00
Markus Pilman 893a38008c
Update cmake/AddFdbTest.cmake 
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
 2020-03-27 09:36:38 -07:00
Markus Pilman 55e5d5b4d9
Update cmake/AddFdbTest.cmake 
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
 2020-03-27 09:36:30 -07:00
Markus Pilman 94e653b6d8
Update contrib/Joshua/scripts/bindingTestScript.sh 
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
 2020-03-27 09:36:21 -07:00
Markus Pilman def7f39328
Update cmake/AddFdbTest.cmake 
Co-Authored-By: Alvin Moore <36203359+AlvinMooreSr@users.noreply.github.com>
 2020-03-27 09:36:08 -07:00
Meng Xu 01d58341aa
Merge pull request #2867 from jzhou77/backup-fix 
Add an option for fast restore workload to restore old backups
 2020-03-26 17:01:04 -07:00
Markus Pilman ac9f79116e Build bindingtester 2020-03-26 16:01:58 -07:00
chaoguang 64148469e8 clang-format the pr 2020-03-26 15:52:30 -07:00
Jingyu Zhou 96a7b4b459 Disable ParallelRestoreApiCorrectnessAtomicRestore test 2020-03-26 14:59:50 -07:00
chaoguang 50f9360400 Add release-notes for report-conflicting-keys 2020-03-26 14:54:22 -07:00