Commit Graph

11569 Commits

Author SHA1 Message Date
Meng Xu a4aad591fd FR:Remove applyStagingKeysBatchLock flow lock 2020-09-23 08:47:37 -07:00
Meng Xu 6da7e56e03 FastRestore:Add more trace events 2020-09-22 19:57:19 -07:00
Meng Xu 08e243a07e Fix assert failure at addCallbackAndDelFutureRef 2020-09-22 15:41:07 -07:00
Meng Xu c16d76745d FastRestore:small fix compilation error 2020-09-22 13:36:26 -07:00
Meng Xu f8c5d85b79 FastRestore:Catch controller error and fix unhandled error occurred in updateHeartbeatTime 2020-09-22 13:32:26 -07:00
Meng Xu 17ece3d477 FastRestore:Fix FastRestoreApplierTransactionRateControl events 2020-09-22 13:25:43 -07:00
Meng Xu b4254473d7 FastRestore:Add transaction rate info tracer 2020-09-22 09:10:11 -07:00
Meng Xu 002b1bec4c FastRestore:Control write traffic at each applier
Controller assigns each applier a write rate.
Applier keeps the write-rate worth of transactions outstanding to DB.

This is to avoid heavily overloading DB while still keep enough
traffic to DB to get a good write throughput.
2020-09-22 08:14:28 -07:00
Meng Xu 6cfc1e876f
Merge pull request #3703 from yliucode/range
Extend backup container to support query restorable files by key ranges
2020-09-18 14:18:14 -07:00
Young Liu 7ac443df51 Add release notes 2020-09-17 18:17:33 -07:00
Young Liu 6032a21d30 Fix comment 2020-09-17 18:02:23 -07:00
Young Liu 60ccaa2ae9 Support finding max restorable version for key ranges when keyspace's max restorable version is unavailable 2020-09-17 17:58:58 -07:00
Young Liu d4e3e98957 Use new getRestoreSet in RestoreController 2020-09-17 17:56:36 -07:00
Young Liu 71309a79f9 Merge branch 'release-6.3' into range 2020-09-17 17:54:33 -07:00
Xin Dong ba2dc2dc73
Merge pull request #3783 from ajbeamon/update-go-subspace-range-docs
Give more detail when describing the range API on a subspace in Go
2020-09-17 14:52:50 -07:00
A.J. Beamon 507e6d0395 Merge branch 'release-6.3' into update-go-subspace-range-docs 2020-09-17 11:32:21 -07:00
Meng Xu f9e229ac1e
Merge pull request #3784 from ajbeamon/fix-switch-statement-fallthrough
Fix fall through in switch statement
2020-09-17 10:03:17 -07:00
A.J. Beamon 0c98a145b9
Fix whitespace 2020-09-17 08:09:21 -07:00
A.J. Beamon 303e1dc0c6 Fix fall through in switch statement 2020-09-17 08:07:53 -07:00
A.J. Beamon 1882410334 Give more detail when describing the range API on a subspace in Go. 2020-09-16 16:25:12 -07:00
Evan Tschannen 1fe31aa9e8
Merge pull request #3706 from yliucode/tlogs
Report missing old tlogs in recovery before storage servers are fully recovered
2020-09-16 14:07:06 -07:00
Young Liu c34bdfdfab
Merge pull request #2 from xumengpanda/yang/range-metadata-file
Review comments and integrate extended getRestoreSet API into fast restore
2020-09-15 22:31:41 -07:00
Xiaoge Su 73b7c20344
Merge pull request #3780 from xumengpanda/mengxu/atomicOpTest-PR
Add MinV2 and AddV2 test into AtomicOp workload
2020-09-15 12:47:38 -07:00
Meng Xu ed2d261266 Add MinV2 and AndV2 into AtomicOps test workload 2020-09-15 10:01:36 -07:00
Meng Xu 8224e17a08 Integrate extended getRestoreSet API into fast restore
The extended getRestoreSet provides a much smaller set of backup files for small keyrange restore;
This commit integrate it into fast restore so that fast restore does not have to filter out unneeded backup files.
2020-09-15 09:38:43 -07:00
Meng Xu f9a5b727a6 Add comments questions and TODOs 2020-09-14 16:28:01 -07:00
A.J. Beamon 792c639d99
Merge pull request #3721 from AlvinMooreSr/bindingtester
BindingTester Updates
2020-09-14 10:45:17 -07:00
Alvin Moore b060f53bab Added support for randomizing the port from 4000 to 4999 2020-09-12 00:39:36 -04:00
Alvin Moore 2d0b9fb12b Declared local status variables
Added checks to ensure that cluster died when killed via cli
Changed error message
2020-09-11 17:44:09 -04:00
Xin Dong f3f36de4f1
Merge pull request #3768 from xumengpanda/mengxu/fr-bug-fix-PR
Fast Restore: Add write conflict range on appliers to reduce resolver's load
2020-09-10 11:18:54 -07:00
Young Liu 1867ee1f5f Change cli output format 2020-09-09 22:34:36 -07:00
Meng Xu ce92f1a224 FR:Init StagingKey when created
The key field was used in various places, such as figuring out the conflict key range.
We should not leave it empty
2020-09-09 16:12:32 -07:00
Meng Xu 5c5abd7afa FastRestoreApplier:Calculate conflict range in applyStagingKeysBatch 2020-09-09 15:00:25 -07:00
Meng Xu 2febbe74ce FastRestoreApplier:Fix conflict range inverted due to invalid memory access
Rerpot error on loader and applier if not error_code_operation_cancelled error ever happens
2020-09-09 14:40:21 -07:00
Xin Dong f570b4fb3c
Merge pull request #3763 from xumengpanda/mengxu/fr-bug-fix-PR
Restore: Simulate blob error when restore read files
2020-09-09 14:33:42 -07:00
Young Liu 1615bd1a1d minor improvement 2020-09-09 13:57:26 -07:00
Meng Xu f10e9ea679 FastRestoreApplier:Add write conflict range 2020-09-09 12:12:14 -07:00
Young Liu 1155d015c9 fetch current log generation as well 2020-09-09 11:54:58 -07:00
Meng Xu e69ee87f33 Refactor blob failure to simulateBlobFailure function 2020-09-09 10:54:21 -07:00
Alvin Moore 5e06f41c59 Corrected spelling
Removed unused lines from file
2020-09-09 13:42:59 -04:00
Meng Xu c9171e15b8 Change FastRestoreApplierPrecomputeResultSkipUnexpectedBackupMutation to SevError 2020-09-09 10:42:51 -07:00
Evan Tschannen bfe691c4f4
Merge pull request #3756 from Daniel-B-Smith/rocks-autodownload
Enable the RocksDB engine by default with GCC
2020-09-09 10:28:53 -07:00
Meng Xu 5880bd9fa7 BackupContainer:Fix getSnapshotFileKeyRange_impl on blob http error 2020-09-09 08:47:43 -07:00
Meng Xu 83d1350d8d FastRestore:Handle retriable blob error 2020-09-09 07:29:17 -07:00
Meng Xu 20733ae1eb Simulate throwing blob http error immediately after read blob file 2020-09-09 06:47:19 -07:00
Meng Xu 6d58064964 Buggify _parsePartitionedLogFileOnLoader with blob http error 2020-09-09 06:46:08 -07:00
Meng Xu cffb54ba0b Buggify decodeLogFileBlock with blob http error 2020-09-09 06:44:02 -07:00
Meng Xu f3986d201d Buggify backup decodeRangeFileBlock with blob connection error 2020-09-09 06:40:41 -07:00
Meng Xu 9eeed8144e FastRestore:Fix bug in precomputeResult 2020-09-08 16:37:12 -07:00
Young Liu 69c417073b add release notes 2020-09-08 13:37:30 -07:00