Commit Graph

23432 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard 950ac1c867 Improve encapsulation for TLogQueueInfo and StorageQueueInfo 2022-10-27 21:18:35 -07:00
Josh Slocum 4d3553481f
Blob connection provider test (#8478)
* Refactoring test blob metadata creation

* Implementing BlobConnectionProviderTest

* createRandomTestBlobMetadata supports blobstore and works outside simulation
2022-10-27 10:44:06 -05:00
Vaidas Gasiunas c6adb3a98c
Building fdb_c_shim to a shared library (#8586) 2022-10-27 12:37:20 +02:00
Markus Pilman 2bf9c2f448
Merge pull request #8588 from sfc-gh-mpilman/bugfixes/fix-build-dependencies
Fix AWS SDK build and removed check for old build system
2022-10-26 12:36:08 -06:00
Dennis Zhou deeedfc3f8
Merge pull request #8537 from sfc-gh-dzhou/unblob
blob: allow purge ranges to begin and end in unblobbified regions
2022-10-26 11:11:09 -07:00
Markus Pilman 989731f7f4 Fix AWS SDK build and removed check for old build system 2022-10-26 11:48:10 -06:00
Aaron Molitor f620f391f5 make same change to Dockerfile.eks (from #8583) 2022-10-26 12:24:37 -05:00
Josh Slocum 623e6ef761
adding delay in bw forced shutdown to prevent crash races (#8552) 2022-10-26 12:22:41 -05:00
Nim Wijetunga 6f37f55917
Restore System Keys First in Backup/Restore Workloads (#8475)
* system key restore ordering

* restore system keys before regular data

* atomic restore backup fix

* change testing

* fix compile error

* fix compile issue

* fix compile issues

* Trigger Build

* only split restore if encryption is enabled

* revert knob changes

* Update fdbserver/workloads/AtomicSwitchover.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbserver/workloads/AtomicSwitchover.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbserver/workloads/BackupCorrectness.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* Update fdbserver/workloads/AtomicRestore.actor.cpp

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>

* add todo

* strengthen check

* seperate system restore for atomic restore

* address pr comments

* address pr comments

Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-10-26 09:38:27 -07:00
Josh Slocum ab6953be7d
Blob Granule read-driven compaction (#8572) 2022-10-26 09:02:50 -07:00
Aaron Molitor b8b7b46d8f update kubectl and awscli 2022-10-26 10:52:05 -05:00
Marian Dvorsky 3c5d3f7a94
Fix SpanContext for GP:getLiveCommittedVersion (#8565)
* Fix SpanContext for GP:getLiveCommittedVersion
2022-10-26 16:29:28 +02:00
Junhyun Shim 32099bfce5
Merge pull request #8564 from sfc-gh-jshim/enable-authz-benchmark-in-mako
Enable authz/TLS-enabled benchmark in mako
2022-10-26 14:55:53 +02:00
Junhyun Shim 2917598dc4 Merge remote-tracking branch 'origin/main' into enable-authz-benchmark-in-mako 2022-10-26 12:49:13 +02:00
Aaron Molitor e4116f8aee cleanup shell script, remove set -x, add more detailed logging 2022-10-25 23:23:22 -05:00
Xiaoxi Wang bb0236433c
Merge pull request #8540 from sfc-gh-xwang/feature/main/storageMetrics
Make MockStorageServer serve StorageMetrics related request
2022-10-25 17:29:21 -07:00
Xiaoxi Wang 0a5e596758 fix network failure check in unit test 2022-10-25 16:43:00 -07:00
Xiaoxi Wang 36d9de9072 change UNREACHABLE to ASSERT(false); change function name 2022-10-25 15:43:24 -07:00
Trevor Clinkenbeard 0f4fddfa17
Merge pull request #8480 from sfc-gh-tclinkenbeard/reject-tag-throttled-txns
Reject transactions that have been tag throttled too long
2022-10-25 15:34:07 -07:00
Jingyu Zhou 744c391608
Merge pull request #8539 from vishesh/cc-fail-later
Don't fail ConsistencyCheck on first mismatch
2022-10-25 15:33:11 -07:00
Jingyu Zhou 6a3ccda58c
Merge pull request #8575 from sfc-gh-mpilman/bugfixes/testharness-reporting
ignore injected errors in old fdb versions
2022-10-25 14:47:37 -07:00
sfc-gh-tclinkenbeard e8e7c873d8 Merge remote-tracking branch 'origin/main' into reject-tag-throttled-txns 2022-10-25 14:28:55 -07:00
Trevor Clinkenbeard 25f3a99b3d
Merge pull request #8568 from sfc-gh-tclinkenbeard/make-tracecounters-method
Encapsulate `CounterCollection`
2022-10-25 14:27:56 -07:00
Markus Pilman ad2888423c ignore injected errors in old fdb versions 2022-10-25 14:48:04 -06:00
Junhyun Shim ade4cb6b72 Merge remote-tracking branch 'origin/main' into enable-authz-benchmark-in-mako 2022-10-25 22:42:24 +02:00
Jingyu Zhou 1808c7091f
Merge pull request #8567 from sfc-gh-mpilman/bugfixes/testharness-reporting
fix summarize bug
2022-10-25 13:34:19 -07:00
Trevor Clinkenbeard ca0b068f2b Remove unnecessary forward declaration
Co-authored-by: Markus Pilman <markus.pilman@snowflake.com>
2022-10-25 13:26:51 -07:00
sfc-gh-tclinkenbeard 31a48f4046 Disable GLOBAL_TAG_THROTTLING by default 2022-10-25 13:10:15 -07:00
sfc-gh-tclinkenbeard f339819758 Merge remote-tracking branch 'origin/main' into reject-tag-throttled-txns 2022-10-25 11:59:35 -07:00
Xiaoxi Wang 5a8adca1f7 solve review comments: mark const; add comments; template abbreviation 2022-10-25 10:56:24 -07:00
sfc-gh-tclinkenbeard 74212eeacf Encapsulate CounterCollection 2022-10-25 10:17:15 -07:00
Markus Pilman f88b8e2351 fix summarize bug 2022-10-25 11:11:42 -06:00
Junhyun Shim 33c8a80614 Update Mako documentation for authz/TLS enablement 2022-10-25 16:13:01 +02:00
Junhyun Shim e32affc9f0 Add TLS and token file option to mako 2022-10-25 15:58:20 +02:00
Jingyu Zhou 0ae568a872
Merge pull request #8556 from jzhou77/fix
Fix stack overflows
2022-10-24 16:46:35 -07:00
Ankita Kejriwal ce733cd1a1
Merge pull request #8538 from sfc-gh-akejriwal/monitorusage
Add functionality to get tenants over storage quota and improve the relevant monitors
2022-10-24 16:28:07 -07:00
Hui Liu e2dc50d220
Merge pull request #8508 from sfc-gh-huliu/storageinterf
Implement StorageServerInterface for BlobMigrator
2022-10-24 16:10:31 -07:00
Zhe Wu 0140991d15 Rename NewPhysicalShardReason to RetryFindDstReason 2022-10-24 15:18:20 -07:00
Zhe Wu fc9295ab66 Address comments 2022-10-24 15:18:20 -07:00
Zhe Wu 22047385c4 Count the detailed reason for new physical shard creation during data move 2022-10-24 15:18:20 -07:00
Hui Liu f2289ced27 Add StorageServerInterface for BlobMigrator 2022-10-24 13:12:07 -07:00
Jingyu Zhou d672f91160
Merge pull request #8513 from sfc-gh-anoyes/anoyes/hide-idempotency-ids 2022-10-24 13:06:17 -07:00
Ankita Kejriwal 4f149b8705
Merge pull request #8545 from sfc-gh-tclinkenbeard/improve-fdbcli-quota-error
Improve error messages for `fdbcli` quota commands
2022-10-24 12:18:35 -07:00
Xiaoxi Wang db72a29c06 fix compile error after rebase 2022-10-24 11:16:23 -07:00
Jingyu Zhou a8f821e152 Fix stack overflows
The loop is transformed by actor compiler into recursions that may cause stack
overflows. Thus, I added yield() to unwind stack and refactor the parsing code
so that the subsequent files are blocked until previous ones have finished.
2022-10-24 11:13:11 -07:00
Dennis Zhou 136a325fdc blob/testing: randomly purge the whole range instead of just active 2022-10-24 11:08:04 -07:00
Dennis Zhou 070e4c133e blob/testing: remove setRange() and call (un)blobbifyRange() directly
This also fixes a few wrong setRange(true/false).
2022-10-24 11:08:04 -07:00
Dennis Zhou 474955b10d blob: allow purge as long as it doesn't start in a range 2022-10-24 11:08:04 -07:00
Dennis Zhou 48d6e725c2 blob: convert listBlobbifiedRangesActor() to take a Transaction 2022-10-24 11:08:04 -07:00
Dennis Zhou 07f1247885 blob: getBlobRanges() use transaction instead of RYWTransaction 2022-10-24 11:08:04 -07:00