Commit Graph

2626 Commits

Author SHA1 Message Date
He Liu b52edd8658 Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency 2022-10-10 11:00:05 -07:00
He Liu 1931e3266f Resolve comments. 2022-10-10 10:28:19 -07:00
Steve Atherton b7ce834d28 Merge commit '7c89cd705faee52d5d78e6c77665cb7cc4502f58' into storageserver-pml 2022-10-07 11:39:42 -07:00
Markus Pilman ea1325a552
Merge pull request #8319 from sfc-gh-tclinkenbeard/add-rare-code-probe-annotation
Add `rare` code probe decoration
2022-10-07 09:39:00 -06:00
Steve Atherton 3228afefd3 Unrevert #7578 - storage server PriorityMultiLock and PML rewrite. 2022-10-06 23:41:28 -07:00
A.J. Beamon f8186988bc
Merge pull request #8378 from sfc-gh-ajbeamon/tenant-support-for-database-switching
Add tenant support for database switching
2022-10-05 14:57:33 -07:00
Sreenath Bodagala e83229ed8b
- Version vector specific: Propagate the latest commit version, as (#8393)
part of the read request, when reading from a storage server.
2022-10-05 16:23:56 -04:00
Jon Fu 2fe1d19e95 Merge branch 'main' of github.com:apple/foundationdb into metacluster-status 2022-10-05 11:48:04 -07:00
Josh Slocum 8592f7e253
Added test check to verify change feeds get cleaned up at the end of blob granule tests (#8322)
* implemented check, but it doesn't always work

* cleanup
2022-10-05 12:58:05 -05:00
Markus Pilman 45a31884d5 Address review comments 2022-10-05 09:30:25 -06:00
Markus Pilman 67076aae4c Fix compilation errors 2022-10-05 07:52:54 -06:00
Markus Pilman 550488b020 Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide
# Conflicts:
#	bindings/c/CMakeLists.txt
#	fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
#	fdbserver/BackupWorker.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/CommitProxyServer.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/PhysicalShardMove.actor.cpp
#	flow/CMakeLists.txt
2022-10-04 18:27:48 -06:00
Markus Pilman 97dfc6823f fixed build with OPEN_FOR_IDE 2022-10-04 17:01:02 -06:00
A.J. Beamon 55b880432e
Merge pull request #8379 from sfc-gh-ajbeamon/literal-string-ref-removal
LiteralStringRef removal
2022-10-04 09:41:42 -07:00
A.J. Beamon 53593c77ac When updating watches after a database switch, lookup the tenant again to get the ID on the new cluster. Update the switching test to set up tenants properly. 2022-10-03 09:24:13 -07:00
sfc-gh-tclinkenbeard fec791be62 Merge remote-tracking branch 'origin/main' into split-failure-injection-workloads 2022-09-30 18:00:48 -07:00
A.J. Beamon e1fe28b78b Switch some usages of LiteralStringRef to use the _sr suffix 2022-09-30 16:04:16 -07:00
Ata E Husain Bohra 442ed0de10
Update TenantEntryCache workload refresh to handle fault injection (#8369)
* Update TenantEntryCache workload refresh to handle fault injection

Description

 diff-1: Add trace event to log wait

Patch update TenantEntryCache::testCacheRefresh to better handle fault
injection causing longer cluster recovery/availability delays. The
test is modified to loop for a threshold time waiting for next round
of tenantCache refresh cycle to happen. Otherwise throws 'timed_out'
error.

Testing

devRunCorrectness - 100K
2022-09-30 15:48:40 -07:00
Nim Wijetunga 90ce4053f4
Repair Dead Data Center Outside Quiet Database (#8359)
* fix snap test

* repair dead data center
2022-09-30 13:59:02 -07:00
Ankita Kejriwal 999809e79b Increase the data in the workload and add ASSERTs for expected values. 2022-09-29 19:21:44 -07:00
Ankita Kejriwal 5a3cd51255 Remove temporary TraceEvents and comments from previous commits 2022-09-29 18:53:11 -07:00
Ankita Kejriwal c2b6b288b7 Fixes in the GetEstimatedRangeSize workload
* Run the workload on a trusted client rather than untrusted
  clients. This allows the workload to be substantially simplified
  as well as enables testing for the case where no tenant is
  present.
* Explicitly pass tenant to BulkSetup so that the setup phase
  can be run in parallel for multiple tenants without causing a race.
2022-09-29 18:34:47 -07:00
Ankita Kejriwal de90e1aab1 Improve the code in the actors and server, simplify the workload 2022-09-29 18:34:35 -07:00
Ankita Kejriwal f63934117d Make the storage metrics function tenant aware [WIP]
This change makes the getEstimatedRangeSizeBytes function tenant aware.
Previously, this function would return the size of the requested
keyspace even if the tenant in the Transaction or DatabaseContext did
not match the tenant corresponding to the keyspace.

Also make some improvements to the new workload.
2022-09-29 18:32:04 -07:00
Ankita Kejriwal 85ba47f4c8 Add a workload to test getEstimatedRangeSizeBytes funtionality 2022-09-29 18:28:39 -07:00
Jon Fu 461e42bfe1 restructure updater code and add capacity check in metacluster management workload 2022-09-29 16:24:02 -07:00
Ata E Husain Bohra 03f1d13be3
Enable encryption authentication configurability (#8312)
* Enable encryption authentication configurability

Description

 diff-1: Remove memcpy due to auth-token computation
         Address review comments

Patch proposes major changes:
1. Enable FDB to choose encryption authentication as a configurable
parameter. Fix issues choosing ENCRYPT_HEADER_AUTH_TOKEN_NONE mode.
2. Introduce AES_CMAC as supported encryption authentication scheme.

Patch allows cluster to govern: if encryption authentication needs to
enabled, if yes, then choose from two supported schemes:
1. HMAC_SHA_256
2. AES_256_CMAC

Testing

devRunCorrectness - 100K
BlobCipher unittests
EncryptionOps.toml
BlobGranuleCorrectness/BlobGranuleCorrectnessClean
2022-09-29 16:18:55 -07:00
Nim Wijetunga 232e23c87e
Snapshot Backup Encryption (#8095)
* inital commit

* add encryption support

* refactor passing encryption keys

* disable encryption codepath

* pass enable encryption cli arg for fdbbackup

* change encryption key passing

* only enable encryption for backup if tenant mode is required

* revert server knobs

* ignore json trace files

* --amend

* integrate ekp

* add comments

* comments

* comments

* revert knobs

* fix bug

* modify encrypted backup logic

* use common prefix

* check boundaries between begin and end key

* return system encrypt domain

* modify

* add tenant entry cache to backup

* slight revert

* format

* Trigger Build

* format

* use tenant name to fetch keys

* Trigger Build

* Trigger Build

* merge

* fix getting ekp

* fix bug

* address pr comments

* Trigger Build

* Trigger Build

* remove faulty check
2022-09-29 14:45:47 -07:00
sfc-gh-tclinkenbeard 2434f18c5c Enable failure injection for all simulation tests 2022-09-29 14:14:24 -07:00
A.J. Beamon 93d08b55ba
Merge pull request #8350 from sfc-gh-ajbeamon/fix-metacluster-workload-heap-use-after-free
Fix heap use after free in metacluster management workload
2022-09-29 13:38:32 -07:00
A.J. Beamon 5d075d3aff Fix heap use after free 2022-09-29 12:34:36 -07:00
Xiaoxi Wang 3578832c69 merge upstream/main 2022-09-28 14:01:50 -07:00
A.J. Beamon c8bb15e8ee
Merge pull request #8216 from sfc-gh-ajbeamon/backup-support-for-tenants
Include tenant and metacluster metadata in backups
2022-09-28 12:00:24 -07:00
Trevor Clinkenbeard ef13985feb
Merge pull request #8320 from sfc-gh-tclinkenbeard/read-write-fungible
Make read and write quotas fungible
2022-09-28 10:59:37 -07:00
Nim Wijetunga 251afa3610
Set g_simulator fields for restart tests (#8318)
* fix tests

* fix tenant concurrency

* Set g_simulator variables for restart tests

* fix machine attrition for restarting
2022-09-28 10:09:26 -07:00
A.J. Beamon 3353103d9d Fix filtering of potential backup mutations in commit proxy and backup worker; add code probe to ensure we are testing default backup sharing and add some tests to hit it 2022-09-27 15:25:30 -07:00
Xiaoxi Wang 3428fe467a Merge branch 'main' of https://github.com/apple/foundationdb into feature/dd-refactor-simple 2022-09-27 11:55:35 -07:00
Xiaoxi Wang e476e814b2 merge upstream/main 2022-09-27 10:15:26 -07:00
sfc-gh-tclinkenbeard ba8fbc9573 Merge remote-tracking branch 'origin/main' into read-write-fungible 2022-09-26 23:26:14 -07:00
A.J. Beamon 1b8512e949
Merge pull request #8314 from sfc-gh-ajbeamon/metacluster-management-workload-expansion
Improve Metacluster Management Workload
2022-09-26 15:51:28 -07:00
A.J. Beamon b0c3534463 Handle invalid tenant configuration when a group and assigned cluster are specified for a tenant and the assigned cluster doesn't match the group's cluster 2022-09-26 14:32:26 -07:00
Jingyu Zhou 248c82eb3f
Merge pull request #8229 from sfc-gh-tclinkenbeard/guard-gsimulator
Guard a few uses of gsimulator.
2022-09-26 13:52:42 -07:00
A.J. Beamon 05615107da Minor refactoring to remove throw internal_error lines 2022-09-26 13:14:24 -07:00
A.J. Beamon 093db25d11 Add tenant group support to the tenant management workload and test configure and rename tenant operations. Fix a few error types picked up by the test. 2022-09-26 13:14:24 -07:00
Jon Fu 1d00ca3261 address code review comments 2022-09-26 10:37:58 -07:00
sfc-gh-tclinkenbeard ce0ad38c94 Merge remote-tracking branch 'origin/main' into guard-gsimulator 2022-09-26 10:18:43 -07:00
Jon Fu 76543cbd8a remove debug trace 2022-09-26 10:02:16 -07:00
Jon Fu 2d1a6b4434 Merge branch 'main' of github.com:apple/foundationdb into metacluster-assigned-cluster 2022-09-26 09:59:20 -07:00
Jon Fu 6b2c932e75 modify tests to work with error case of changing preferred cluster on retries 2022-09-26 09:59:06 -07:00
Evan Tschannen 86c27754fb
Fixed a bug where rollbackVersion >= data->storageVersion() (#8290)
* fix: the log router could see popped() versions between recoveredAt and the recovery txn version which cause remote tlogs to be stopped at a version in this range, which causes storage servers to read at versions in this range

* fix: a spurious correctness bug resulted from the consistency check running during a recovery
2022-09-26 09:19:53 -07:00
Dennis Zhou 080acd811f
tuple: user defined type support (#8300)
* tuple: add support for usertype as last part of tuple

The expectation is user defined types come at the end as there is no
delimiter standard. In that case, if we encounter a user defined type,
we append it to the end of the tuple and allow users to transform that
type by parsing raw string returned.

* blob: use Tuple::unpackUserType() for key alignment

We want blob granule key alignment to be able to understand user data
types. This upgrades support and allows user data types to be the last
key in a blob granule and for us to be able to parse that properly.
2022-09-26 09:18:37 -07:00
sfc-gh-tclinkenbeard 7fc5c196c4 Make read and write quotas fungible 2022-09-25 21:00:11 -07:00
sfc-gh-tclinkenbeard 985958c260 Add rare code probe decoration 2022-09-25 15:28:32 -07:00
Dan Adkins df2c1374cb
Correct a couple of comments in simulator and simulated workloads. (#8310) 2022-09-24 22:52:13 -07:00
Markus Pilman 5774249e5b
Revert "[DRAFT] Redwood PriorityMultiLock enable different launch limits to be specified based on different priority level." 2022-09-23 12:22:47 -06:00
Markus Pilman 82d8c17d00
Merge pull request #8272 from sfc-gh-dadkins/sfc-gh-dadkins/bugs/reboot-not-kill
Downgrade kill to reboot in Rollback workload.
2022-09-23 11:32:04 -06:00
Markus Pilman fe4c33fabb
Merge pull request #8302 from sfc-gh-huliu/testfix2
disable MoveKeysWorkload for a few tests that need to manipulate dd
2022-09-23 11:02:04 -06:00
Hui Liu 9fd16bd38e disable MoveKeysWorkload for a few tests that need to manipulate dd 2022-09-23 08:56:27 -07:00
Hui Liu c9b4fc5761 disallow MoveKeysWorkload running in parallel 2022-09-22 17:00:01 -07:00
Josh Slocum 430f6e9670
Fix purge at latest racing with other checking threads at the end of BlobGranuleVerify (#8281) 2022-09-22 16:32:33 -07:00
Jon Fu 2eea93f170 move cluster assignment into loop to test error case 2022-09-22 14:43:08 -07:00
Jon Fu e342a9db43 Merge branch 'main' of github.com:apple/foundationdb into metacluster-assigned-cluster 2022-09-22 14:39:27 -07:00
Jon Fu 8a6e68cf63 adjust check if existing entry cluster does not match 2022-09-22 14:39:03 -07:00
A.J. Beamon bd006526d6
Merge pull request #8251 from sfc-gh-ajbeamon/fdbcli-tenant-group-metadata
Fdbcli command to get tenant group metadata
2022-09-22 14:17:08 -07:00
A.J. Beamon 97a325adab Add an fdbcli command to get tenant group metadata 2022-09-22 13:24:21 -07:00
Xiaoxi Wang be1cc6c111
Update fdbserver/workloads/IDDTxnProcessorApiCorrectness.actor.cpp
Co-authored-by: Bharadwaj V.R <bharadwaj.vr@snowflake.com>
2022-09-22 13:06:21 -07:00
Steve Atherton 7831f6b2f1
Merge pull request #7578 from sfc-gh-fzhao/RedwoodIOLaunchLimit
[DRAFT] Redwood PriorityMultiLock enable different launch limits to be specified based on different priority level.
2022-09-22 12:29:17 -07:00
Nim Wijetunga 37b93a6232
fix tests (#8286) 2022-09-22 11:34:06 -07:00
Jon Fu b559be1184 assign cluster outside of retry loop 2022-09-22 11:22:41 -07:00
Jon Fu e91afa15b6 Merge branch 'main' of github.com:apple/foundationdb into metacluster-assigned-cluster 2022-09-22 11:14:27 -07:00
A.J. Beamon fda0d7223d Update backup to include system key ranges needed for tenants. Run simulated backup tests with tenants. 2022-09-22 10:00:13 -07:00
Steve Atherton 04b4960786 Merge branch 'main' into fzhao/RedwoodIOLaunchLimit
# Conflicts:
#	fdbserver/VersionedBTree.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/ReadWrite.actor.cpp
2022-09-22 00:39:51 -07:00
Jon Fu 7a09b701cc
Merge pull request #8141 from sfc-gh-jfu/network-disable-bypass
Introduce network option for disabling mvc bypass
2022-09-21 17:33:48 -07:00
Hui Liu bf943bfd57
Merge pull request #8273 from sfc-gh-huliu/testfix
disable MoveKeysWorkload when running SpecialKeySpaceCorrectness test
2022-09-21 17:32:44 -07:00
Jon Fu 937fb38dcc address review comments 2022-09-21 16:23:12 -07:00
Hui Liu f115599a67 disable MoveKeysWorkload when running SpecialKeySpaceCorrectness test 2022-09-21 16:12:20 -07:00
Josh Slocum 4794ebabcc
Merge pull request #8262 from sfc-gh-dzhou/purge-fixes
blob: java api fixes (purge + verify)
2022-09-21 17:25:54 -05:00
Dan Adkins 5aa9a36559 Downgrade kill to reboot in Rollback workload.
Indiscriminately killing machines can lead to unrecoverable clusters,
even if we avoid killing the protected coordinators.
2022-09-21 15:22:31 -07:00
Dennis Zhou 4ea4546cb6 blob/java: verifyBlobRange() with latestVersion 2022-09-21 14:07:16 -07:00
Dennis Zhou e353169a50 blob: teach purge about latestVersion
This teaches purgeBlobGranules about latestVersion and rejects
versions <= 0.
2022-09-21 14:04:58 -07:00
Josh Slocum 6270016bed
Seq insert perf fixes main (#8264)
* Force flushing granules post-split to guarantee parent feeds get cleaned up

* fixing bug and cleaning up split finalize code
2022-09-21 12:36:02 -07:00
Jon Fu 0343ca9c53 Merge branch 'main' of github.com:apple/foundationdb into network-disable-bypass 2022-09-21 11:33:27 -07:00
Hui Liu c237186ec3
Merge pull request #8252 from sfc-gh-huliu/fixtest
Fix correctness test: skip lock check in MoveKeysWorkload
2022-09-20 16:34:22 -07:00
Trevor Clinkenbeard 3b5117ca92
Merge pull request #8233 from sfc-gh-mpilman/features/disable-failure-injection-for-workload
All workloads to disable some failures
2022-09-20 16:17:35 -07:00
Nim Wijetunga eadb769cfa
Encrypt Backup Mutation Log (#8159)
* encrypt backup mutation log

* format

* address pr comments

* format

* fix bug

* revert knobs

* address pr comments
2022-09-20 15:43:39 -07:00
Chaoguang Lin 9628561235
Add DataDistributionMetrics workload into correctness packages, (#8237)
which makes the code probe hit in nightly tests.
2022-09-20 15:33:15 -07:00
Hui Liu 52f1bef8ec Fix correctness test: skip lock check in MoveKeysWorkload 2022-09-20 15:16:56 -07:00
Jon Fu 4bbc2ad597 Merge branch 'main' of github.com:apple/foundationdb into metacluster-assigned-cluster 2022-09-20 09:34:06 -07:00
Jon Fu 1f778f9d76 Merge branch 'main' of github.com:apple/foundationdb into network-disable-bypass 2022-09-20 09:30:53 -07:00
Andrew Noyes 4ac5498f2e Fix a UBSAN diagnostic caused by use of uninitialized memory
The diagnostic is

fdbserver/workloads/DiskFailureInjection.actor.cpp:57:64: runtime error: load of value 208, which is not a valid value for type 'bool'

The fix is simply to initialize the default value of verificationMode
2022-09-20 08:54:45 -07:00
Markus Pilman 62ac01b481 All workloads to disable some failures 2022-09-19 16:42:55 -06:00
Chaoguang Lin 125137b987
Change the special key space correctness workload to hit code probe (#8214) 2022-09-19 15:01:21 -07:00
sfc-gh-tclinkenbeard 0ec06f5675 Merge remote-tracking branch 'origin/main' into guard-gsimulator 2022-09-19 13:16:48 -07:00
Josh Slocum 0f3f493c28
Merge pull request #8218 from sfc-gh-jslocum/aligned_purge
fixes for non-aligned blob range calls
2022-09-19 15:00:02 -05:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
Markus Pilman e1627e0a78 Merge remote-tracking branch 'origin/main' into features/always-inject-faults 2022-09-19 09:38:55 -06:00
Josh Slocum 88f88707f5 fixes for non-aligned blob range calls 2022-09-16 19:06:15 -05:00
Markus Pilman ef2d301305 Merge remote-tracking branch 'origin/main' into features/always-inject-faults 2022-09-16 16:41:16 -06:00
Jon Fu 1abac8ea9f check on shared state ptr in native api and add to test spec in api tester 2022-09-16 15:11:33 -07:00
Markus Pilman 9441795c8e address review comments 2022-09-16 14:40:10 -06:00
sfc-gh-ngoyal 1bd97fe628
Recruit new singleton for consistency checker. (#5804)
* Recruit new singleton for consistency checker.

* Recruit the consistency checker only if enabled.

* Add a yield in monitorConsistencyChecker().

* Minor fixes.

* Consistency check workload enhancements.

* Minor fixes and clarifications.

* clang format

* Clang format.

* Minor fixes, cleanup, debug tracing.

* Misc.

* Move the consistency scan information from dbconfig to a key backed object.

* Move consistency scan config out of db cofig to a state object and feature rename.

* ConsistencyCheck workload refactor.

* devFormat

* Update fdbcli/ConsistencyScanCommand.actor.cpp

* Review Comments.

Co-authored-by: negoyal <neelam.goyal@gmail.com>
Co-authored-by: Ata E Husain Bohra <ata.husain@snowflake.com>
2022-09-16 09:03:06 -07:00
Xiaoxi Wang 7169a8b132 merge upstream/main 2022-09-15 21:27:26 -07:00
Fuheng Zhao ac65c3f569 merge upstream main 2022-09-15 14:19:19 -07:00
He Liu 6f7968b618 Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency 2022-09-15 10:15:33 -07:00
sfc-gh-tclinkenbeard b9fbd8c130 Fix -Wlogical-op-parentheses warning 2022-09-15 09:00:33 -07:00
sfc-gh-tclinkenbeard 82adc1e856 Make g_simulator a pointer 2022-09-15 09:00:33 -07:00
Lukas Joswiak cfbd04ae78 Add explicit check for a simulated network 2022-09-14 14:14:49 -07:00
Jon Fu 51b9861354 check data cluster entry for capacity and account for different thrown errors 2022-09-14 12:57:46 -07:00
Xiaoxi Wang 1f1a66be39 merge upstream/main 2022-09-14 12:32:28 -07:00
Xiaoxi Wang 674da807ad add readMoveKeysLock method and catch movekeys_conflict() in the test workload 2022-09-14 12:17:47 -07:00
Ata E Husain Bohra d2b82d2c46
Introduce "default encryption domain" (#8139)
* Introduce "default encryption domain"

Description

In current FDB native encryption data at-rest implementation,
an entity getting encrypted (mutation, KV and/or file) is categorized
into one of following encryption domains:
1. Tenant domain, where, Encryption domain == Tenant boundaries
2. FDB system keyspace - FDB metadata encryption domain
3. FDB Encryption Header domain - used to generate digest for
plaintext EncryptionHeader.

The scheme doesn't support encryption if an entity can't be categorized
into any of above mentioned encryption domains, for instance, non-tenant
mutations are NOT supported.

Patch extend the encryption support for mutations for which corresponding
Tenant information can't be obtained (Key length shorter than TenantPrefix)
and/or mutations do not belong to any valid Tenant
(FDB management cluster data) by mapping such mutations to a
"default encryption domain".

TODO

CommitProxy driven TLog encryption implementation requires every transaction
mutation to contain 1 KV, not crossing Tenant-boundaries. Only exception to
this rule is ClearRange mutations. For now ClearRange mutations are mapped
to 'default encryption domain', in subsequent patch appropriate handling
for ClearRange mutations shall be proposed.

Testing

devRunCorrectness - 100k
2022-09-14 10:58:32 -07:00
Xiaoxi Wang b1e48e06f8 remove meaningless comparison in InitDataDistribution 2022-09-14 10:56:00 -07:00
He Liu 28e5a70dbe Clean up SS validate storage. 2022-09-14 10:53:54 -07:00
Jon Fu e3f54dba2f throw invalid_tenant_configuration and add to metacluster management test 2022-09-14 10:22:51 -07:00
Xiaoxi Wang 85f119c13b fix destination team issue 2022-09-14 00:17:00 -07:00
Xiaoxi Wang 46eadd53cf finish getInitialDataDistribution test verification 2022-09-13 22:43:15 -07:00
Lukas Joswiak 424bb87f3e Apply feedback 2022-09-13 16:53:54 -07:00
Lukas Joswiak 2a26777d64 Disable coordinator change when using simple configuration database 2022-09-13 16:53:54 -07:00
Lukas Joswiak 74ac617a34 Add support for changing coordinators to the configuration database
Configuration database data lives on the coordinators. When a change
coordinators command is issued, the data must be sent to the new
coordinators to keep the database consistent.
2022-09-13 16:53:54 -07:00
Trevor Clinkenbeard b641bd6c04
Merge pull request #8168 from sfc-gh-tclinkenbeard/add-doappendiffits-unittest
Add `/Atomic/DoAppendIfFits` unit test
2022-09-13 15:18:59 -07:00
Xiaoxi Wang bafcacc1eb fix MockDDTxnProcessor::getInitialDataDistribution segment fault in assertion part; IDDTxnProcessorApiWorkload get mock initData finish 2022-09-13 14:39:12 -07:00
Xiaoxi Wang 2ae01bdf2d
Merge pull request #8162 from sfc-gh-xwang/feature/main/moveKey
Implement txnProcessor->moveKeys(const MoveKeysParams& params)
2022-09-13 14:11:49 -07:00
He Liu 958b28497e Merge branch 'main' of https://github.com/apple/foundationdb into validate-data-consistency 2022-09-13 13:55:01 -07:00
Markus Pilman cebf49798e Merge remote-tracking branch 'origin/main' into features/always-inject-faults 2022-09-13 12:36:12 -06:00
sfc-gh-tclinkenbeard 2bea5b88bf Add /Atomic/DoAppendIfFits unit test 2022-09-13 11:35:39 -07:00
Xiaoxi Wang eaff46cd27 IDDTxnProcessorApiWorkload setup - Get Initial DD in real cluster 2022-09-13 11:04:23 -07:00
Trevor Clinkenbeard 9f1e04cf9f
Merge pull request #8147 from sfc-gh-tclinkenbeard/improve-backup-code-coverage
Check backup agent activity in BackupAndRestoreCorrectnessWorkload
2022-09-13 09:32:40 -07:00
Markus Pilman 3e31820078 fix compilation issues 2022-09-13 10:29:29 -06:00
Markus Pilman 6e97bfb15a Added RandomMoveKeys 2022-09-12 17:20:57 -06:00
Markus Pilman 389ea4c952 added additional workloads 2022-09-12 17:16:32 -06:00
Markus Pilman acd24d6c81 Merge remote-tracking branch 'origin/main' into features/always-inject-faults 2022-09-12 16:44:16 -06:00
Xiaoxi Wang 949b1c1af9 Add MoveKeysParams struct; use txnProcessor->moveKeys() 2022-09-12 15:40:18 -07:00
He Liu 56cf51e2a0 Audit allKeys in auditStorage test. 2022-09-12 14:32:34 -07:00
Markus Pilman 59ce49913a
Merge pull request #8146 from sfc-gh-tclinkenbeard/improve-code-coverage
Increase the number of unit tests run in `RandomUnitTests.toml`
2022-09-12 15:10:47 -06:00
Fuheng Zhao ee99de7cf3
Merge branch 'apple:main' into RedwoodIOLaunchLimit 2022-09-12 10:58:12 -07:00
sfc-gh-tclinkenbeard 924c198a5b Run 10 unit tests within RandomUnitTests.toml 2022-09-11 00:36:18 -07:00
sfc-gh-tclinkenbeard cc34658a32 Check backup agent activity in BackupAndRestoreCorrectnessWorkload 2022-09-10 23:43:11 -07:00
Yi Wu d831c87d14
Add encryption metrics (#8070)
Adding the following metrics:
* BlobCipherKeyCache hit/miss
* EKP: KMS requests latencies
* For each component that using encryption, they now need to pass a UsageType enum to the encryption helper methods (GetEncryptCipherKeys/GetLatestEncryptCipherKey/encrypt/decrypt) and those methods will help to log get cipher key latency samples and encryption/decryption cpu times accordingly.
2022-09-09 18:43:09 -07:00
Jon Fu bfd2d138f7 set assignedcluster in tenantmanagement workload 2022-09-09 16:10:42 -07:00
He Liu e6846e1ed5 Handle auditStorage API errors. 2022-09-09 15:04:34 -07:00
Jon Fu 7381cf6ef4 update test assertions 2022-09-09 12:11:02 -07:00
Jon Fu 328e4b3bf9 use state variable for error 2022-09-09 10:15:25 -07:00
Jon Fu 75a096a5e5 Merge branch 'main' of github.com:apple/foundationdb into jfu-tenant-special-key-space 2022-09-09 10:12:19 -07:00
Jon Fu 8a24afb152 fix miscellaneous errors in special keys tests 2022-09-09 10:07:37 -07:00
He Liu 9e911956d9 Handle GetKeyValuesReply errors. 2022-09-08 15:41:22 -07:00
Junhyun Shim bc47f90aff
Merge pull request #8125 from sfc-gh-jshim/knob-tokenless-tenant-access
Add a knob to allow token-less tenant data access for untrusted clients
2022-09-08 17:52:32 +02:00
A.J. Beamon 726d5215a0
Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (#8108)
* Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (no need to guard)

* Enable the relaxed special key access in transactions that need to use special key-space APIs introduced in 7.2
2022-09-08 17:22:36 +02:00
Junhyun Shim 3023096962 Add a knob to allow token-less tenant data access for untrusted clients 2022-09-08 14:53:01 +02:00
He Liu 071ed41caa ManagementAPI auditStorage passed test. 2022-09-07 22:34:14 -07:00
Dennis Zhou 7fc4a92d09
Merge pull request #8089 from sfc-gh-jslocum/unblobbify_idempotent
Blob range idempotency test and fixes
2022-09-07 17:31:05 -07:00