891331caed
Merge pull request #8881 from sfc-gh-huliu/fixinit
...
Init blobGranulesEnabled in ISimulator
2022-11-18 17:07:57 -08:00
de8285f11d
Merge pull request #8834 from sfc-gh-huliu/status
...
Show blob restore in fdbcli status command
2022-11-18 16:03:30 -08:00
bee0377b4d
Init blobGranulesEnabled in ISimulator
2022-11-18 15:53:06 -08:00
fc1d29f429
Arena: avoid null-pointer reference in ASSERT ( #8875 )
...
Narrow the self-reference ASSERT in ArenaBlock::dependOn to only
the cases where we're trying to make a reference from self -> other.
It's allowed for self.getPtr() to be null, in which case we create
the ArenaBlock anew and there's no danger of self-reference.
Fixes Issue #8869 .
2022-11-18 18:08:58 -05:00
ebd5b48f6e
Merge pull request #8876 from yao-xiao-github/main
...
Reuse RocksDB options and CF options.
2022-11-18 13:55:55 -08:00
5f1c4f2989
Disable all failure injection workloads for special key space correctness tests ( #8866 )
2022-11-18 14:06:14 -05:00
95cd0a4660
Merge pull request #8859 from yao-xiao-github/release-7.2
...
[release-7.2] Reuse RocksDB options and CF options.
2022-11-18 11:03:11 -08:00
877a9388b8
Merge pull request #8748 from LukasMoll/bindings-python-fdb-flake8
...
/bindings/python/fdb Applied flake8
2022-11-18 08:41:18 -08:00
2ac48e0b76
Merge pull request #8857 from sfc-gh-jfu/metacluster-status-schema
...
Add metacluster status details to schemas
2022-11-18 08:18:10 -08:00
7c20002332
Merge pull request #8559 from sfc-gh-jshim/authz-security-tests
...
Authz security simulation tests
2022-11-18 16:48:46 +01:00
2a74624cbd
Merge pull request #8848 from neethuhaneesha/suggest-compacts
...
Rocksdb suggest compact range checks
2022-11-17 20:01:19 -08:00
eda9d701dc
Set timeout to 5 sec. ( #8865 )
2022-11-17 19:56:26 -08:00
3e5cad1508
Merge pull request #8837 from sfc-gh-akejriwal/monitorusage
...
Add a TraceEvent to export the size of tenant groups
2022-11-17 17:15:31 -08:00
4b52560dbc
Merge branch 'main' of github.com:apple/foundationdb into monitorusage
2022-11-17 15:47:39 -08:00
f6612ebd00
When selecting physical shard and it is not available for the remote team, retry selecting a new dst physical shard
2022-11-17 15:44:17 -08:00
d865e77f06
RocksDB 7.7.3 version upgrade ( #8858 )
2022-11-17 15:39:22 -08:00
973d055193
add data cluster fields to Schemas.cpp
2022-11-17 15:04:11 -08:00
186d30be95
Merge pull request #8823 from xumengpanda/mengxu/main-tmp
...
Increase memtable and writebuffer size for rocksdb simulation test
2022-11-17 14:45:14 -08:00
2e62822183
Show blob restore in fdbcli status command
2022-11-17 14:22:59 -08:00
b46c9eb67d
Rocksdb suggest compact range checks
2022-11-17 14:20:18 -08:00
ce041576f5
Merge remote-tracking branch 'origin/main' into authz-security-tests
2022-11-17 22:44:31 +01:00
180b531691
Merge pull request #8856 from jzhou77/fix
...
Fix a log router race condition that blocks remote tlogs forever
2022-11-17 13:15:39 -08:00
50d616bb69
add metacluster status details to schemas
2022-11-17 12:59:59 -08:00
f285a91f6c
Add more debug events
2022-11-17 11:54:21 -08:00
c6ebdd8ae8
Send error when LogRouterPeekPopped happens
...
Otherwise, the remote tlog won't get a response and the parallel peek requests
will never be cleared, blocking subsequent peeks. As a result, remote tlog will
no longer be able to pop the log router, which in turn can no longer peek tlogs.
The whole remote side will become blocked.
2022-11-17 11:54:14 -08:00
8e8c4b4489
Merge pull request #8170 from sfc-gh-sgwydir/ddsketch
...
Use DDSketch for sample data
2022-11-17 10:38:12 -08:00
349f5821d7
Merge pull request #8592 from sfc-gh-jfu/tenant-list-filter
...
Add option to filter the tenant list command for metacluster
2022-11-17 09:04:58 -08:00
7c65ca6648
Merge remote-tracking branch 'origin/main' into authz-security-tests
2022-11-17 17:48:44 +01:00
950ca22ee2
Merge pull request #8633 from sfc-gh-xwang/feature/main/dataApi
...
Add MockStorageServer data API implementations
2022-11-16 20:28:02 -08:00
f3e9eb528e
Merge branch 'main' of github.com:apple/foundationdb into monitorusage
2022-11-16 17:06:49 -08:00
3a5cf32f0b
Trying to re-use existing physical shard during data movement
2022-11-16 16:36:00 -08:00
8971b5907c
add comments; mark some methods const
2022-11-16 13:18:40 -08:00
2edb2fbe44
Merge pull request #8833 from sfc-gh-satherton/actor-collection-constant-time
...
Rewrote ActorCollection to be O(1) instead of O(log n) for actor add and completion
2022-11-16 11:11:58 -08:00
91fc3fef4a
[EAR]: Remove usage of EncryptDomainName for Encryption at-rest operations ( #8715 )
...
* [EAR]: Remove usage of EncryptDomainName for Encryption at-rest operations
Description
diff-1: Address review comments
EncryptDomainName is an auxillary information, given EAR encryption domain
matches with Tenants, EncryptDomainName maps to TenantName in the current
code. However, this mapping adds EAR depedency has multiple drawbacks:
1. In some scenarios obtaning consistent mapping of TenantId <-> TenantName
is difficult to maintain. For instance: StorageServer (SS) TLog mutation
pop loop, it is possible that same commit batch contains: TenantMap update
mutation as well as a Tenant user mutation. SS would parse TenantMap update
mutation (FDB System Keyspace encryption domain), process the mutation, but,
doesn't apply it to the process local TenantMap. SS then attempts to process,
Tenant user mutation and fails to decrypt the mutation given TenantMetadaMap
isn't updated yet.
2. FDB codebase uses EncryptDomainId matching TenantId, TenantName is used as
an auxillary information source and feels better to be handled by an
external KMS.
Major changes include:
1. EAR to remove TenantName dependency across all participating processes
such as: CommitProxy, Redwood, BlobGranule and Backup agent.
2. Update EKP and KmsConnector APIs to avoid relying on "domainName"
information being passed around to external KMS EAR endpoints.
Testing
devRunCorrectness - 100K
EncryptKeyProxyTest - 100K
EncryptionOps Test - 100K
2022-11-16 10:26:39 -08:00
1f93f301fb
Merge branch 'main' of github.com:apple/foundationdb into monitorusage
2022-11-16 10:08:23 -08:00
55c5d2d266
Merge remote-tracking branch 'origin/main' into authz-security-tests
2022-11-16 17:03:06 +01:00
72642c7142
Fix StorageServerCollection::addCost write cost calculation
2022-11-16 08:01:54 -08:00
9a258e087b
Track commit costs while ReportCommitCostEstimationRequest is in-flight
2022-11-16 07:37:00 -08:00
6b77bcfc35
fix merge conflict upstream/main
2022-11-16 07:34:42 -08:00
65cd272f63
Merge remote-tracking branch 'origin/main' into authz-security-tests
2022-11-16 16:02:43 +01:00
a21f657429
Add Encryption Code Probes for BlobGranule ( #8779 )
...
* add code probes
* address pr comments
* address pr comments
* Trigger Build
* fix
2022-11-16 06:13:55 -08:00
0af0079a62
several getChangeFeedMutations performance improvements based on cpu profiling ( #8741 )
2022-11-16 07:56:54 -06:00
632bddd4de
Merge remote-tracking branch 'origin/main' into authz-security-tests
2022-11-16 10:18:34 +01:00
bfefbfee8c
Merge pull request #8705 from sfc-gh-jshim/authz-accept-base64-for-jwt-tenant-name
...
Make token's 'tenants' field base64-encoded (cf. base64url)
2022-11-16 10:17:10 +01:00
959f77d03d
Merge pull request #8730 from sfc-gh-xwang/fix/main/restoreStats
...
Fix the omit StorageWiggler.restoreStats() bug
2022-11-15 22:37:39 -08:00
f9e87240d0
Add unit test that verifies choose/when behavior needed by ActorCollection.
2022-11-15 18:58:50 -08:00
959bf9f4e7
Merge branch 'main' of github.com:apple/foundationdb into monitorusage
2022-11-15 18:32:21 -08:00
b2451e600a
Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/dataApi
2022-11-15 17:04:49 -08:00
70a26ac314
Adding delays between requests to force sim time to advance at a reasonable rate ( #8784 )
2022-11-15 16:29:22 -08:00
56c2651dfb
Simplification after understanding why i is always valid after runnerHandler() is started.
2022-11-15 16:24:39 -08:00
Hui Liu