Commit Graph

21545 Commits

Author SHA1 Message Date
A.J. Beamon 7324f28441 Add the ability for StringRef.eat to return whether a separator was found. Add a serialize function and equality operators to ClusterConnectionString. Remove unused variable. 2022-07-19 11:55:58 -07:00
A.J. Beamon 1b81e72604 Add a Tuple::makeTuple function to easily construct a tuple. Update Tuple to allow all types to be passed via .append() so they can be used with makeTuple. 2022-07-19 11:50:58 -07:00
A.J. Beamon 0d20cd2068
Merge pull request #7618 from sfc-gh-ajbeamon/fix-compile-warnings
Fix a few compile warnings about unused variables
2022-07-19 11:10:42 -07:00
A.J. Beamon c4cd72bea1 Fix a few compile warnings about unused variables 2022-07-19 10:01:41 -07:00
Xiaoxi Wang 491938fdf5 Merge branch 'main' of https://github.com/apple/foundationdb into feature/dd-refactor-incremental 2022-07-19 09:51:44 -07:00
A.J. Beamon 0e03fbd9e7
Merge pull request #7605 from sfc-gh-ajbeamon/fix-trace-log-detail-formatting
Remove some instances of using printable or toString in a trace event detail
2022-07-19 08:24:58 -07:00
Dan Lambright 775ab3323e
update tag client cache on getKeyLocation (#7613) 2022-07-19 11:24:17 -04:00
Josh Slocum 78d4d85f3b Adding non-tss delay injection to SS as well 2022-07-19 09:59:14 -05:00
Josh Slocum 0d9bb9f4a5 Added targeted storage server restarts at critical metadata points 2022-07-19 08:33:43 -05:00
Josh Slocum 2573c0018c cleanup after passing tests 2022-07-19 08:00:25 -05:00
Josh Slocum 679bb05d34 change feed move/destroy race fixes 2022-07-19 07:19:45 -05:00
Trevor Clinkenbeard 1eec6f993d
Merge pull request #7572 from sfc-gh-akejriwal/tenantquota
Introduce storage quotas per tenant
2022-07-18 20:00:22 -07:00
Josh Slocum 0a4aef16d0 Several bugs with new change feed metadata 2022-07-18 19:32:48 -05:00
A.J. Beamon e65ac1a0c1 Remove one more include and some extraneous return statements. Add a NullCodec that passes through the arguments unchanged. 2022-07-18 16:37:04 -07:00
A.J. Beamon 38855fd786 Remove some instances of using printable or toString in a trace event detail. 2022-07-18 16:24:33 -07:00
Ankita Kejriwal 2757262ec9
Simplify storageQuotaKey() function implementation
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2022-07-18 15:56:11 -07:00
Josh Slocum bfab550435 cleanup 2022-07-18 16:26:03 -05:00
Xiaoxi Wang 2909614b0c smaller function 2022-07-18 14:21:50 -07:00
Josh Slocum 340a01763b disabling merge convergence check for now 2022-07-18 16:09:14 -05:00
Renxuan Wang 570db7a7d2
Fix processDiskReadSeconds and processDiskWriteSeconds. (#7608) 2022-07-18 13:47:45 -07:00
Jingyu Zhou d28fb2b809
Merge pull request #7607 from sfc-gh-jshim/dlopen-nodelete-for-sanitizer
Apply RTLD_NODELETE for USE_SANITIZER builds
2022-07-18 13:25:33 -07:00
Trevor Clinkenbeard 18a2c3f8a8
Merge pull request #7489 from sfc-gh-tclinkenbeard/cstate-pimpl
Mark `CoordinatedState::getConflict` const
2022-07-18 13:22:18 -07:00
Josh Slocum 4540e2879c Change Feed Metadata refactor to fix races, simplify model, and clean up code 2022-07-18 12:46:11 -05:00
Junhyun Shim 8b70c7050d Apply RTLD_NODELETE for USE_SANITIZER builds
This allows their stack trace to be symbolized in CI/Jenkins XSAN runs
2022-07-18 19:25:12 +02:00
Andrew Noyes 989e323991
Attempt to fix windows build (#7600)
* Attempt to fix windows build

The windows build is complaining that some symbols in flow are
duplicate, which seems fair since we're currently compiling all the flow
sources and _also_ linking to flow for flowlinktest. This change makes
it so that we only link flow, and don't compile the flow source files
again.

* Remove source files from fdbclient and fdbrpc link tests
2022-07-18 08:59:03 -07:00
Josh Slocum ac21cbe957 Testing data is not readable after pop 2022-07-18 08:22:20 -05:00
Junhyun Shim d1dc82629e Clean up TenantInfo and related headers 2022-07-18 14:37:17 +02:00
Junhyun Shim ca29cd9f41 Make RequestStreams accept empty TenantInfo
+ Make Authz tokens survive Transaction resets
+ Make tokens explicitly resettable by assigning empty value to authz token TR option
+ Fill out missing trace for some failed token verifications
+ Trace non-sensitive token parts upon failed verification
2022-07-18 13:30:55 +02:00
Josh Slocum 150346af64 Explicitly avoiding large keys problem in single value 2022-07-15 21:08:46 -05:00
Josh Slocum 08186f9245 More efficient merge intent and granule history serialization 2022-07-15 20:38:06 -05:00
Xiaoge Su bb05c83e19 fixup! Update documentation per comments 2022-07-15 17:21:41 -07:00
Ankita Kejriwal 882df774c0 Storage quota: use transaction options and minor fixes in the test 2022-07-15 16:34:04 -07:00
Jon Fu 8a31fa87a8 add other operations to test workload 2022-07-15 16:22:37 -07:00
Ata E Husain Bohra 3acba4e5d7 Fix snapshotFormatUnitTest infinite loop
Patch addresses an issue where loop generating random buffer
for the test might run in an infinite loop if KeySpace gets
exhausted but the 'targetBuffer' bytes aren't generated.

Description

Testing
2022-07-15 14:34:33 -07:00
A.J. Beamon 02ab3b05ab
Merge pull request #7590 from sfc-gh-ajbeamon/update-prefix-encoding-in-tenant-metadata-json
Use a more standard encoding in the JSON metadata reported for tenants
2022-07-15 14:32:53 -07:00
Josh Slocum fb4fc8af9d
Merge pull request #7587 from sfc-gh-ahusain/ahusain-hybrid_encryption_active
BlobGranule encryption and compression support
2022-07-15 15:57:51 -05:00
Josh Slocum 306610bfcb batch periodic merging in blob manager 2022-07-15 15:52:10 -05:00
Ata E Husain Bohra 05e7f34c80 BlobFile Encryption and compression support
Fix windows CI build issue.

Description

Testing
2022-07-15 11:04:02 -07:00
He Liu 4dc742493e
Add cleanUpChangeFeeds(). (#7598)
Co-authored-by: He Liu <heliu@apple.com>
2022-07-15 10:29:09 -07:00
Ata E Husain Bohra 1da88d5d8c BlobFile Encryption and compression support
Fix formatting issues

Description

Testing
2022-07-14 22:46:14 -07:00
Josh Slocum 866dda5763 Adding check for merge convergence 2022-07-14 20:49:43 -05:00
Ata E Husain Bohra f288abebc2 BlobFile Encryption and compression support
Fix formatting issues and rename KNOB

Description

Testing
2022-07-14 17:22:00 -07:00
Ankita Kejriwal 6f130b4a69 Change `quota` type to `unit64_t` to avoid macOS compilation issue 2022-07-14 17:09:03 -07:00
Ata E Husain Bohra f6f117592d BlobFile Encryption and compression support
- Limit verbose logging under DEBUG_MACRO
 - Update/Add code documentation

Description

Testing
2022-07-14 17:04:14 -07:00
Ata E Husain Bohra 3a3811d6cc BlobFile Encryption and compression support
Few fixes
 - Update BlobGranuleCipherKeysMeta to use 'std::string' to
   persist 'Initialization Vector'
 - Update WriteSnapshot to persist 'BlobGranuleCipherKeysMeta' as part
   of BlobFileIndex

Description

Testing
2022-07-14 17:04:14 -07:00
Ata E Husain Bohra 24b2de8de8 BlobFile Encryption and compression support
Description

Testing
2022-07-14 17:04:14 -07:00
Ankita Kejriwal bb05321d24 Introduce storage quotas per tenant.
This change adds:
* ability to store the mapping from tenants to quota in the system keyspace,
* a setter and getter function
* a new workload to test this functionality

FDBCORE-2437
2022-07-14 16:35:12 -07:00
Xiaoxi Wang 4cbe5dd683
Merge pull request #7595 from sfc-gh-xwang/feature/dd-refactor-incremental
[DD testability] add resumeRelocation method
2022-07-14 15:24:56 -07:00
Yi Wu 7d7ce0909f
Restart tests carry forward encryption knobs value (#7497)
Previously to get around the issue that EKP is not present when restart test switching encryption from on to off and read encrypted data, EKP was made to start in simulation regardless of encryption knob. This PR revert that change, and instead force restart test not to change encryption knob, by passing previous encryption knob through restartInfo.ini file. Also since we don't allow downgrading an encrypted cluster to previous version, disable encryption in downgrade tests.

Also adding an assert to allow reading encrypted mutations only if encryption knob is on. We may reconsider allowing switching encryption on/off for existing cluster, but for now we don't allow it.
2022-07-14 14:45:17 -07:00
A.J. Beamon 7bca503528 Use narrow includes in key backed types; remove some unnecessary return statements. 2022-07-14 13:26:43 -07:00