1b01c1b76c
Merge commit '53a3744de729639ec2d8ad7ba8ee899355a8f28d' into redwood-queue-error-handling
2023-03-04 03:29:36 -08:00
8847e70be0
Merge pull request #9306 from kakaiu/add-physical-shard-meta-data-to-checkpoint
...
Dump checkpoint metadata to sst file
2023-03-03 14:45:50 -08:00
57ff58fd1a
EKP Retry Loop on KMS Connection Failures ( #9524 )
...
EKP Retry Loop
2023-03-03 09:41:20 -08:00
b5639339ad
address comments
2023-03-03 09:04:25 -08:00
b0d595c7d7
Merge commit '3d9f37d1d12f95d52ef2f6bf57f123fdf6101602' into redwood-queue-error-handling
2023-03-03 01:09:04 -08:00
b2d3b35774
Add option to FlowMutex to wait forever if there is an error and use this in Redwood's pager queue cursor. This avoids a broken_promise error being seen before shutdown after a queue read sees an IO error or timeout.
2023-03-02 16:41:45 -08:00
544890a6cd
Merge branch 'main' into transaction-debug-logging
2023-03-01 10:09:17 -08:00
a00cdc8396
Rename template type and variable for TraceEvent::moveTo
2023-03-01 08:43:18 -08:00
b6f0d2095a
Cases where newBuf == buf assertion fails have been observed in Valgrind ( #9528 )
2023-03-01 15:49:11 +01:00
a714c0d4cd
Fix missing initialization of the err variable
2023-02-28 16:55:57 -08:00
310fc2ff4e
Merge branch 'main' into transaction-debug-logging
2023-02-28 14:18:51 -08:00
87ac857aeb
Make debug logging functions pure virtual on the transaction interfaces. Rename the function on TraceEvent to be more generic.
2023-02-28 11:11:06 -08:00
5bebb5b4aa
Merge pull request #9492 from sfc-gh-vgasiunas/vgasiunas-api-version-defs
...
Centralize definition of API Version for Java, Python and C API
2023-02-28 12:04:02 -07:00
0abb33a9a5
Add the ability to print messages or log trace events based on a transaction's result
2023-02-28 09:06:54 -08:00
29a406948a
Merge pull request #9370 from sfc-gh-mpilman/features/tenant-lock-fdbcli
...
fdbcli commands for tenant lock
2023-02-27 16:18:51 -08:00
bcc05b1058
Improve support for prebuilt boost
2023-02-27 15:38:58 -06:00
b811881f41
Allow unthrottled, unsuppressed traces for security-related events ( #9459 )
...
* Define API for unsuppressable TraceEvent types
Add trace checking tests for authz trace events
* Revert temporary configurations used for debugging
* Simplify/Modernize flow audit logging API
- Do event type whitelist checks at compile time
- Use ""_audit literal API instead of a tag struct
- Replace int with a lightweight struct for tracking/modifying TraceEvent enablement
* Revert installing signal handler for SIGTERM and refactor test script
Move trace checker to local_cluster.py
* Lengthen public key refresh interval and add more audited events
* Try and make MSVC and Mac build happy
* consteval > constexpr
'inline consteval' still causes link errors in Mac builds
2023-02-27 21:51:13 +01:00
80eb84de3c
Merge pull request #9488 from sfc-gh-satherton/redwood-first-commit-record
...
Fix correctness failures in encryption enforcement related to Redwood instances killed during initialization.
2023-02-27 12:10:02 -08:00
bf9e5cdc7e
Fix a typo in one actor helper filter(...) ( #9499 )
...
As title.
2023-02-27 11:13:59 -08:00
7c19e0d846
fix UID formatting
2023-02-27 11:44:38 -07:00
f031724f40
Address review comments
2023-02-27 09:55:40 -08:00
ba726fac87
Replace hardcoded API version checks for 720 and 730
2023-02-27 16:18:01 +01:00
7f35b395d3
Define the latest Java bindings API version in one place
2023-02-27 11:53:01 +01:00
92bfa1d578
Throw an error if reopened Redwood file is recovered but to a point prior to when the BTree was created if the Encryption Mode of the cluster is not known.
2023-02-26 21:15:48 -08:00
2f9cbba2a6
Avoid double evaluation of macro arguments
2023-02-24 16:20:26 -08:00
acf1dfb245
Fix formatting
2023-02-24 13:00:57 -08:00
4a38bb4c3f
Allow performing assert comparisons (e.g. ASSERT_EQ) with any traceable type
2023-02-24 12:53:01 -08:00
8695fc15fc
Merge remote-tracking branch 'origin/main' into features/tenant-lock2
2023-02-22 13:12:23 -07:00
5969616af8
Merge commit '6de85e7cd8e9dd74a571de9e04679e669bcbb5b6' into client-read-options
2023-02-21 20:46:20 -08:00
15d8548c0e
Merge remote-tracking branch 'origin/main' into features/tenant-lock2
...
# Conflicts:
# fdbserver/ApplyMetadataMutation.cpp
# fdbserver/storageserver.actor.cpp
2023-02-21 13:39:35 -07:00
81f8c360db
Merge pull request #8811 from sfc-gh-tclinkenbeard/expose-tag-throttled-duration
2023-02-21 10:47:35 -08:00
06ee5f1e76
Disable client memory wipe test for all XSAN builds ( #9409 )
2023-02-21 14:10:44 +01:00
246fd1dd4e
Remove auto cache option since there is no meaningful implementation of this yet. Change places using trState in a native Transaction to set cache mode or Low/Normal/High priority to use the new transaction options instead.
2023-02-21 02:50:30 -08:00
398079db3a
Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration
2023-02-20 17:54:06 -08:00
1aef6cb5f7
Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration
2023-02-17 20:41:59 -08:00
18e1a78815
Merge pull request #9396 from sfc-gh-satherton/platform-total-space-correction
...
Adjust reported total space for a disk by the reserved space which normal users cannot use.
2023-02-17 10:05:26 -08:00
aba2188491
Merge pull request #9399 from sfc-gh-satherton/read-after-shutdown-protection
...
Shut down PriorityMultiLocks more gracefully
2023-02-17 10:04:51 -08:00
cc1c1d5435
Merge commit '99b23ac04d302b1edc6db04f1488a20f8f772ae1' into platform-total-space-correction
2023-02-16 22:06:49 -08:00
ccba8731dd
Math bug fix.
...
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2023-02-16 15:48:16 -08:00
d9c126a2d9
Introduce WipedString for Arena block holding AuthZ tokens ( #9381 )
...
* Enable secure allocation mode in Arena
This mode allows zeroing out blocks holding sensitive data after use
* Introduce WipedString to all token-holding memory
Also introduce a option flag "sensitive"
* Make pointer equivalency a hard requirement for non-ASAN builds
So that we can detect when Arena/malloc/memory-wipe behavior changes
2023-02-16 10:44:32 +01:00
a1c804bc87
Added PriorityMultiLock::halt() and used that instead of kill() which avoids broken_promise errors if the lock user does not stop all of its lock-taking actors before destructing the lock and itself.
2023-02-15 23:32:17 -08:00
4b19eee33e
Adjust reported total space for a disk by the reserved space which normal users cannot use.
2023-02-15 19:01:30 -08:00
f2ff385018
Prevent the same cluster from being restored concurrently
2023-02-15 12:05:51 -08:00
8c94b340ce
EaR: Update encryption methods to make 'cipherHeaderKey' optional ( #9378 )
...
* EaR: Update encryption methods to make 'cipherHeaderKey' optional
Description
diff-1: Address review comments
Major changes includes:
1. Update BlobCipher Encrypt/Decrypt classes to make 'headerCipher' optional
2. Update GetEncryptionCipherKeys actor methods to make 'headerCipherKey' optional
3. Update the usage across all encryption participant methods
Testing
BlobCipherUnitTest
EnryptedBackupCorrecctness
BlobGranuleCorrectness*
devRunCorrectness - 100K
2023-02-15 08:56:11 -08:00
a5ca96ddf6
Merge pull request #9342 from sfc-gh-ajbeamon/metacluster-mgmt-restore
...
Metacluster restore support
2023-02-14 20:41:20 -08:00
3d882a99c5
EaR: Refactor encryption header std::variant serializer and versioning ( #9345 )
...
Changes:
1. Make binary serializer natively support `std::variant`. Serialize size is 1 byte (the type index, i.e. `std::variant::index()`), plus the serialize size of the actual type stored in the `std::variant`. Update `BlobCipherEncryptHeaderRef` to use the `std::variant` binary serializer
3. Remove `flagsVersion` and `algoHeaderVersion` from `BlobCipherEncryptHeaderRef`. The former is replaced by `flags.index() + 1`, and the latter is moved into each of the algorithm-specific sub-headers. Each sub-header types will have nesting version-specific subtypes to handle serialization of that specific version (e.g. for `AesCtrNoAuth` it has a `AesCtrNoAuthV1` subtype).
2023-02-14 20:19:27 -08:00
fe18c87ac6
EaR: commit proxy fetch additional cipher keys post-resolution ( #9308 )
...
Commit proxy needs to fetch additional cipher keys post-resolution, since tenant ids for raw access requests and cross-tenant clear ranges are calculated after resolution.
2023-02-14 13:05:51 -08:00
7284e691fb
Fix a few minor restore bugs and add a dry-run mode. Some improvements to the fdbcli output.
2023-02-14 12:28:55 -08:00
f3b58a063f
Fix some merge issues and review comments
2023-02-13 15:32:44 -08:00
98407809d9
Merge branch 'main' into metacluster-mgmt-restore
...
# Conflicts:
# fdbcli/MetaclusterCommands.actor.cpp
# fdbclient/Metacluster.cpp
# fdbclient/include/fdbclient/MetaclusterManagement.actor.h
# fdbserver/workloads/MetaclusterManagementWorkload.actor.cpp
# tests/CMakeLists.txt
2023-02-13 12:30:33 -08:00
Steve Atherton