Commit Graph

20147 Commits

Author SHA1 Message Date
Xiaoxi Wang e9bf7c0851 fix unknown schema check 2022-04-25 17:23:20 -07:00
Xiaoxi Wang 2ede89b623 fix getUint64 bug 2022-04-25 14:46:50 -07:00
Xiaoxi Wang 182f244e2e fix schema format 2022-04-25 10:28:45 -07:00
Xiaoxi Wang adeaf52db3
Merge branch 'apple:main' into ppw-store 2022-04-25 09:40:45 -07:00
Ray Jenkins 1c5bf135d5
Revert "Migrate to OpenTelemetry tracing. (#6855)" (#6941)
This reverts commit 5df3bac110.
2022-04-25 09:29:56 -05:00
Vaidas Gasiunas a9ff7b52e4
Merge pull request #6944 from sfc-gh-vgasiunas/vgasiunas-upgrade-test-fix-concurrent-download
Upgrade Tests: Avoid corrupting downloaded files in case of parallel test execution
2022-04-25 16:23:26 +02:00
Vaidas Gasiunas 9921a9bd02 Upgrade Tests: Avoid corrupting downloaded files in case of parallel test execution 2022-04-25 15:02:38 +02:00
Xiaoxi Wang 269e94cb71 add store type to status json 2022-04-24 22:53:06 -07:00
Evan Tschannen a40c8f231a fix: perpetual wiggle would not react to a pause that happened while starting the wiggle 2022-04-24 20:35:02 -07:00
Xiaoxi Wang 99d2335220 add storeType to metadata; updateStorageMetadata; combine storeTypeTracker 2022-04-23 00:03:57 -07:00
Bharadwaj V.R 9b66447783
Merge pull request #6762 from sfc-gh-bvr/block-down
Fail upon running server versions that are too old and incompatible with the newest software that has run on a cluster
2022-04-22 17:28:49 -07:00
Bharadwaj V.R 08323de905 fix formatting 2022-04-22 15:10:24 -07:00
Bharadwaj V.R 588b2fa509
Merge branch 'main' into block-down 2022-04-22 14:53:09 -07:00
Bharadwaj V.R 988a70f064
Merge pull request #6858 from sfc-gh-bvr/dbcorever
Track newest and lowest compatible protocol versions in DBCoreState
2022-04-22 14:46:21 -07:00
Bharadwaj V.R 4a5c2268da
Merge branch 'apple:main' into block-down 2022-04-22 14:45:54 -07:00
Markus Pilman 681314b0d9
Merge pull request #6937 from sfc-gh-ahusain/ahusain-fix-build
Fix Build: use boost::hash to compute hash for std::pair
2022-04-22 15:39:17 -06:00
Ata E Husain Bohra 6c9030408e Fix Build: use boost::hash to compute hash for std::pair
Description

Fix Build: use boost::hash to compute hash for std::pair

Testing

1. Build - gcc/clang
2. Simulation test: EncryptKeyProxyTest, EncryptionOps
3. Unit test: flow/BlobCipher
4. Running 10k correctness Joshua run
2022-04-22 13:16:30 -07:00
Sreenath Bodagala 706164d8ae - Address seg fault in "storageRecruiter" actor 2022-04-22 12:38:59 -07:00
Xiaoxi Wang b8fbd18199
Merge pull request #6927 from sfc-gh-xwang/main
Run completeDest after clear destination set
2022-04-22 09:23:36 -07:00
Vaidas Gasiunas ac057d79fb
Merge pull request #6930 from sfc-gh-vgasiunas/vgasiunas-upgrade-tests-check-binaries
Check the hash sum of the dowloaded binaries in the upgrade tests, retry on errors
2022-04-22 17:59:11 +02:00
Ata E Husain Bohra 670d40ef79
FDB native KMS Connector Framework (#6846)
* FDB native KMS Connector Framework

Description

Major changes includes:
1. Framework code to enable FDB native KMS connector implementation.
2. SERVER_KNOBS->KMS_CONNECTOR_TYPE controls the connector type selection.
3. KmsConnectorInterface endpoint definitions, every KMSConnector
   implementation needs to support defined endpoints.
4. Update EncryptKeyProxy to leverage KmsConnectorInterface endpoints
   to fetch encryption keys on-demand and/or periodic refreshes.
   Integrate SimKmsConnector implementation.
5. Implement SimKmsConnector by leveraging existing SimKeyProxy
   implementation.

Testing

Unit test: fdbserver/SimKmsConnector
Simulation: EncryptKeyProxy
2022-04-22 08:53:39 -07:00
Bharadwaj V.R 822eb9ec26
Merge branch 'apple:main' into dbcorever 2022-04-22 08:08:34 -07:00
Vaidas Gasiunas 0a9b2470da Upgrade Tests: Check the hash sum of the dowloaded files, retry the download on errors 2022-04-22 16:37:11 +02:00
Vaidas Gasiunas 44c5f87ac9
Merge pull request #6919 from sfc-gh-vgasiunas/vgasiunas-client-tmp-dir
Store client library copies in a build-local temporary directory
2022-04-22 15:46:57 +02:00
Bharadwaj V.R ed08cfbf52
Merge branch 'apple:main' into block-down 2022-04-22 06:19:38 -07:00
Vaidas Gasiunas 1064035c46 Update generated.go to include the client_tmp_dir option 2022-04-22 14:33:18 +02:00
Vaidas Gasiunas bc0096683f Remove an unused class member 2022-04-22 14:14:09 +02:00
Vaidas Gasiunas 9897712307 Merge remote-tracking branch 'apple/main' into vgasiunas-client-tmp-dir 2022-04-22 13:00:09 +02:00
Vaidas Gasiunas f9cb9deb1d Use build-local temp directory for C API tests 2022-04-22 11:57:29 +02:00
Vaidas Gasiunas 444b6c5479 Default the client temp directory to /tmp 2022-04-22 11:44:44 +02:00
Vaidas Gasiunas caaf43da12
Merge pull request #6900 from sfc-gh-jslocum/bg_client_api_tests
Blob granule client API tests
2022-04-22 09:21:31 +02:00
A.J. Beamon 1352083d4c
Merge pull request #6884 from sfc-gh-clin/deprecate-speical-keys
Remove the client profiling special keys and update related documentations
2022-04-21 21:51:16 -07:00
Xiaoxi Wang 960a13f233 format 2022-04-21 21:40:49 -07:00
Xiaoxi Wang f9e57396e2 completeDest 2022-04-21 21:23:42 -07:00
Zhe Wang 6c9ff6ee5e
Add sharded rocksdb type (#6862)
* add-sharded-rocksdb-type

* address comments

Co-authored-by: Zhe Wang <zhewang@Zhes-MacBook-Pro.local>
2022-04-21 22:53:14 -04:00
Chaoguang Lin 4b1d22d684 Polish the documentation 2022-04-21 19:28:50 -07:00
Ata E Husain Bohra 04ecd8e08f
Revert "Revert "Update 'salt' details for EncryptHeader AuthToken details (#6881)" (#6902)" (#6922)
Description

Major changes proposed:
1. This reverts commit f38b2e8209.
2. Also add fix for Valgrind failure due to unintialized variables.
3. Improve checks to catch is cipherKey details cached in BlobCipherKeyCache
   isn't as expected

Testing

Overall correctness: 10K (20220421-193911-ahusain-foundationdb-a730e5cb38541e20)
EncyrptionOps correctness: 100K (20220421-194315-ahusain-foundationdb-29c598a8b9420430)
EncryptionOps Valgrind: 100 (20220421-194434-ahusain-foundationdb-7fc5f98eddc0921a)
2022-04-21 18:57:56 -07:00
Xiaoxi Wang 9f1dceba26 Merge branch 'main' of https://github.com/apple/foundationdb 2022-04-21 16:41:54 -07:00
Ata E Husain Bohra 5b5b45c7cb
Update EKPServer LookupKeysByKeyIds API (#6898)
* Update EKPServer LookupKeysByKeyIds API

Description

Major changes include:
1. Update EKPServer LookupKeysByKeyIds API to include
   'encryptDomainId' as input along with 'baseCipherId'
2. Unify EKPServer response definition to return
   std::vector<EKPBaseCipherDetails>
3. Update SimKmsProxy interfaces as per above mentioned
   changes.

Testing

1. Update EncryptKeyProxyTest simulation test.
2. Update SimEncyrptKeyProxy unit test
2022-04-21 14:56:09 -07:00
Chaoguang Lin 879eb98dec Rename registerSpecialKeySpaceModule to registerSpecialKeysImpl 2022-04-21 12:54:34 -07:00
Chaoguang Lin 588041f977 Polish words according to comments 2022-04-21 12:40:32 -07:00
Vaidas Gasiunas b4fa9e23f6 Upgrade Tests: Use build local tmp directory for temp client library copies 2022-04-21 19:27:19 +02:00
Vaidas Gasiunas e48ab69c6b An option to specify a custom directory for temporary client library copies 2022-04-21 19:27:19 +02:00
Chaoguang Lin 9f110b0f51 Update documentations and comments 2022-04-21 10:19:44 -07:00
Tao Lin 98b82f1ac3 Fix GetMappedRange test when rangeResult has more 2022-04-21 10:16:27 -07:00
Bharadwaj V.R 449a315c06
Merge branch 'apple:main' into block-down 2022-04-21 09:37:42 -07:00
Bharadwaj V.R c20fb6ef6d
Merge branch 'apple:main' into dbcorever 2022-04-21 09:37:29 -07:00
Markus Pilman f38b2e8209
Revert "Update 'salt' details for EncryptHeader AuthToken details (#6881)" (#6902)
This reverts commit a38318a6ac.
2022-04-21 09:04:40 -07:00
Josh Slocum d26db64f08 only adding bg_url knob to local cluster config if blob granules enabled 2022-04-21 10:47:36 -05:00
Junhyun Shim aefe20e362
Merge pull request #6915 from sfc-gh-vgasiunas/vgasiunas-upgrade-tests-log-dump
Upgrade Tests: Dump entire log in case of errors
2022-04-21 16:02:00 +02:00