Commit Graph

20494 Commits

Author SHA1 Message Date
Xiaoxi Wang 5a431980d2 Merge branch 'main' of https://github.com/apple/foundationdb into features/debug-macro 2022-05-20 12:18:20 -07:00
Xiaoxi Wang 6b996dca29 add debug macro 2022-05-20 12:17:45 -07:00
Chaoguang Lin 5e8c3130e6 Extend the reboot interface to accept comma-delimited string;
Only fetch requested worker interfaces in rebootWorkerActor
Add killall test to multiprocess_fdbcli_tests
Make the rebootWorker only sends requests if all the requested workers are valid
Add comments; Organize print texts; Fix the bug clear the map every time run kill/suspend/expensive_data_check
2022-05-20 12:06:33 -07:00
Hao Fu fc8d04b19d
Make java binding work for prefetch match index modes. (#7186)
* Make java binding work for prefetch match index modes.

It adds boundaryAndExist in return value, so that caller can see
whether a boundary index is orphan or not.

* Make Java code work with index match modes

* Add test for boundaryAndExist in java binding test
2022-05-20 09:10:02 -07:00
Ray Jenkins dcd771bd07
Use semicolon for consistency with all other spans. (#7201) 2022-05-20 10:18:00 -05:00
Ray Jenkins d1efc05a9b
Remove parentContext.traceID from MessagePack serialization. (#7197)
The OTEL receiver does not utilize the parent Trace id, only parent Span id. Essentially
it relies on Trace ids implicitly matching during trace assembly. Therefore, we may remove
these 16 bytes in serialization.
2022-05-19 16:43:06 -05:00
He Liu 78dc7c5d77 Resolving comments. 2022-05-19 13:27:21 -07:00
He Liu 54c566e57d Drop RocksDB column family before create a new column family from
import.
2022-05-19 13:27:21 -07:00
He Liu 61490022b4 Enabled checkpoint, restore test for RocksDB format. 2022-05-19 13:27:21 -07:00
He Liu bc509d9572 Added fetchCheckpointKeyValuesQ in storage server. 2022-05-19 13:27:21 -07:00
He Liu 05bbe174c3 Implemented checkpoint(), restore() for KeyValueStoreRocksDB, for the
new format.
2022-05-19 13:27:21 -07:00
He Liu 5233d2a6ac ServerCheckpoint.actor.* 2022-05-19 13:27:21 -07:00
He Liu f8dd9acc7f Cleaned up RocksDBCheckpointUtil* 2022-05-19 13:27:21 -07:00
He Liu 9797a2b663 Added StorageServerInterface::fetchCheckpointKeyValues API. 2022-05-19 13:27:21 -07:00
Trevor Clinkenbeard d7f0928a48
Merge pull request #7161 from sfc-gh-bvr/ddspruce
Minor readability improvement (I believe) to DD code
2022-05-19 08:45:22 -07:00
A.J. Beamon 993a79b42c
Restart protocol monitoring logic when we cycle the coordinator, even if its the same coordinator. Add some defensive code in case we fetch the protocol version for an absent peer. (#6397) 2022-05-19 10:00:23 +02:00
Xiaoxi Wang 6c11fc74ba add debug traces 2022-05-18 15:20:23 -07:00
Trevor Clinkenbeard b55b67641c
Fix -Wlogical-op-parentheses warning (#7190) 2022-05-18 12:31:43 -07:00
Hao Fu dcacb30688
Add missing secondary queries tests for index prefetch (#7185)
* Add missing secondary queries tests for index prefetch

This change adds test for missing secondary queries for index prefetch,
in this case, MATCHED_ONLY mode would NOT return KV and UNMATCHED_ONLY
mode would return KV.

* remove default value for params
2022-05-18 12:13:19 -07:00
Hao Fu f00606fd71
Enable MATCHED and UNMATCHED mode for index prefetch (#7162)
* Enable MATCHED and UNMATCHED mode for index prefetch

MATCHED mode returns index entries whose secondary KVs are present,
UNMATCHED mode returns index entries whose secondary KVs are absent.

Note that the conflict read range of this txn is set in 2 steps:
* Set the conflict range for primary query according to request
* Set the conflict ranges for secondary queries according to responses.

As a result, conflicts of different match_index mode are taken care of.

* Fix c binding
2022-05-18 09:16:28 -07:00
Ata E Husain Bohra 728869466d
Update EncryptBaseCipher cache to be index using {baseCipherId, domainId} (#7183)
Description

Major changes proposed in the patch includes:
1. Update EncryptKeyProxy EncyrptBaseCipherKeyId cache to be indexed
   using {encryptDomainId, baseCipherId} instead of only 'baseCipherId'
2. Enhance RESTKmsConnector 'error' tag to encapsulte: errorMessage
   and errorCode information

Testing

1. Updated EncyrptKeyProxy test
2. Updated RESTKmsConnector unit test
2022-05-18 06:16:40 -07:00
Vaidas Gasiunas 5205b565ab
Upgrade Tests: Testing downgrade from 7.2 to 7.1 (#7179) 2022-05-18 11:59:29 +02:00
Xiaoxi Wang dc03bd0a29
Merge pull request #7182 from sfc-gh-xwang/main
remove duplicate recoveryCount++
2022-05-17 16:20:51 -07:00
Xiaoxi Wang 9b28b46a1d remove duplicate recoveryCount++ 2022-05-17 14:10:01 -07:00
Ray Jenkins 7594f5c0f9
Mako add distributed tracing client (#7180)
* Add support for specifying distributed client tracer in Mako.

Introduces a new command line option --distributed_tracer_client. Defaults to disabled, user may specify
disabled, log_file, network_lossy, or sim_end.

Testing locally and verified with distributed tracing receiver. Note: We use a simple integer here
to denote tracer type rather directly using the TracerType enum in flow/Tracing.h. Including Tracing.h pulls in
FDBTypes.h and results in conflicts with several classes, including Database.
2022-05-17 15:23:14 -05:00
Josh Slocum a43c98519d
Switching char* to std::string for ProcessInfo to have it own memory (valgrind errors) (#7176) 2022-05-17 12:41:04 -07:00
Vaidas Gasiunas 840786ced5
Merge pull request #7177 from sfc-gh-vgasiunas/vgasiunas-api-tester-verbose
Api Tester: print workload progress statistics in regular time intervals
2022-05-17 21:28:50 +02:00
Bharadwaj V.R f43edb6799
Merge branch 'apple:main' into ddspruce 2022-05-17 11:04:59 -07:00
Vaidas Gasiunas 9966e3f7f7 Api Tester: print workload progress statistics in regular time intervals 2022-05-17 17:35:01 +02:00
Xiaoxi Wang 8e2a78bf3c
Merge pull request #7087 from sfc-gh-xwang/features/read-skew
Add SkewedReadWriteWorkload
2022-05-16 16:13:57 -07:00
Alec Grieser 940ca2c6b8
Merge pull request #7166 from imperatorx/keyarrayresult-add-getter
Update KeyArrayResult.java
2022-05-16 14:03:24 -07:00
A.J. Beamon ea57e11e3c
Merge pull request #7168 from sfc-gh-rjenkins/client_tmp_dir-unique-code
client_tmp_dir needs a unique code in bindings.
2022-05-16 14:00:43 -07:00
Jingyu Zhou b5ce29ee5f
Merge pull request #7143 from sfc-gh-anoyes/anoyes/use-libcoro-valgrind
Use libcoro for valgrind
2022-05-16 13:17:48 -07:00
Ray Jenkins 3edc3431fe Update go bindings. 2022-05-16 14:48:40 -05:00
A.J. Beamon 0cdf27a995
Merge pull request #7154 from sfc-gh-svemuri/metacluster-numtenants-status
Add tenant information to status
2022-05-16 12:42:37 -07:00
Ray Jenkins b0ea1a2ce8 client_tmp_dir needs a unique code in bindings.
Duplicate code caused distributed_client_tracer to break.
2022-05-16 13:41:30 -05:00
Sagar Vemuri 290b903825 Do not add tenant-info to json if it cannot be correctly determined 2022-05-16 11:30:03 -07:00
Xiaoxi Wang ef0d49eb93
Update tests/CMakeLists.txt 2022-05-16 11:13:18 -07:00
Sagar Vemuri 7a22a6ea82 Add a null pointer check for extra safety 2022-05-16 11:09:10 -07:00
Sagar Vemuri ebf11d5d48 Update schemas with the tenants information 2022-05-16 11:09:10 -07:00
Sagar Vemuri 7d785fe6ba Add number of tenants to status 2022-05-16 11:09:10 -07:00
imperatorx 5e01c171be
Update KeyArrayResult.java
Add missing getter
2022-05-16 20:02:36 +02:00
Clement Pang 89c5901228 make MappedKeyValue constructor public 2022-05-16 09:49:50 -07:00
Junhyun Shim 0d86a959c8
Merge pull request #7165 from sfc-gh-jshim/mtls-test-helpers
Fix Mac build failing from boringssl
2022-05-16 14:32:26 +02:00
Junhyun Shim 3fe23ce666 Merge remote-tracking branch 'origin/main' into mtls-test-helpers 2022-05-16 13:19:45 +02:00
Junhyun Shim 5308a2727f Fix failing Mac build from boringssl
X509V3_EXT_conf_nid is part of 'libdecrepit' in BoringSSL.
Use X509V3_EXT_nconf_nid instead.
2022-05-16 11:58:42 +02:00
Vaidas Gasiunas 194d5d99e3
Merge pull request #7157 from sfc-gh-vgasiunas/vgasiunas-fix-upgrade-test
Restoring checks after each step in upgrade test
2022-05-16 09:12:22 +02:00
Xiaoxi Wang 4f3a7b7e7f refactor ReadWriteWorkload 2022-05-13 15:26:39 -07:00
Bharadwaj V.R 74b43cd834
Merge branch 'apple:main' into ddspruce 2022-05-13 13:42:37 -07:00
Bharadwaj V.R 30922268d7 Minor readability improvement (I believe) to DD code; mostly, replacing .first, .second of pairs with better names through bindings) 2022-05-13 13:03:00 -07:00