Commit Graph

1824 Commits

Author SHA1 Message Date
Hui Liu 05a0a3a9c8 Fix trace event error 2022-09-30 15:55:43 -07:00
Hui Liu 9db48eb10c fix killMachine - make sure we have at least 1 blob worker in a dc 2022-09-30 11:18:27 -07:00
Xiaoxi Wang e476e814b2 merge upstream/main 2022-09-27 10:15:26 -07:00
Dan Adkins df2c1374cb
Correct a couple of comments in simulator and simulated workloads. (#8310) 2022-09-24 22:52:13 -07:00
Dan Adkins 48e1b06d29 Add comments about the use of protectedAddresses. 2022-09-22 17:01:01 -07:00
Dan Adkins c5366e1116 sim: don't allow calls to killProcess or killInterface to kill protected processes.
killProcess_internal assumes that we've already checked and won't attempt to kill
a protected process. It's up to killProcess and killInterface, which are invoked
externally from simulated workloads, to enforce that constraint.
2022-09-21 13:05:56 -07:00
Junhyun Shim 2e2dd119fe
Increase test coverage of TLS server-client combination (#8057)
Explore all combinations where the server and the client
each takes one of the following TLS/cert-chain configuration:
  No TLS, no chain, valid chain, expired chain, valid self-signed, expired self-signed

Negative cases are indirectly tested using timeouts:
  i.e. if it times out, it means it's not connecting

+ Fix TSAN data race error in original test
  (Calling FlowTransport::bind() with an active network thread running in background)
+ Properly redirect-and-capture subprocess stdout instead of just printing
  (CTest tends to truncate subprocess console outputs)
2022-09-20 13:47:35 +02:00
A.J. Beamon 4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
Steve Atherton 4cbe8be459
Merge pull request #8201 from sfc-gh-satherton/kaio-latency-pctile
Changed AsyncFileKAIO Read/WriteSync latency metrics to LatencySample
2022-09-16 10:46:38 -07:00
sfc-gh-ngoyal 1bd97fe628
Recruit new singleton for consistency checker. (#5804)
* Recruit new singleton for consistency checker.

* Recruit the consistency checker only if enabled.

* Add a yield in monitorConsistencyChecker().

* Minor fixes.

* Consistency check workload enhancements.

* Minor fixes and clarifications.

* clang format

* Clang format.

* Minor fixes, cleanup, debug tracing.

* Misc.

* Move the consistency scan information from dbconfig to a key backed object.

* Move consistency scan config out of db cofig to a state object and feature rename.

* ConsistencyCheck workload refactor.

* devFormat

* Update fdbcli/ConsistencyScanCommand.actor.cpp

* Review Comments.

Co-authored-by: negoyal <neelam.goyal@gmail.com>
Co-authored-by: Ata E Husain Bohra <ata.husain@snowflake.com>
2022-09-16 09:03:06 -07:00
Xiaoxi Wang 7169a8b132 merge upstream/main 2022-09-15 21:27:26 -07:00
Steve Atherton 4817838207 Change now() to timer() but minimize the calls to only once per io_submit() and once per get_events() for more accurate timings which will only measure the time the IO spent in the kernel (including any necessary pre-op truncate). 2022-09-15 20:38:02 -07:00
Steve Atherton 0e0a10ab45 Merge commit '59b04d46cff720df5d267daa8dc8a60c25466f74' into kaio-latency-pctile 2022-09-15 20:27:02 -07:00
Steve Atherton 2bf90ca5ec Change KAIO latency metrics to use LatencySample for easier usability. Rename a SQLite-specific knob to indicate it is specific to SQLite. 2022-09-15 13:27:23 -07:00
sfc-gh-tclinkenbeard 82adc1e856 Make g_simulator a pointer 2022-09-15 09:00:33 -07:00
Lukas Joswiak d6b25d6ce8 Initialize configuration database type in simulator 2022-09-14 14:14:49 -07:00
Xiaoxi Wang 1f1a66be39 merge upstream/main 2022-09-14 12:32:28 -07:00
Lukas Joswiak 424bb87f3e Apply feedback 2022-09-13 16:53:54 -07:00
Lukas Joswiak 74ac617a34 Add support for changing coordinators to the configuration database
Configuration database data lives on the coordinators. When a change
coordinators command is issued, the data must be sent to the new
coordinators to keep the database consistent.
2022-09-13 16:53:54 -07:00
Junhyun Shim b5b2988fd2
Merge pull request #8166 from sfc-gh-jshim/authz-tls-test-set-timeout
Set timeout on Authz TLS unit test
2022-09-13 20:39:39 +02:00
Junhyun Shim 106bacfa6e Set timeout on Authz TLS unit test 2022-09-13 18:22:12 +02:00
Markus Pilman 59ce49913a
Merge pull request #8146 from sfc-gh-tclinkenbeard/improve-code-coverage
Increase the number of unit tests run in `RandomUnitTests.toml`
2022-09-12 15:10:47 -06:00
Markus Pilman 4d8323481c
Merge pull request #8134 from sfc-gh-jshim/authz-code-probe-coverage
Authz TokenCache code probe coverage
2022-09-12 15:09:38 -06:00
Xiaoxi Wang 806d0a8599 merge upstream/main; solve conflicts 2022-09-12 12:34:35 -07:00
Jingyu Zhou 9429ca992c
Merge pull request #8066 from sfc-gh-tclinkenbeard/fix-includes
Use full header file paths in includes
2022-09-12 09:43:24 -07:00
sfc-gh-tclinkenbeard 39c6989673 Remove some simulation unit tests.
These tests should not be run in simulation, because they either run too
long or break determinism.
2022-09-11 00:36:18 -07:00
Junhyun Shim dddc895b30 Merge remote-tracking branch 'origin/main' into authz-code-probe-coverage 2022-09-09 19:40:18 +02:00
Junhyun Shim 727e000b2a Apply clang format 2022-09-09 15:31:29 +02:00
Junhyun Shim 89800dbb03 Cover TokenCache code probes unhit by unit test
* Token with invalid signature
* Valid token doesn't reference tenant
2022-09-09 15:30:07 +02:00
Junhyun Shim 3023096962 Add a knob to allow token-less tenant data access for untrusted clients 2022-09-08 14:53:01 +02:00
Xiaoxi Wang 83bb9efa8d format if() endif() 2022-09-07 09:31:29 -07:00
Xiaoxi Wang 12d6dc8589 Don't add test authz_tls_unittest when OPEN_FOR_IDE=ON 2022-09-07 09:31:29 -07:00
sfc-gh-tclinkenbeard 3dec9749d3 Mark PromiseStream::send non-const 2022-09-02 10:41:56 -07:00
Josh Slocum 663b1b9711 Handling request_maybe_delivered in reply promise stream and injecting faults to make sure the code path is exercised 2022-09-01 12:57:09 -05:00
Xiaoxi Wang b18561bc31 Merge branch 'main' of https://github.com/apple/foundationdb into feature/dd-refactor-simple 2022-09-01 09:29:30 -07:00
Xiaoxi Wang fbb6016c40 pass /MockGlobalState/initialAsEmptyDatabaseMGS 2022-08-31 22:26:10 -07:00
sfc-gh-tclinkenbeard b10a2a58f7 Use full paths in includes 2022-08-31 14:39:14 -07:00
Junhyun Shim f7ef5edab4 Merge remote-tracking branch 'origin/main' into authz-general-tls-and-integration-test 2022-08-29 16:35:57 +02:00
Junhyun Shim b4e428fa33 Address review comments
- Revert JWT timestamp from double back to uint64_t
- Describe in comment the need for wait loop at the beginning of watchPublicKeyJwksFile()
- Revert CLI args definition in fdbserver.actor.cpp that was
  incorrectly tabbed with a bad clang-format run
2022-08-29 12:49:03 +02:00
Junhyun Shim c2f577dac8 Add key type and DS algorithm dimension to TokenSign benchmark
Also
- Measure performance with RSA keys
- Make RSA key 2048 > 4096 bits
2022-08-26 22:58:36 +02:00
Jingyu Zhou 29e2b705c4 Suppress AllLocalAlternativesFailed events 2022-08-26 13:55:16 -07:00
Jingyu Zhou 37a367b965
Merge pull request #7698 from RenxuanW/main
Complete the proxy support for backup to S3.
2022-08-26 10:32:22 -07:00
Junhyun Shim 9a13f6d2cf Include signing performance in TokenSign benchmark 2022-08-26 15:32:35 +02:00
Junhyun Shim 6bcfbf421e Merge remote-tracking branch 'origin/main' into authz-general-tls-and-integration-test 2022-08-26 09:20:31 +02:00
Jingyu Zhou 05e463f79f
Merge pull request #7994 from jzhou77/main
Fix missing localities for fdbserver
2022-08-25 16:01:33 -07:00
Jingyu Zhou 808182e258 Add more traces for load balancing 2022-08-25 14:25:11 -07:00
Junhyun Shim 3180dc23f6 Move Authz TLS test from fdbrpc/ to fdbrpc/tests
fdb_find_sources CMake command adds all sources directly under fdbrpc/,
which would end up embedding the test binary into libfdbrpc.a
Avoid this by placing the test binary under subdirectory fdbrpc/tests/
2022-08-25 19:26:16 +02:00
Chaoguang Lin 06aa6ee5ff
Add system monitor for flowprocess (#6925)
* Update network address in trace logs; Add system monitor for flowprocess

* Create a new trace file with the correct process address for flowprocess

* Remove unused debugging traces

* Add a new error lock_file_failure; Change please_reboot_remote_kv_store to please_reboot_kv_store; Add the code to only reboot the kv store but not the worker; Remove some unnecessay traces

* Add error handling for file_not_found in handleIOErrors

* Format worker.actor.cpp file
2022-08-24 00:40:38 -07:00
Junhyun Shim 1f5e7d7817 Do float-aware equality comparison with token timestamps 2022-08-24 00:47:05 +02:00
Junhyun Shim e264d39ed8 Work around failing test build on Windows
IF(NOT WIN32) add_flow_target() doesn't seem to keep AuthzTlsTest.cpp
from being considered for build. Wrap it in #ifndef to work around the problem.
2022-08-23 22:04:28 +02:00