Commit Graph

20915 Commits

Author SHA1 Message Date
sfc-gh-tclinkenbeard 840dac1fa3 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-22 22:17:33 -07:00
Lukas Joswiak 75423a100c Move shared_ptr to save a reference increment and decrement 2022-06-22 14:50:17 -07:00
Lukas Joswiak 4f2b1807e4 Use shared_ptr to track initialization across threads 2022-06-22 14:50:17 -07:00
Lukas Joswiak 1b1a9d4df5 Initialize on main thread 2022-06-22 14:50:17 -07:00
Lukas Joswiak 88557d9169 Simplify function call when transaction is null 2022-06-22 14:50:17 -07:00
Lukas Joswiak b80ed948f1 Check initialization status before accessing field 2022-06-22 14:50:17 -07:00
Lukas Joswiak 4c2bb0b44e Fix undefined behavior from accessing field of uninitialized object 2022-06-22 14:50:17 -07:00
Johannes Scheuermann 7bff4af14a
Initial support for Prometheus endpoint and pprof for debugging (#7359)
* Initial support for Prometheus endpoint and pprof for debugging
2022-06-22 08:07:48 +01:00
Ata E Husain Bohra e1ca0ef9a2
Defer recoveredDiskFiles wait if Encryption data at-rest is enabled (#7414)
* Defer recoveredDiskFiles wait if Encryption data at-rest is enabled

Description

In the current code ClusterController startup wait for 'recoveredDiskFiles'
future to complete before triggered 'clusterControllerCore' actor, which
inturn starts 'EncryptKeyProxy' (EKP) actor resposible to fetch/refresh
encryption keys needed for ClusterRecovery as well interactions with
KMS.

Patch addresses a circular dependency where StorageServer initialization
depends on EKP, but, CC doesn't recruit EKP till 'recoveredDiskFiles' completes
which includes SS initialization. Given 'recoveredDiskFiles' is an optimization,
the patch proposes deferring the 'recoveredDiskFiles' future completion until
new Master recruitment is done as part of ClusterRecovery (unblock EKP singleton)

Testing

Ran 500K correctness runs: 20220618-055310-ahusain-foundationdb-61c431d467557551
Recorded failures doesn't seems to be related to the change.
2022-06-21 18:18:57 -07:00
Bharadwaj V.R 8cf2be030f
Build a TenantCache for use by DD (#7207)
* Add an DD tenant-cache-assembly actor
* Add basic tenant list monitoring for tenant cache. 
* Update DD tenant cache refresh to be more efficient and unit-testable
* Remove the DD prefix in the tenant cache class name (and associated impl and UT class names); there is nothing specific to DD in it; DD uses it; other modules may use it in the future
* Disable DD tenant awareness by default
2022-06-21 16:29:30 -07:00
Lukas Joswiak 9ca8a3c683 Reenable status json for dynamic knobs, add unit test 2022-06-21 11:43:05 -07:00
Johannes Scheuermann 4b0c4a32b0
Add testing for Kubernetes sidecar (#7105)
* Refactor python sidecar and add unit tests

* Fix issue trying to send error response multiple times

* Fix imports and TLS handling

* Correct config variable in ssl reload
2022-06-21 19:39:53 +01:00
Johannes M. Scheuermann 8da4bb9d07 Add openssl for debugging in container image 2022-06-21 12:14:06 -05:00
sfc-gh-tclinkenbeard 2391e58fb2 Merge remote-tracking branch 'origin/main' into global-tag-throttling3 2022-06-21 10:09:15 -07:00
Dan Lambright c48d569024
fix a fault injection bug in txn store recovery (#7405)
* fix a fault injection bug in txn store recovery

* Update LogSystemDiskQueueAdapter.actor.cpp

typo

* recoverLoc can be overwritten, so on reset use the stored range start
2022-06-21 12:33:58 -04:00
sfc-gh-tclinkenbeard bcd8785767 Remove old debug tracing 2022-06-20 00:59:22 -07:00
sfc-gh-tclinkenbeard a025d7bdbc Ignore GlobalTagThrottling.toml test 2022-06-19 23:10:54 -07:00
Josh Slocum 34e6a8f942
Merge pull request #7399 from sfc-gh-jslocum/bg_tenant_improvements
Bg tenant improvements
2022-06-17 11:19:41 -05:00
Markus Pilman 5aacaf891c
Merge pull request #7321 from sfc-gh-ajbeamon/multiple-tenant-creation
Support creating multiple tenants in the same transaction
2022-06-17 10:10:09 -06:00
Josh Slocum 1cc466e068 fixes and review comments 2022-06-17 08:17:44 -05:00
Trevor Clinkenbeard b7e4d5440d
Merge pull request #7369 from sfc-gh-tclinkenbeard/fix-unused-var
Fix `-Wunused-but-set-variable` warning in `DDSketchBase::percentile`
2022-06-16 23:42:11 -07:00
sfc-gh-tclinkenbeard 111e28d0ea Merge remote-tracking branch 'origin/main' into fix-unused-var 2022-06-16 17:20:18 -07:00
sfc-gh-tclinkenbeard 308e0113a8 Update fdbcli quota usage output 2022-06-16 16:26:45 -07:00
Xiaoxi Wang 6bb4e341f9
Merge pull request #7110 from sfc-gh-xwang/features/ppw-pause-state
Adding paused/running wiggling status to status json and also the last running/paused timestamp
2022-06-16 14:27:18 -07:00
sfc-gh-tclinkenbeard 99d243197e Add fdbcli quota command 2022-06-16 14:07:16 -07:00
Xiaoxi Wang a311cc28cc solve some comments 2022-06-16 11:07:21 -07:00
Josh Slocum b3597ef3a8 Added plumbing for tenant-aware purge granules 2022-06-16 13:04:34 -05:00
Jon Fu b891b424ea
Merge pull request #7387 from sfc-gh-jfu/jfu-mako-tenant-rows
When provided tenants in mako, divide number of rows by number of tenants.
2022-06-16 12:55:44 -04:00
sfc-gh-tclinkenbeard fabc751b0d Add /GlobalTagThrottler/UpdateQuota unit test 2022-06-15 23:32:39 -07:00
sfc-gh-tclinkenbeard 1cfd019382 Add /GlobalTagThrottler/SkewedMultiClientActiveThrottling unit test 2022-06-15 23:06:01 -07:00
sfc-gh-tclinkenbeard dd08c2b180 Add multiclient unit tests for GlobalTagThrottler 2022-06-15 22:57:15 -07:00
sfc-gh-tclinkenbeard 1505ef86db Mark RkTagThrottleCollection::RkTagThrottleData::getTargetRate const 2022-06-15 21:33:23 -07:00
sfc-gh-tclinkenbeard 5c2bcb275a Add GlobalTagThrottlerTesting namespace 2022-06-15 19:59:18 -07:00
sfc-gh-tclinkenbeard 8dccf7c10c Added /GlobalTagThrottler/MultiTagThrottling unit test 2022-06-15 19:39:58 -07:00
sfc-gh-tclinkenbeard d692516f64 Add /GlobalTagThrottler/WriteThrottling unit test 2022-06-15 19:28:25 -07:00
sfc-gh-tclinkenbeard a423fc77db Use multiple storage servers for GlobalTagThrottler unit tests 2022-06-15 19:15:57 -07:00
sfc-gh-tclinkenbeard bef2f5dfaf Recalculate rate in each iteration of testClient 2022-06-15 19:12:25 -07:00
sfc-gh-tclinkenbeard 9179382fc3 Add TestStorageServers class to mock storage servers for GlobalTagThrottler testing 2022-06-15 19:08:34 -07:00
sfc-gh-tclinkenbeard f18dbedb68 Add /GlobalTagThrottler/ActiveThrottling unit test 2022-06-15 17:52:40 -07:00
sfc-gh-tclinkenbeard da452afde4 Add testGetTPSLimit function for global tag throttler unit testing 2022-06-15 17:38:37 -07:00
sfc-gh-tclinkenbeard 0216740c0c Add /GlobalTagThrottler/Simple unit test 2022-06-15 17:21:54 -07:00
sfc-gh-tclinkenbeard 77e3d81f55 Track per-tag rates separately for each storage server in GlobalTagThrottler 2022-06-15 15:07:07 -07:00
Andrew Noyes 83aceb216c
Use absl::GetStackTrace for slow task profiler (#7374)
* Make SlowTask workload runnable in joshua

* Remove SignalSafeUnwind, and use absl::GetStackTrace for slow task profiler
2022-06-15 14:53:52 -07:00
Andrew Noyes 0fea3fb731
Save a bunch of copies in the trace thread (#7392)
Currently, a std::string is copied unnecessarily for every key and value
in a trace event.

This actually showed up in a jemalloc heap profile while I was
investigating something unrelated. I was surprised to see it since these
allocations should have a very short lifetime.
2022-06-15 12:29:15 -07:00
Jon Fu a96928be2d Merge branch 'main' of github.com:apple/foundationdb into jfu-mako-tenant-rows 2022-06-15 12:15:23 -07:00
Johannes Scheuermann c9b4ff3302
Add support for lumberjack in logging and update example to 7.1 (#7357) 2022-06-15 19:41:01 +01:00
Jingyu Zhou a32127a0b9
Merge pull request #7391 from sbodagala/main
Do not always try to figure out the sequencer locality
2022-06-15 13:50:55 -04:00
Ata E Husain Bohra 9396b691b7
Generate GNU compatible build-id for mockkms golang binary (#7389)
* Generate GNU compatible build-id for mockkms golang binary

Description

 diff-1: Fix compilation issue

Generate GNU compatible build-id for mockkms golang binary
Leverage "cgo" to generate build-id

Testing

Debian package build, verified the GNU build-id
2022-06-15 10:43:46 -07:00
Sreenath Bodagala 2c85bb71c1 - Do not try to figure out the sequencer locality if knob
ENABLE_VERSION_VECTOR_HA_OPTIMIZATION is disabled.
2022-06-15 16:08:31 +00:00
Ata E Husain Bohra 8808d93813
Fix bugs in EncyrptKeyProxy actor (#7388)
Description

Major changes include:
1. GetEncryptByKeyIds cache elements can expire.
2. Update iterator after erasing an element during refresh encryption keys
   operation.

Testing

EncryptKeyProxyTest
2022-06-14 21:22:25 -07:00