Commit Graph

18917 Commits

Author SHA1 Message Date
Trevor Clinkenbeard 6a28bddd35
Fix file names in copyright headers (#6578) 2022-03-14 13:24:25 -07:00
Lukas Joswiak d0da6c63c1 Rollforward out of date nodes, compaction fixes 2022-03-14 11:20:56 -07:00
Steve Atherton 415d1958d4
Build configuration fix: USE_JEMALLOC=OFF had no effect, did not disable using jemalloc on platforms that support it. (#6590) 2022-03-14 11:00:34 -07:00
Vaidas Gasiunas 09bb37ce3e ApiTester: retry a transaction after all futures are ready 2022-03-13 21:37:46 +01:00
sfc-gh-tclinkenbeard 8dcac2f76d Fix typos 2022-03-13 10:02:11 -03:00
Bharadwaj V.R 8be519a5d8 Make getServerMetrics private to TCServerInfo class and rename some methods to remove redundant reference to 'server' 2022-03-12 17:30:50 -04:00
Bharadwaj V.R 7bad6c5093 Make server API for load metrics 2022-03-12 17:30:50 -04:00
Bharadwaj V.R 2d2f7ed2c6 Refactor team space methods and create new TCServerInfo method to expose space metrics 2022-03-12 17:30:50 -04:00
Bharadwaj V.R c5a17f48c0 Make some TCServerInfo methods that are only used in DDTCUnitTest class private 2022-03-12 17:30:50 -04:00
Bharadwaj V.R 7cb5c8957c Change dd doc to match recent encapsulation changes to TC data structures 2022-03-12 17:30:50 -04:00
Vaidas Gasiunas 8ba3c107ff ApiTester: cancel transaction instead of deleting to avoid crashes when it is accessed by pending callbacks 2022-03-12 14:12:37 +01:00
Ben Collins 32f2731773
Fix static initialization order for maxAllowedVersion (#6570)
* Fix static initialization order for maxAllowedVersion

As it was defined, maxAllowedVerion was statically and globally initialized with a dependency on CLIENT_KNOBS. Initialization order was not spelled out and therefore this relied on chance specifics of order. In some cases these two were constructed in the wrong order and led to a segfault on startup. Fix by deferring initialization of maxAllowedVerion.

* Fix formatting with clang-format
2022-03-11 11:02:35 -08:00
Josh Slocum 5d1affa2c2 Cleaning up and improving TSS debugging traces for range read mismatches 2022-03-11 11:38:58 -06:00
Josh Slocum 26e95d43ef SS needs to persist its TSS pair ID to make metrics accurate (and avoid buggify issues in simulation) 2022-03-11 11:38:58 -06:00
Vaidas Gasiunas 3f2c9e5ff8 ApiTester: Pass future vector per copy to continueAfterAll to avoid race conditions in error handing path 2022-03-11 18:00:31 +01:00
sfc-gh-tclinkenbeard 184f0d7586 Improve const correctness of storage wiggler classes 2022-03-11 09:20:53 -04:00
sfc-gh-tclinkenbeard 82f709c9d3 Use CRTP to remove duplicate code from Smoother.h 2022-03-11 08:40:24 -04:00
sfc-gh-tclinkenbeard 3a6568580a Improve encapsulation of TimerSmoother 2022-03-11 07:47:26 -04:00
Mohamed Oulmahdi dc5b129d10
Merge pull request #6581 from Doxense/enable-windows-encryption
Enable encryption for Windows
2022-03-11 11:31:34 +01:00
Mohamed Oulmahdi 9e81945301 Enable encryption for Windows 2022-03-11 10:06:07 +01:00
sfc-gh-tclinkenbeard e00820cdd7 Reduce mutable access to *QueueInfo objects in Ratekeeper.actor.cpp 2022-03-11 01:04:13 -04:00
sfc-gh-tclinkenbeard 9aca5f8011 Mark several methods const 2022-03-11 00:47:35 -04:00
sfc-gh-tclinkenbeard a7bce77987 Mark Smoother::smooth* methods const 2022-03-11 00:44:43 -04:00
sfc-gh-tclinkenbeard 0f8435e407 Make DDTeamCollection::addTeamsBestOf private 2022-03-11 00:11:51 -04:00
sfc-gh-tclinkenbeard 7a5105a9d5 Add type safety to boolean parameters in DDTeamCollection 2022-03-11 00:10:12 -04:00
sfc-gh-tclinkenbeard 01e00f3c7d Make DDTeamCollection::addTeam private 2022-03-10 23:35:56 -04:00
Steve Atherton 30957c3a43 Fix unclear comment. 2022-03-10 14:22:45 -08:00
Ata E Husain Bohra 944ec48415
Introduce a simulate EncryptKeyVaultProxy interface (#6576)
Description

Major changes proposed are:
1. Rename ServerKnob->ENABLE_ENCRYPT_KEY_PROXY to
   ServerKnob->ENABLE_ENCRYPTION. Approach simplifies enabling
   controlling encyrption code change using a single knob (desirable)
2. Implement EncyrptKeyVaultProxy simulated interface to assist
   validating encyrption workflows in simulation runs. The interface
   is leveraged to satisfy "encryption keys" lookup which otherwise
   gets satisfied by integrating organization preferred Encryption
   Key Management solution.

Testing

Unit test to validate the newly added code
2022-03-10 12:06:49 -08:00
Vaidas Gasiunas a9ee2b99e2 ApiTester: fix boost target for sanitizer builds 2022-03-10 19:30:22 +01:00
Vaidas Gasiunas bb7d26a6eb Merge remote-tracking branch 'apple/main' into vgasiunas-system-tester 2022-03-10 19:13:54 +01:00
Vaidas Gasiunas 9e52456eeb ApiTester: Test cancelling transaction with pending futures; Refactor transaction executor to improve concurrency handing and memory management 2022-03-10 19:13:23 +01:00
Tao Lin e2c7c30faf
GetMappedRange support serializable & check RYW & continuation (#6181) 2022-03-10 10:05:44 -08:00
Renxuan Wang 92ce0de404 When storing coordinators string, we should concatenate by comma.
We are splitting by comma (402fa4dd9e/fdbclient/SpecialKeySpace.actor.cpp (L1686)). So when we concatenating, if we use ", ", there will be an extra space, causing hostname parsing error. NetworkAddress happens not to have the same issue because it uses sscanf (402fa4dd9e/flow/network.cpp (L119)).
2022-03-09 14:28:22 -08:00
Yi Wu e6950abae6 Use jemalloc for SQLite/Redwood page cache allocation 2022-03-09 11:48:08 -08:00
Vaidas Gasiunas bd0bf1cfc2 ApiTester: Address concurrency issues in the workload management 2022-03-09 14:38:07 +01:00
Vaidas Gasiunas 8d2c3a4773 ApiTester: Use thread-local random generators 2022-03-09 13:28:12 +01:00
A.J. Beamon 402fa4dd9e
Merge pull request #6428 from sfc-gh-ajbeamon/fdb-tenant-add-interfaces
Add server-side and protocol support for tenants
2022-03-08 15:33:18 -08:00
A.J. Beamon 63b8b838a2
Merge pull request #6558 from sfc-gh-ajbeamon/special-keys-limits
Add support for a limits hint in special keys range reads
2022-03-08 15:32:59 -08:00
A.J. Beamon 693bcb4582
Merge pull request #6565 from sfc-gh-jfu/jfu-fix-conflict-range-bug
Apply stricter condition for special keys check
2022-03-08 14:52:59 -08:00
A.J. Beamon d71e91eea5 Add a comment about limitsHint. 2022-03-08 14:35:22 -08:00
A.J. Beamon 6a2e54d955 Rename OPTIONAL to avoid windows macro conflict 2022-03-08 13:45:29 -08:00
A.J. Beamon 6f7adb04a5 Send a version along with GetKeyServerLocationsRequests to allow the proxy to reject missing tenants more quickly. Change the commit proxy tenant map to be unversioned. 2022-03-08 13:33:56 -08:00
Jon Fu 24d0c158b4 Apply stricter condition for special keys check 2022-03-08 16:22:09 -05:00
A.J. Beamon 5003a0d662 Remove unnecessary parameter from clampRangeToTenant 2022-03-08 12:01:51 -08:00
A.J. Beamon f39a53787f Ignore changes that happen in the system key-space when evaluating whether a transaction should have conflicted in the conflict range test. 2022-03-08 07:46:07 -06:00
neethuhaneesha 212deb05e9
Merge pull request #6501 from neethuhaneesha/rocksdb-CompBytesLimiter
Rocksdb knobs for compaction, storageserver canCommit() waiting if rocksdb overloaded
2022-03-07 14:59:34 -08:00
Markus Pilman 2dc75ce5dc
Merge pull request #6495 from Doxense/fix-ignoring-test-exit-code
Fix ignoring test exit codes
2022-03-07 22:36:06 +01:00
Andrew Noyes a827e1cbd5 Flush all output streams in flushAndExit
Also detect reading past the end of the infile for debug_determinism tracing
2022-03-07 13:25:52 -08:00
Yao Xiao 36edfb8820
Update error type for timeouts. (#6503) 2022-03-07 13:14:20 -08:00
Steve Atherton 8f8f95931b In the SQLite storage engine, destroy and create new cursors after SQLITE_CURSOR_MAX_LIFETIME_BYTES of KV read usage because cursor usage increases page cache memory usage in SQLite (internal page cache, not AsyncFileCached) by pinning pages which are not freed until the cursor is destroyed. 2022-03-07 11:20:59 -08:00