32f2731773
Fix static initialization order for maxAllowedVersion ( #6570 )
...
* Fix static initialization order for maxAllowedVersion
As it was defined, maxAllowedVerion was statically and globally initialized with a dependency on CLIENT_KNOBS. Initialization order was not spelled out and therefore this relied on chance specifics of order. In some cases these two were constructed in the wrong order and led to a segfault on startup. Fix by deferring initialization of maxAllowedVerion.
* Fix formatting with clang-format
2022-03-11 11:02:35 -08:00
5d1affa2c2
Cleaning up and improving TSS debugging traces for range read mismatches
2022-03-11 11:38:58 -06:00
944ec48415
Introduce a simulate EncryptKeyVaultProxy interface ( #6576 )
...
Description
Major changes proposed are:
1. Rename ServerKnob->ENABLE_ENCRYPT_KEY_PROXY to
ServerKnob->ENABLE_ENCRYPTION. Approach simplifies enabling
controlling encyrption code change using a single knob (desirable)
2. Implement EncyrptKeyVaultProxy simulated interface to assist
validating encyrption workflows in simulation runs. The interface
is leveraged to satisfy "encryption keys" lookup which otherwise
gets satisfied by integrating organization preferred Encryption
Key Management solution.
Testing
Unit test to validate the newly added code
2022-03-10 12:06:49 -08:00
e2c7c30faf
GetMappedRange support serializable & check RYW & continuation ( #6181 )
2022-03-10 10:05:44 -08:00
402fa4dd9e
Merge pull request #6428 from sfc-gh-ajbeamon/fdb-tenant-add-interfaces
...
Add server-side and protocol support for tenants
2022-03-08 15:33:18 -08:00
63b8b838a2
Merge pull request #6558 from sfc-gh-ajbeamon/special-keys-limits
...
Add support for a limits hint in special keys range reads
2022-03-08 15:32:59 -08:00
d71e91eea5
Add a comment about limitsHint.
2022-03-08 14:35:22 -08:00
6a2e54d955
Rename OPTIONAL to avoid windows macro conflict
2022-03-08 13:45:29 -08:00
6f7adb04a5
Send a version along with GetKeyServerLocationsRequests to allow the proxy to reject missing tenants more quickly. Change the commit proxy tenant map to be unversioned.
2022-03-08 13:33:56 -08:00
212deb05e9
Merge pull request #6501 from neethuhaneesha/rocksdb-CompBytesLimiter
...
Rocksdb knobs for compaction, storageserver canCommit() waiting if rocksdb overloaded
2022-03-07 14:59:34 -08:00
8f8f95931b
In the SQLite storage engine, destroy and create new cursors after SQLITE_CURSOR_MAX_LIFETIME_BYTES of KV read usage because cursor usage increases page cache memory usage in SQLite (internal page cache, not AsyncFileCached) by pinning pages which are not freed until the cursor is destroyed.
2022-03-07 11:20:59 -08:00
58f5cea39b
fix typo
2022-03-07 11:20:24 -06:00
04f678d515
Fixing sample logic
2022-03-07 11:20:24 -06:00
665e182bcc
TSS Metrics improvements
2022-03-07 11:20:24 -06:00
3ae98189af
Server processes send unknown_tenant responses back to clients, which is meant to be retried after updating the tenant cache. Fix bug where key selectors could be truncated after applying the tenant prefix.
2022-03-06 21:54:21 -08:00
72a34945ce
Add the ability to disable tenants. Server processes verify the ID of tenants being read or written.
2022-03-06 21:54:21 -08:00
b4feddea8d
Fix a couple bugs. Don't store old versions in tenant map indefinitely.
2022-03-06 21:54:21 -08:00
c630cb45cd
Fix formatting
2022-03-06 21:54:21 -08:00
e898e59d18
Fix merge issue that caused tenant serialization problems. Fix case where arena was passed by value rather than reference. Watches should check the tenant map at the latest version rather than the watch version.
2022-03-06 21:54:21 -08:00
c99c7cf55f
Apply tenant prefixes on the storage server. The commit proxy can serve tenant location requests. Fix some bugs.
2022-03-06 21:54:21 -08:00
b8192828d9
Add support for the global tenant prefix to tenant map entries
2022-03-06 21:54:21 -08:00
9c4773164e
Add ID to requests
2022-03-06 21:54:21 -08:00
5fa9d3e1b7
Add a tenant parameter to read and commit requests. Store a map of all tenants on commit proxy and storage servers. Add an option to require tenant mode.
2022-03-06 21:54:21 -08:00
36435af4f5
Add support for a limits hint in special keys. Fix a typo and resolve some comment formatting weirdness.
2022-03-06 21:52:39 -08:00
ddb42f33e5
Merge pull request #6475 from sfc-gh-ajbeamon/fix-trace-double-escaping
...
Fix several instances where trace events were double-escaping strings
2022-03-04 16:10:49 -08:00
1a5bf25b5c
Update code base to use fmt 8.1.1
2022-03-04 15:52:06 -08:00
41d1e4b5d5
Fix several instances where we were converting keys, ranges, and key selectors into strings before passing them into trace events. This had the effect of double escaping certain characters.
2022-03-04 13:58:36 -08:00
1555c0e2aa
Fix conflict between two recent merges by using the new errorUnsuppressed function.
2022-03-04 12:41:21 -08:00
8796a763a5
Rocksdb knobs for compaction, storageserver canCommit() waiting if rocksdb overloaded.
2022-03-04 12:41:17 -08:00
cdebda35ab
Merge pull request #5725 from sfc-gh-jfu/jfu-grv-cache
...
Add transaction option for clients to use cached read versions
2022-03-04 09:17:27 -08:00
49bd0c34cc
Merge pull request #6493 from neethuhaneesha/rocks-compaction
...
RocksDB increasing background threads to speedup compaction.
2022-03-04 07:24:19 -08:00
83e0368eaa
RocksDB increasing background threads to speedup compaction.
2022-03-03 15:14:37 -08:00
e73c0a31e6
add wiggle_server_ids and wiggle_server_addresses in status json
2022-03-02 10:03:23 -08:00
5e39da7741
fixup! Reformat the source
2022-02-28 18:03:52 -08:00
02e396274f
Fix the compile error on NativeAPI.actor.cpp
2022-02-28 18:03:52 -08:00
aadd706a0a
Port back addressed comments for fixing get key location issue
2022-02-28 16:08:57 -08:00
fe957deef8
Merge pull request #6399 from sfc-gh-bvr/fdb#4271
...
Introduce a new server knob and use it to test if storage servers are…
2022-02-28 13:02:23 -08:00
233c918ffb
Replace printf() and fprintf() with fmt::print().
2022-02-25 19:06:57 -08:00
f7eb66441d
Try eliminating warnings in macOS and Windows CI builds.
...
MacOS warnings are format warnings, e.g., `format specifies type 'long' but the argument has type 'Version' (aka 'long long')`.
Windows warnings are `ACTOR does not contain a wait() statement`.
2022-02-25 19:06:57 -08:00
7a9217a392
Add contrib/debug_determinism ( #6389 )
...
* Add contrib/debug_determinism
Add an instrumentation-based technique for debugging unseen mismatches. Also guard a few existing sources of nondeterminism that don't affect unseen with the DEBUG_DETERMINISM macro.
Also change the simulated run loop to not run as the only task inside the real run loop, since that was a source of nondeterminism.
Also fix nondeterminism from calling timer_int
* Add StorageMetadataType::currentTime
Basically a deterministic-in-simulation version of timer_int that we can
use instead of timer_int for StorageMetadataType::createdTime
2022-02-25 12:54:31 -08:00
06b1d06d38
Support hostname in coordinators commands.
2022-02-24 23:02:29 -08:00
70d723f70c
Move the construction of ConnectionString in changeQuorumChecker() to coordinatorsCommitActor().
2022-02-24 23:02:29 -08:00
3e761aef3b
Add resetConnectionString() function to ClusterConnectionString.
2022-02-24 23:02:29 -08:00
0bdae35a2c
Address comments and move updateFailedEndpointRefreshTime
2022-02-24 16:35:30 -08:00
9c99c59b0c
Fix get key location overloading proxies
2022-02-24 16:35:30 -08:00
250a88e682
Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement.
2022-02-24 12:25:52 -08:00
f14e08a991
addRocksDBPerfContextMetrics
2022-02-23 22:29:07 -05:00
9ab21fe873
remove unused knob
2022-02-23 13:49:53 -05:00
102dcb30c6
Merge branch 'main' into fdb#4271
2022-02-23 10:04:19 -08:00
ce1d71472d
Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache
2022-02-23 12:35:25 -05:00
Ben Collins