7d7ce0909f
Restart tests carry forward encryption knobs value ( #7497 )
...
Previously to get around the issue that EKP is not present when restart test switching encryption from on to off and read encrypted data, EKP was made to start in simulation regardless of encryption knob. This PR revert that change, and instead force restart test not to change encryption knob, by passing previous encryption knob through restartInfo.ini file. Also since we don't allow downgrading an encrypted cluster to previous version, disable encryption in downgrade tests.
Also adding an assert to allow reading encrypted mutations only if encryption knob is on. We may reconsider allowing switching encryption on/off for existing cluster, but for now we don't allow it.
2022-07-14 14:45:17 -07:00
b51e9430a5
fix timeout buggify
2022-06-29 22:46:50 -07:00
570dfd6760
add timeout handler
2022-06-29 15:30:37 -07:00
d35445a868
enforce include modularization in cmake
2022-06-23 14:37:35 -06:00
9ca8a3c683
Reenable status json for dynamic knobs, add unit test
2022-06-21 11:43:05 -07:00
6bb4e341f9
Merge pull request #7110 from sfc-gh-xwang/features/ppw-pause-state
...
Adding paused/running wiggling status to status json and also the last running/paused timestamp
2022-06-16 14:27:18 -07:00
a311cc28cc
solve some comments
2022-06-16 11:07:21 -07:00
7acd184a38
Merge pull request #7339 from jzhou77/fix-status-memory
...
Add rss_bytes to process memory and fix available_bytes calculation
2022-06-08 13:10:51 -07:00
b9ff6bc129
Address AJ's comments
2022-06-08 09:38:32 -07:00
bbf8cb4b02
GetEncryptCipherKeys helper function and misc encryption changes ( #7252 )
...
Adding GetEncryptCipherKeys and GetLatestCipherKeys helper actors, which encapsulate cipher key fetch logic: getting cipher keys from local BlobCipherKeyCache, and on cache miss fetch from EKP (encrypt key proxy). These helper actors also handles the case if EKP get shutdown in the middle, they listen on ServerDBInfo to wait for new EKP start and send new request there instead.
The PR also have other misc changes:
* EKP is by default started in simulation regardless of. ENABLE_ENCRYPTION knob, so that in restart tests, if ENABLE_ENCRYPTION is switch from on to off after restart, encrypted data will still be able to be read.
* API tweaks for BlobCipher
* Adding a ENABLE_TLOG_ENCRYPTION knob which will be used in later PRs. The knob should normally be consistent with ENABLE_ENCRYPTION knob, but could be used to disable TLog encryption alone.
This PR is split out from #6942 .
2022-06-07 21:00:13 -07:00
217ba24b6f
Add rss_bytes to process memory and fix available_bytes calculation
...
Since memory is now limited with RSS size, add RSS size in status json for
reporting. Also change how available_bytes is calculated from:
(available + virtual memory) * process_limit / machine_limit
to:
(available memory) * process_limit / machine_limit
2022-06-07 16:44:14 -07:00
13a77dd5a2
change priority knob; change PromiseStream to FutureStream; remove comments; add on_sr check
2022-05-25 17:09:34 -07:00
382f0fc4a2
merge upstream/main
2022-05-17 10:20:51 -07:00
290b903825
Do not add tenant-info to json if it cannot be correctly determined
2022-05-16 11:30:03 -07:00
7d785fe6ba
Add number of tenants to status
2022-05-16 11:09:10 -07:00
2717cee1f9
Merge branch 'features/read-skew' into readaware
2022-05-09 16:12:09 -07:00
ed9ca1e3da
fix future segment fault
2022-05-09 15:14:58 -07:00
e6615c29a3
add perpetual wiggler state to json
2022-05-09 11:51:08 -07:00
13bbd062c4
add storage wiggler state
2022-05-08 22:06:11 -07:00
9f56d6ec38
store future
2022-05-07 22:25:16 -07:00
7c37d172b9
solve some comments
2022-05-03 17:21:08 -07:00
75a90be0dd
refactor Status retrieval
2022-05-03 15:26:24 -07:00
69985ba251
Merge branch 'main' of https://github.com/apple/foundationdb into readaware
2022-05-02 10:53:22 -07:00
9a521ad36f
Merge branch 'main' of https://github.com/apple/foundationdb into ppw-store
2022-05-02 10:48:29 -07:00
c69a07a858
Check in the new Hostname logic. ( #6926 )
...
* Revert #6655 .
20220407-031010-renxuan-c101052c21da8346 compressed=True data_size=31004844 duration=4310801 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=1:04:15 sanity=False started=100047 stopped=20220407-041425 submitted=20220407-031010 timeout=5400 username=renxuan
* Revert #6271 .
20220407-051532-renxuan-470f0fe6aac1c217 compressed=True data_size=30982370 duration=3491067 ended=100002 fail_fast=10 max_runs=100000 pass=100002 priority=100 remaining=0 runtime=0:59:57 sanity=False started=100141 stopped=20220407-061529 submitted=20220407-051532 timeout=5400 username=renxuan
* Revert #6266 .
Remove resolving-related functionalities in connection string. Connection string will be used for storing purpose only, and non-mutable.
20220407-175119-renxuan-55d30ee1a4b42c2f compressed=True data_size=30970443 duration=5437659 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=0:59:31 sanity=False started=100154 stopped=20220407-185050 submitted=20220407-175119 timeout=5400 username=renxuan
* Add hostname to coordinator interfaces.
* Turn on the new hostname logic.
* Add the corresponding change in config txns.
The most notable change is before calling basicLoadBalance(), we need to call tryInitializeRequestStream() to initialize request streams first.
Passed correctness tests.
* Return error when hostnames cannot be resolved in coordinators command.
* Minor fixes.
2022-04-27 21:54:13 -07:00
101b6717e3
remove TSS storage metadata part
2022-04-27 16:22:13 -07:00
a8bc81a09d
add getStoreTypeStr method
2022-04-26 13:35:15 -07:00
e9bf7c0851
fix unknown schema check
2022-04-25 17:23:20 -07:00
2ede89b623
fix getUint64 bug
2022-04-25 14:46:50 -07:00
e8477e15ce
Merge remote-tracking branch 'upstream/main' into readaware
2022-04-25 09:55:52 -07:00
adeaf52db3
Merge branch 'apple:main' into ppw-store
2022-04-25 09:40:45 -07:00
269e94cb71
add store type to status json
2022-04-24 22:53:06 -07:00
8a0ce5bfc7
Rename isInvalidMagic and fix formatting
2022-04-20 09:17:28 -07:00
96ab7c0254
Update fdbserver/Status.actor.cpp
...
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
2022-04-20 08:52:14 -07:00
11f2827c82
Make names even across change and add lowest-compatible-version to status json
2022-04-15 13:57:58 -07:00
ce5bcc0261
Remove system key tracking newest server version
2022-04-15 10:57:11 -07:00
576957d335
Fix version status reporting
2022-04-15 09:52:42 -07:00
3787ddae89
Add trace-based status reporting for server version
2022-04-15 09:04:52 -07:00
d9f7b97315
Add software version to system key for status reports
2022-04-14 07:19:41 -07:00
831bee34f8
Add server version to status json
2022-04-13 08:23:02 -07:00
2f2ece073c
Add sw version tracking to DBCoreState
2022-04-13 08:05:22 -07:00
8d3f851495
merge upstream/mainA
2022-04-12 17:03:09 -07:00
aba9d85560
merge main
2022-04-06 09:57:52 -07:00
60c55e0785
Merge remote-tracking branch 'origin/version-vector-prototype' into vv
2022-04-05 11:17:39 -04:00
cfcf0f152c
Merge branch 'main-4a085fc84' into vv
...
Fix Conflicts:
fdbclient/NativeAPI.actor.cpp
fdbserver/ClusterRecovery.actor.cpp
fdbserver/MasterInterface.h
fdbserver/masterserver.actor.cpp
flow/error_definitions.h
2022-03-30 22:28:06 -07:00
00b57d4cce
Merge branch 'main-67eba5ec7' into vv
...
Fix Conflicts:
fdbclient/CommitProxyInterface.h
fdbclient/NativeAPI.actor.cpp
fdbclient/StorageServerInterface.h
fdbserver/CommitProxyServer.actor.cpp
fdbserver/storageserver.actor.cpp
2022-03-30 20:05:55 -07:00
6744e9e4f9
Change timestamps used in storage server metadata and perpetual wiggle metrics to epoch seconds, stored as doubles, and stringified as either floating point epoch seconds or timestamp strings of the form "2013-04-28 20:57:01.000 +0000".
2022-03-30 18:57:06 -07:00
e9659b5dd4
Merge branch 'master-PR-6500' into vv
...
Fix Conflicts:
fdbclient/CommitProxyInterface.h
fdbclient/NativeAPI.actor.cpp
fdbserver/masterserver.actor.cpp
2022-03-30 14:53:49 -07:00
d93b57dd88
conflict solving
2022-03-24 20:45:51 -07:00
1b631a9263
solve conflict with main
2022-03-24 16:29:11 -07:00
Yi Wu