Commit Graph

21820 Commits

Author SHA1 Message Date
Josh Slocum 01b7e5395e
Merge pull request #7530 from sfc-gh-jslocum/forgot_java_bindings
Java bindings for BG Purge (#6816)
2022-07-25 08:56:51 -05:00
Bharadwaj V.R dcf09fd691
Move DD queue code over to using movement-reasons rather than priority (#7614)
* Use enum variables to invoke Priority checking

* add an explicit isDataMovementForReadBalancing function

* Set up RelocateShard in terms of data movement reason instead of priority

* Remove isMountainChopperPriority

* Remove isDiskRebalancePriority

* Fix formatting

* Fix misnamed DataMovementReason::TEAM_HEALTHY

Co-authored-by: Zhongxing Zhang <zhongxing.zhang@snowflake.com>
2022-07-25 00:50:37 -07:00
Junhyun Shim 2a172e1586 Fix WinGDI name collision by defining macro NOGDI
TenantState::ERROR collides with WinGDI's #define ERROR 0
2022-07-25 09:11:34 +02:00
A.J. Beamon 2990294f6a Make the metacluster experimental by updating the name of the command used to configure one. 2022-07-24 19:11:25 -07:00
A.J. Beamon 077003c4f5 Cleanup tenant tombstones periodically 2022-07-24 18:24:19 -07:00
Jingyu Zhou 16519a9e5f Update status json doc with fetch_storage_wiggler_stats_timeout error 2022-07-24 15:24:21 -07:00
Jingyu Zhou ed8c64f1d3 Reduce a DNS lookup in Status generation 2022-07-24 15:17:46 -07:00
Jingyu Zhou da1865d6ac Add timeout during Status generation
Resolving hostname might be blocking.
2022-07-24 13:49:34 -07:00
Xiaoxi Wang 7b6f1ca712
Merge pull request #7621 from sfc-gh-xwang/feature/dd-refactor-incremental
[DD Testability] Add unittest for resuming infligh relocations
2022-07-23 22:14:45 -07:00
He Liu 03755d2489 Fixed compiler warnings. 2022-07-23 20:06:26 -07:00
Xiaoge Su 8ea3ef5639 fixup! Let clang accepts crc32_wrapper.c 2022-07-23 16:38:10 -07:00
Josh Slocum ee1b0cdd43 making purge failures fail test 2022-07-23 13:10:03 -05:00
Andrew Noyes 83a317ef88
Disable guard pages for valgrind build. Closes #7610 (#7672) 2022-07-22 16:51:05 -07:00
Jon Fu 84269bac3e Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys 2022-07-22 16:42:27 -07:00
Jon Fu 3cddeab5c8 fix double counting of config mutations 2022-07-22 16:14:25 -07:00
Jon Fu 1d3a129070 fix compile errors from merge 2022-07-22 16:09:08 -07:00
A.J. Beamon 524c3131c5 Use a more specific error if we reject a tenant property change due to it needing to be moved 2022-07-22 15:43:51 -07:00
A.J. Beamon 2b6f349649 Better API version handling for fdbcli tenant commands 2022-07-22 15:37:48 -07:00
Jon Fu 43c7146d86 Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys 2022-07-22 15:21:13 -07:00
A.J. Beamon 41c1bef3bf Merge branch 'feature-tenant-groups' into feature-metacluster 2022-07-22 14:26:02 -07:00
A.J. Beamon f6e3019a10 Merge branch 'main' into feature-tenant-groups 2022-07-22 14:09:55 -07:00
A.J. Beamon 9459d0c6ca
Merge pull request #7661 from sfc-gh-ajbeamon/tenant-metadata-key-backed-types
Use key-backed types for tenants
2022-07-22 14:01:45 -07:00
Jon Fu 818a9fb057 fix some test logic to account for some missing error handling in special keys and txn case 2022-07-22 13:24:59 -07:00
Josh Slocum ea9018460a cleanup and polish 2022-07-22 15:13:32 -05:00
A.J. Beamon 0d7d775547 Remove unused function 2022-07-22 12:52:00 -07:00
Andrew Noyes 79f60691e9
Don't pass non-pod to variadic function (#7668)
fdbserver/storageserver.actor.cpp:654:24: error: cannot pass object of non-trivial type 'const std::string' (aka 'const basic_string<char, char_traits<char>, allocator<char>>') through variadic function; call will abort at runtime [-Wnon-pod-varargs]

Just use fmt instead
2022-07-22 12:51:44 -07:00
A.J. Beamon 9b66e0c1c7 Fix some subtle differences between branches 2022-07-22 12:51:24 -07:00
A.J. Beamon b94c000ea8 Merge branch 'feature-tenant-groups' into feature-metacluster 2022-07-22 12:39:00 -07:00
Josh Slocum 33053a0a7a Bug fix and cleanup 2022-07-22 14:34:38 -05:00
A.J. Beamon b735bf8eb0 Add comments to deleteTenantTransaction. Fetch the tenant ID from the first transaction in deleteTransaction to use on subsequent retries. 2022-07-22 12:34:27 -07:00
A.J. Beamon 193a0dbf4f Some test fixes 2022-07-22 12:28:42 -07:00
A.J. Beamon b308aa6604 Merge branch 'feature-tenant-groups' into feature-metacluster 2022-07-22 11:38:23 -07:00
Lukas Joswiak 7fb427f4ec Update fdbserver/GrvProxyServer.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-22 10:37:29 -07:00
Lukas Joswiak a26344b877 Update fdbserver/GrvProxyServer.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-22 10:37:29 -07:00
Lukas Joswiak 8395c6cf3f Fix Tuple error 2022-07-22 10:37:29 -07:00
Lukas Joswiak 5fa8e3f7d0 Simplify refresh condition 2022-07-22 10:37:29 -07:00
Lukas Joswiak 703aa1d279 Mess with timeout values 2022-07-22 10:37:29 -07:00
Lukas Joswiak 6c17ca038a Add delay to allow for global config sync time 2022-07-22 10:37:29 -07:00
Lukas Joswiak 40d403ed5f Reduce global configuration system key reads from proxy
Clients now poll the proxy for the latest global config for a specific
version. The proxy now periodically requests the latest global
configuration data and stores it in memory, enabling it to respond
immediately to clients with the appropriate version.
2022-07-22 10:37:29 -07:00
Lukas Joswiak 56dfdbda83 Add migration timeout 2022-07-22 10:37:29 -07:00
Lukas Joswiak c32f3f8746 Update fdbclient/include/fdbclient/CommitProxyInterface.h
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-22 10:37:29 -07:00
Lukas Joswiak e024db85a0 Update fdbserver/GrvProxyServer.actor.cpp
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
2022-07-22 10:37:29 -07:00
Lukas Joswiak 7a48a53778 Perform migration automatically on proxy boot 2022-07-22 10:37:29 -07:00
Lukas Joswiak 2e99d5f6cc Batch global config refresh requests 2022-07-22 10:37:29 -07:00
Lukas Joswiak c33e44b0f4 Proxy GlobalConfig reads through GRV proxies
Clients should avoid reading system keys unless authorized. Under global
config, each client reads from the system keyspace to check for new
global config keys. This commit moves these reads to a server role (the
GRV proxies) and sends the results back to GlobalConfig for an in-memory
update.
2022-07-22 10:37:29 -07:00
Josh Slocum 095a5a4868 First version of key-sorted delta files 2022-07-22 11:43:49 -05:00
He Liu 7a8be255cd
Ss shard management (#7340)
* Storage server shard management with physical shards.

* Cleanup.

* Resolved comments.

* Added `UnlimintedCommitBytes`.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-22 09:30:44 -07:00
Jingyu Zhou c759357687
Merge pull request #7662 from xis19/databaseContext
Add TraceEvent to the construction/destruction of DatabaseContext
2022-07-22 09:29:56 -07:00
A.J. Beamon 237081905d Add support for tenant groups 2022-07-22 09:27:31 -07:00
Xiaoge Su 25d9fa856c fixup! Update code per comments 2022-07-21 23:14:31 -07:00