Commit Graph

21545 Commits

Author SHA1 Message Date
Nim Wijetunga 33f236a068 merge 2022-07-26 10:37:26 -07:00
A.J. Beamon d31324102d Add comments explaining the use of the TransactionState tenant() and hasTenant() functions 2022-07-26 10:17:57 -07:00
Nim Wijetunga 7cc58cb1af fix pr issues 2022-07-26 10:14:59 -07:00
Xiaoxi Wang 47af598749
Merge pull request #7701 from liquid-helium/resume-data-move-timing
Make sure resumeFromDataMoves() starts after resumeFromShards().
2022-07-26 09:45:14 -07:00
A.J. Beamon e81ed7ce51 Include encode.h in decode.h to get access to the BUFFERSIZE macro 2022-07-26 09:29:22 -07:00
A.J. Beamon 59ccdc35ab Disallow configuring the same tenant parameter more than once at a time. Made some slight changes to serialization code for tenants. 2022-07-26 09:04:29 -07:00
A.J. Beamon f2a8af9155 Better API version handling for fdbcli tenant commands 2022-07-26 09:04:29 -07:00
A.J. Beamon 8833c17899 Remove unused function 2022-07-26 09:04:29 -07:00
A.J. Beamon 35fd459002 Some test fixes 2022-07-26 09:04:29 -07:00
A.J. Beamon a64693518a Add support for tenant groups 2022-07-26 09:04:29 -07:00
He Liu e0b8530ad1 Make sure resumeFromDataMoves() starts after resumeFromShards(). 2022-07-25 22:33:24 -07:00
Chaoguang Lin 5045844ea1
Add -r changeclusterkey tool for snapshot restore (#7687)
* Debug version of fdbserver -r changeclusterkey tool

* Remove debug symbols; move function to Coordinaton.actor.cpp

* Format and add traces

* fix comments
2022-07-25 22:12:28 -07:00
Yao Xiao 98bff116a4
Disabled unsupported tests. (#7693) 2022-07-25 21:57:47 -07:00
Zhe Wu 410ad5ff18 TLog track unpopped recovery tag 2022-07-25 20:10:18 -07:00
Nim Wijetunga 26d233c0f8 encryption as a tenant property 2022-07-25 17:19:14 -07:00
Renxuan Wang dc9599f2e9
Get network interfaces ready for https proxy. (#7556)
* Move HTTP from fdbclient/ to fdbrpc/.

* Move md5 and libb64 to contrib/.

* Get network interfaces ready for https proxy.

* Rebase
2022-07-25 17:08:32 -07:00
A.J. Beamon 6719e5a85b
Merge pull request #7676 from liquid-helium/ss-shard-management
Fixed compiler warnings.
2022-07-25 13:28:54 -07:00
A.J. Beamon 7c2ed53aef Fix: during recovery, it was possible for tenant operations to briefly fail because the tenant mode is not known 2022-07-25 13:03:15 -07:00
He Liu 118c2ae806
Fixed memory leak for RocksDb and CheckpointReader. (#7665)
* Fixed memory leak for RocksDb and CheckpointReader.

* Close kvStore at the end of StorageServerCheckpointRestoreTest.
2022-07-25 12:36:29 -07:00
Josh Slocum 6115a406d4 formatting 2022-07-25 13:52:47 -05:00
Jingyu Zhou 88cfaf8793
Merge pull request #7681 from jzhou77/main
Add timeout during Status generation
2022-07-25 11:04:00 -07:00
Kevin Hoxha 058276493f
ddsketch_utility: Add utility scripts for ddsketch (#7602)
* Adds ddsketch_calc.py which implements a class for DDSketch related calculations
* Adds ddsketch_conversion.py for quickly computing a bucket index to a value or vice-versa
* Adds ddsketch_compare.py to compute how similar two ddsketch distributions are
* Adds export_graph.py to graph the ddsketch distribution outputted from mako

The arguments for ddsketch_conversion.py are:
-b, --bucket: the bucket index that we need to calculate the value from (optional)
-v, --value: the value that we need to calculate the bucket index from (optional)
-e, --error_guarantee: the error guarantee for ddsketch (optional, default is 0.005)

The arguments for ddsketch_compare.py are:
--file1: Path to first ddsketch json
--file2: Path to second ddsketch json
--txn1: The transaction type for the first file
--txn2: The transaction type for the second file
--op: The operation name (ex: GRV, GET ...)

The arguments for export_graph.py:
--file: path to ddsketch distribution
--txn, -t: Transaction type from file
--title: title for graph (optional, otherwise "Title" is used)
--savefig: Path to save the image plot (optional)
--op: Which operation to plot
2022-07-25 10:29:33 -07:00
Markus Pilman 80d4c85a2f
Merge pull request #7686 from sfc-gh-jshim/wingdi-name-collision
Fix WinGDI name collision by defining macro NOGDI
2022-07-25 11:26:40 -06:00
Josh Slocum af60e2ea32 Fixed granule purging bug and improved debugging for purging 2022-07-25 12:19:41 -05:00
Jingyu Zhou 72e2933763
Merge pull request #7678 from xis19/clang
fixup! Let clang accepts crc32_wrapper.c
2022-07-25 09:31:07 -07:00
Junhyun Shim 553e6aba75 Make dummy transaction commits tenant-aware
Dummy transaction commits are made when earlier transaction returned
commit_unknown_result or request_maybe_delivered, to affirm the status
of earlier transaction. Without tenant-awareness, these transactions
will fail due to lack of authorization in case of untrusted clients.
2022-07-25 18:20:37 +02:00
Zhe Wu 2fdd9d7407 Add testing mechanism 2022-07-25 09:18:02 -07:00
Zhe Wu 98fbc98274 Error in FetchKey GRV shouldn't fail storageserver 2022-07-25 09:18:02 -07:00
Josh Slocum 01b7e5395e
Merge pull request #7530 from sfc-gh-jslocum/forgot_java_bindings
Java bindings for BG Purge (#6816)
2022-07-25 08:56:51 -05:00
Bharadwaj V.R dcf09fd691
Move DD queue code over to using movement-reasons rather than priority (#7614)
* Use enum variables to invoke Priority checking

* add an explicit isDataMovementForReadBalancing function

* Set up RelocateShard in terms of data movement reason instead of priority

* Remove isMountainChopperPriority

* Remove isDiskRebalancePriority

* Fix formatting

* Fix misnamed DataMovementReason::TEAM_HEALTHY

Co-authored-by: Zhongxing Zhang <zhongxing.zhang@snowflake.com>
2022-07-25 00:50:37 -07:00
Junhyun Shim 2a172e1586 Fix WinGDI name collision by defining macro NOGDI
TenantState::ERROR collides with WinGDI's #define ERROR 0
2022-07-25 09:11:34 +02:00
Jingyu Zhou 16519a9e5f Update status json doc with fetch_storage_wiggler_stats_timeout error 2022-07-24 15:24:21 -07:00
Jingyu Zhou ed8c64f1d3 Reduce a DNS lookup in Status generation 2022-07-24 15:17:46 -07:00
Jingyu Zhou da1865d6ac Add timeout during Status generation
Resolving hostname might be blocking.
2022-07-24 13:49:34 -07:00
Xiaoxi Wang 7b6f1ca712
Merge pull request #7621 from sfc-gh-xwang/feature/dd-refactor-incremental
[DD Testability] Add unittest for resuming infligh relocations
2022-07-23 22:14:45 -07:00
He Liu 03755d2489 Fixed compiler warnings. 2022-07-23 20:06:26 -07:00
Xiaoge Su 8ea3ef5639 fixup! Let clang accepts crc32_wrapper.c 2022-07-23 16:38:10 -07:00
Josh Slocum ee1b0cdd43 making purge failures fail test 2022-07-23 13:10:03 -05:00
Andrew Noyes 83a317ef88
Disable guard pages for valgrind build. Closes #7610 (#7672) 2022-07-22 16:51:05 -07:00
Jon Fu 84269bac3e Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys 2022-07-22 16:42:27 -07:00
Jon Fu 3cddeab5c8 fix double counting of config mutations 2022-07-22 16:14:25 -07:00
Jon Fu 1d3a129070 fix compile errors from merge 2022-07-22 16:09:08 -07:00
A.J. Beamon 2b6f349649 Better API version handling for fdbcli tenant commands 2022-07-22 15:37:48 -07:00
Jon Fu 43c7146d86 Merge branch 'feature-tenant-groups' of github.com:sfc-gh-ajbeamon/foundationdb into jfu-tenant-rename-special-keys 2022-07-22 15:21:13 -07:00
A.J. Beamon f6e3019a10 Merge branch 'main' into feature-tenant-groups 2022-07-22 14:09:55 -07:00
A.J. Beamon 9459d0c6ca
Merge pull request #7661 from sfc-gh-ajbeamon/tenant-metadata-key-backed-types
Use key-backed types for tenants
2022-07-22 14:01:45 -07:00
Jon Fu 818a9fb057 fix some test logic to account for some missing error handling in special keys and txn case 2022-07-22 13:24:59 -07:00
Josh Slocum ea9018460a cleanup and polish 2022-07-22 15:13:32 -05:00
A.J. Beamon 0d7d775547 Remove unused function 2022-07-22 12:52:00 -07:00
Andrew Noyes 79f60691e9
Don't pass non-pod to variadic function (#7668)
fdbserver/storageserver.actor.cpp:654:24: error: cannot pass object of non-trivial type 'const std::string' (aka 'const basic_string<char, char_traits<char>, allocator<char>>') through variadic function; call will abort at runtime [-Wnon-pod-varargs]

Just use fmt instead
2022-07-22 12:51:44 -07:00