Commit Graph

21402 Commits

Author SHA1 Message Date
Nim Wijetunga 50391c35b1 Merge branch 'main' of github.com:sfc-gh-nwijetunga/foundationdb into nim/tenant-encryption-property
* 'main' of github.com:sfc-gh-nwijetunga/foundationdb: (42 commits)
  Get ShardedRocks ready for simulation test. (#7679)
  fixing specific unit test
  formatting
  addressing review comments
  Fix incorrect deserialization of FdbClientLogEvents::Event (#7707)
  Fix a crash bug during CC shutdown process (#7705)
  addressing review comments
  remove runAfter
  Add comments explaining the use of the TransactionState tenant() and hasTenant() functions
  Make sure resumeFromDataMoves() starts after resumeFromShards().
  Fix: during recovery, it was possible for tenant operations to briefly fail because the tenant mode is not known
  formatting
  Fixed granule purging bug and improved debugging for purging
  making purge failures fail test
  cleanup and polish
  Bug fix and cleanup
  First version of key-sorted delta files
  Added full granule read unit test
  Completed delta format unit test
  delta file test and delta generation
  ...
2022-07-27 08:13:30 -07:00
Nim Wijetunga c0e13b8ae2
Merge pull request #1 from apple/main
merge
2022-07-27 08:12:52 -07:00
A.J. Beamon dec6dbfbfb
Merge pull request #7549 from sfc-gh-ajbeamon/feature-tenant-groups
Add support for tenant groups
2022-07-27 07:56:27 -07:00
Josh Slocum cf0fadab85
Merge pull request #7655 from sfc-gh-jslocum/improved_cf_testing
Improved change feed testing
2022-07-27 08:04:37 -05:00
He Liu edbc373815
Get ShardedRocks ready for simulation test. (#7679)
* Disabled tests for ShardedRocks.

Cleaned up ShardedRocks TraceEvent.

Added assertion in ShardManager::validate().

* Added test trace.

* Make sure TraceEvent contains `ShardedRocks`.

Co-authored-by: He Liu <heliu@apple.com>
2022-07-26 21:49:33 -07:00
Josh Slocum 1ea43aa878 Merge branch 'main' into improved_cf_testing 2022-07-26 18:12:58 -05:00
Josh Slocum 77956dc7ae
Merge pull request #7639 from sfc-gh-jslocum/cf_metadata_rewrite
Change Feed Metadata Rewrite and adding targeted fault injection
2022-07-26 18:10:37 -05:00
Josh Slocum 35056060b0 fixing specific unit test 2022-07-26 17:05:46 -05:00
Josh Slocum 8b5b7a9b92
Merge pull request #7689 from sfc-gh-jslocum/purge_fixes
Granule Purge Fixes
2022-07-26 17:02:24 -05:00
Josh Slocum 5c2681f0b5 formatting 2022-07-26 16:47:07 -05:00
Josh Slocum 0609aa57d1 Merge branch 'main' into improved_cf_testing 2022-07-26 16:44:13 -05:00
Josh Slocum ba31229797 addressing review comments 2022-07-26 16:37:06 -05:00
Josh Slocum 4d2f90977d
Merge pull request #7656 from sfc-gh-jslocum/cf_bw_operational_fixes
Cf bw operational fixes
2022-07-26 16:24:26 -05:00
Josh Slocum c32e1da908
Merge pull request #7673 from sfc-gh-jslocum/delta_files_v2
Sorted Delta Files
2022-07-26 16:04:55 -05:00
A.J. Beamon 4c4eb3fc48
Merge pull request #7704 from sfc-gh-ajbeamon/add-comments-to-tenant-function
Add comments explaining the use of the TransactionState tenant() and hasTenant() functions
2022-07-26 14:01:50 -07:00
Xiaoxi Wang 7125902714
Merge pull request #7709 from sfc-gh-xwang/fix/main/remove-runAfter
remove runAfter actor
2022-07-26 13:38:56 -07:00
Andrew Noyes de331ba080
Fix incorrect deserialization of FdbClientLogEvents::Event (#7707)
We're trying to interpret an Optional<TenantName> here, but the
python truthiness of self.get_bytes(1) is _always_ true, since it's
non-empty. We need to look at the _contents_ of that byte instead.
2022-07-26 13:10:31 -07:00
Jingyu Zhou 698360daf0
Fix a crash bug during CC shutdown process (#7705)
actor_cancel should be thrown, otherwise the rest code can access invalid
memory.
2022-07-26 12:45:39 -07:00
Josh Slocum 15e7a4b186 addressing review comments 2022-07-26 14:20:35 -05:00
Xiaoxi Wang 7cf6037191 remove runAfter 2022-07-26 12:12:56 -07:00
Nim Wijetunga 5fbebac26c modify test 2022-07-26 11:53:07 -07:00
A.J. Beamon 4c36a137c3
Merge pull request #7692 from sfc-gh-ajbeamon/fix-tenants-disabled-during-recovery
Fix: during recovery, it was possible for tenant operations to briefly fail because the tenant mode was unknown
2022-07-26 10:50:18 -07:00
Nim Wijetunga 7af4a103f9 fix tests 2022-07-26 10:45:27 -07:00
Nim Wijetunga 33f236a068 merge 2022-07-26 10:37:26 -07:00
A.J. Beamon d31324102d Add comments explaining the use of the TransactionState tenant() and hasTenant() functions 2022-07-26 10:17:57 -07:00
Nim Wijetunga 7cc58cb1af fix pr issues 2022-07-26 10:14:59 -07:00
Xiaoxi Wang 47af598749
Merge pull request #7701 from liquid-helium/resume-data-move-timing
Make sure resumeFromDataMoves() starts after resumeFromShards().
2022-07-26 09:45:14 -07:00
A.J. Beamon e81ed7ce51 Include encode.h in decode.h to get access to the BUFFERSIZE macro 2022-07-26 09:29:22 -07:00
A.J. Beamon 59ccdc35ab Disallow configuring the same tenant parameter more than once at a time. Made some slight changes to serialization code for tenants. 2022-07-26 09:04:29 -07:00
A.J. Beamon f2a8af9155 Better API version handling for fdbcli tenant commands 2022-07-26 09:04:29 -07:00
A.J. Beamon 8833c17899 Remove unused function 2022-07-26 09:04:29 -07:00
A.J. Beamon 35fd459002 Some test fixes 2022-07-26 09:04:29 -07:00
A.J. Beamon a64693518a Add support for tenant groups 2022-07-26 09:04:29 -07:00
He Liu e0b8530ad1 Make sure resumeFromDataMoves() starts after resumeFromShards(). 2022-07-25 22:33:24 -07:00
Chaoguang Lin 5045844ea1
Add -r changeclusterkey tool for snapshot restore (#7687)
* Debug version of fdbserver -r changeclusterkey tool

* Remove debug symbols; move function to Coordinaton.actor.cpp

* Format and add traces

* fix comments
2022-07-25 22:12:28 -07:00
Yao Xiao 98bff116a4
Disabled unsupported tests. (#7693) 2022-07-25 21:57:47 -07:00
Zhe Wu 410ad5ff18 TLog track unpopped recovery tag 2022-07-25 20:10:18 -07:00
Nim Wijetunga 26d233c0f8 encryption as a tenant property 2022-07-25 17:19:14 -07:00
Renxuan Wang dc9599f2e9
Get network interfaces ready for https proxy. (#7556)
* Move HTTP from fdbclient/ to fdbrpc/.

* Move md5 and libb64 to contrib/.

* Get network interfaces ready for https proxy.

* Rebase
2022-07-25 17:08:32 -07:00
A.J. Beamon 6719e5a85b
Merge pull request #7676 from liquid-helium/ss-shard-management
Fixed compiler warnings.
2022-07-25 13:28:54 -07:00
A.J. Beamon 7c2ed53aef Fix: during recovery, it was possible for tenant operations to briefly fail because the tenant mode is not known 2022-07-25 13:03:15 -07:00
He Liu 118c2ae806
Fixed memory leak for RocksDb and CheckpointReader. (#7665)
* Fixed memory leak for RocksDb and CheckpointReader.

* Close kvStore at the end of StorageServerCheckpointRestoreTest.
2022-07-25 12:36:29 -07:00
Josh Slocum 6115a406d4 formatting 2022-07-25 13:52:47 -05:00
Jingyu Zhou 88cfaf8793
Merge pull request #7681 from jzhou77/main
Add timeout during Status generation
2022-07-25 11:04:00 -07:00
Kevin Hoxha 058276493f
ddsketch_utility: Add utility scripts for ddsketch (#7602)
* Adds ddsketch_calc.py which implements a class for DDSketch related calculations
* Adds ddsketch_conversion.py for quickly computing a bucket index to a value or vice-versa
* Adds ddsketch_compare.py to compute how similar two ddsketch distributions are
* Adds export_graph.py to graph the ddsketch distribution outputted from mako

The arguments for ddsketch_conversion.py are:
-b, --bucket: the bucket index that we need to calculate the value from (optional)
-v, --value: the value that we need to calculate the bucket index from (optional)
-e, --error_guarantee: the error guarantee for ddsketch (optional, default is 0.005)

The arguments for ddsketch_compare.py are:
--file1: Path to first ddsketch json
--file2: Path to second ddsketch json
--txn1: The transaction type for the first file
--txn2: The transaction type for the second file
--op: The operation name (ex: GRV, GET ...)

The arguments for export_graph.py:
--file: path to ddsketch distribution
--txn, -t: Transaction type from file
--title: title for graph (optional, otherwise "Title" is used)
--savefig: Path to save the image plot (optional)
--op: Which operation to plot
2022-07-25 10:29:33 -07:00
Markus Pilman 80d4c85a2f
Merge pull request #7686 from sfc-gh-jshim/wingdi-name-collision
Fix WinGDI name collision by defining macro NOGDI
2022-07-25 11:26:40 -06:00
Josh Slocum af60e2ea32 Fixed granule purging bug and improved debugging for purging 2022-07-25 12:19:41 -05:00
Jingyu Zhou 72e2933763
Merge pull request #7678 from xis19/clang
fixup! Let clang accepts crc32_wrapper.c
2022-07-25 09:31:07 -07:00
Zhe Wu 2fdd9d7407 Add testing mechanism 2022-07-25 09:18:02 -07:00
Zhe Wu 98fbc98274 Error in FetchKey GRV shouldn't fail storageserver 2022-07-25 09:18:02 -07:00