wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
22,942 Commits 36 Branches 304 Tags 267 MiB
Commit Graph

5857 Commits

Author SHA1 Message Date
A.J. Beamon f8186988bc
Merge pull request #8378 from sfc-gh-ajbeamon/tenant-support-for-database-switching 
Add tenant support for database switching
 2022-10-05 14:57:33 -07:00
A.J. Beamon 09050c0b87
Merge pull request #8405 from sfc-gh-ajbeamon/versionstamp-in-key-backed-object-property 
Support versionstamps in KeyBackedBinaryValue
 2022-10-05 13:45:29 -07:00
Sreenath Bodagala e83229ed8b
- Version vector specific: Propagate the latest commit version, as (#8393) 
part of the read request, when reading from a storage server.
 2022-10-05 16:23:56 -04:00
A.J. Beamon 44ab6046e3 Fix missing operator to get address of offset 2022-10-05 12:55:33 -07:00
A.J. Beamon cfb36bd9c3 Support versionstamps in key backed object properties 2022-10-05 12:48:40 -07:00
Jon Fu 2fe1d19e95 Merge branch 'main' of github.com:apple/foundationdb into metacluster-status 2022-10-05 11:48:04 -07:00
Josh Slocum dc917453c1
Targeted blob granules fault injection (#8231) 2022-10-05 13:44:38 -05:00
Hui Liu 9799329b99
Merge pull request #8390 from sfc-gh-huliu/actor 
Misc changes for blob manifest dumper
 2022-10-05 10:30:56 -07:00
Markus Pilman 23edfd0d59 Fix formatting 2022-10-04 18:33:30 -06:00
Markus Pilman 550488b020 Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide 
# Conflicts:
#	bindings/c/CMakeLists.txt
#	fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
#	fdbserver/BackupWorker.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/CommitProxyServer.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/PhysicalShardMove.actor.cpp
#	flow/CMakeLists.txt
 2022-10-04 18:27:48 -06:00
Markus Pilman 97dfc6823f fixed build with OPEN_FOR_IDE 2022-10-04 17:01:02 -06:00
neethuhaneesha a565863189
RocksDB compaction knobs and stats. (#8392) 2022-10-04 15:13:50 -07:00
Hui Liu 50b3a8d3ba misc changes for manifest dumper 
1) move dumper to a dedicated actor
2) include blob manager epoch into the manifest name
3) keep multiple manifest files(up to 5)
 2022-10-04 10:33:43 -07:00
A.J. Beamon 55b880432e
Merge pull request #8379 from sfc-gh-ajbeamon/literal-string-ref-removal 
LiteralStringRef removal
 2022-10-04 09:41:42 -07:00
A.J. Beamon 53593c77ac When updating watches after a database switch, lookup the tenant again to get the ID on the new cluster. Update the switching test to set up tenants properly. 2022-10-03 09:24:13 -07:00
A.J. Beamon 0a1d03c3d7 Formatting fixes 2022-10-03 09:23:17 -07:00
A.J. Beamon 43dab2baeb Creating string refs from literals needs to remove the terminating null byte 2022-10-03 09:20:15 -07:00
Vaidas Gasiunas 415e1aeb1f
Fix for a memory leak in database operations called after create database errors (#8334) 
* ApiTester: cancel pending futures for non-transactional operations

* Clear future callback in case of abort signal to avoid memory leaks

* Abortable future: Make sure all callbacks are cleared in case of premature signal; addressing comments

* Abortable future: remove debugging asserts

* Abortable future: restore the previous method for cancelling wrapped future

* Removing debugging code
 2022-10-01 11:30:52 +02:00
Nim Wijetunga 778cb63f71
Ensure nullptr is not passed to memcpy (#8376) 
* fix

* fix

* Trigger Build
 2022-09-30 17:28:43 -07:00
A.J. Beamon aaa6dbec2d Delete LiteralStringRef type and write out the final usage using a longer form 2022-09-30 16:08:24 -07:00
A.J. Beamon 217630f327 Add a comparison operator for StringRef and char arrays; use it to get rid of a LiteralStringRef usage 2022-09-30 16:07:26 -07:00
A.J. Beamon 111bf9e34d Add macros for literal string ref versions of __FUNCTION__ and __FILE__ 2022-09-30 16:05:47 -07:00
A.J. Beamon e1fe28b78b Switch some usages of LiteralStringRef to use the _sr suffix 2022-09-30 16:04:16 -07:00
Jon Fu 3ce168f604 add metacluster to status schema 2022-09-30 11:58:41 -07:00
Jingyu Zhou c321d633c7
Merge pull request #8361 from halfprice/zhewu/update-default-gray-latency-percentile 
Update default peer latency degradation percentile to 0.5
 2022-09-30 08:34:31 -07:00
Hui Liu db3b908ce2
Merge pull request #8368 from sfc-gh-huliu/fixunittest 
fix benchFromFiles unit test
 2022-09-29 22:15:56 -07:00
Hui Liu dea6da142d fix unittest benchFromFiles 2022-09-29 21:00:12 -07:00
Jon Fu 461e42bfe1 restructure updater code and add capacity check in metacluster management workload 2022-09-29 16:24:02 -07:00
Ata E Husain Bohra 03f1d13be3
Enable encryption authentication configurability (#8312) 
* Enable encryption authentication configurability

Description

 diff-1: Remove memcpy due to auth-token computation
         Address review comments

Patch proposes major changes:
1. Enable FDB to choose encryption authentication as a configurable
parameter. Fix issues choosing ENCRYPT_HEADER_AUTH_TOKEN_NONE mode.
2. Introduce AES_CMAC as supported encryption authentication scheme.

Patch allows cluster to govern: if encryption authentication needs to
enabled, if yes, then choose from two supported schemes:
1. HMAC_SHA_256
2. AES_256_CMAC

Testing

devRunCorrectness - 100K
BlobCipher unittests
EncryptionOps.toml
BlobGranuleCorrectness/BlobGranuleCorrectnessClean
 2022-09-29 16:18:55 -07:00
He Liu 63b8d775a3
Knob for RocksDb behaviors (#8360) 
* Added knob for rocksdb suggest compact.

* Added ROCKSDB_LEVEL_COMPACTION_DYNAMIC_LEVEL_BYTES knob.

Co-authored-by: He Liu <heliu@apple.com>
 2022-09-29 15:40:29 -07:00
Nim Wijetunga 232e23c87e
Snapshot Backup Encryption (#8095) 
* inital commit

* add encryption support

* refactor passing encryption keys

* disable encryption codepath

* pass enable encryption cli arg for fdbbackup

* change encryption key passing

* only enable encryption for backup if tenant mode is required

* revert server knobs

* ignore json trace files

* --amend

* integrate ekp

* add comments

* comments

* comments

* revert knobs

* fix bug

* modify encrypted backup logic

* use common prefix

* check boundaries between begin and end key

* return system encrypt domain

* modify

* add tenant entry cache to backup

* slight revert

* format

* Trigger Build

* format

* use tenant name to fetch keys

* Trigger Build

* Trigger Build

* merge

* fix getting ekp

* fix bug

* address pr comments

* Trigger Build

* Trigger Build

* remove faulty check
 2022-09-29 14:45:47 -07:00
Zhe Wu 78f29ecd88 Update default PEER_LATENCY_DEGRADATION_PERCENTILE to 0.5 2022-09-29 14:18:26 -07:00
Jon Fu e26be7a099 Merge branch 'main' of github.com:apple/foundationdb into metacluster-status 2022-09-29 11:37:50 -07:00
Jon Fu 6357ad1750 pass info through cc data to populate in status 2022-09-28 16:18:44 -07:00
A.J. Beamon c8bb15e8ee
Merge pull request #8216 from sfc-gh-ajbeamon/backup-support-for-tenants 
Include tenant and metacluster metadata in backups
 2022-09-28 12:00:24 -07:00
Trevor Clinkenbeard ef13985feb
Merge pull request #8320 from sfc-gh-tclinkenbeard/read-write-fungible 
Make read and write quotas fungible
 2022-09-28 10:59:37 -07:00
A.J. Beamon 6efe0523e4 Added some comments to new backup related methods 2022-09-28 09:33:55 -07:00
sfc-gh-tclinkenbeard ffe1529f51 Apply clang-format 2022-09-27 19:18:18 -07:00
A.J. Beamon 3353103d9d Fix filtering of potential backup mutations in commit proxy and backup worker; add code probe to ensure we are testing default backup sharing and add some tests to hit it 2022-09-27 15:25:30 -07:00
Jingyu Zhou 772a9ab9fc
Merge pull request #8185 from vishesh/issue-8127-exclude 
Make exclusion less pessimistic when warning about low space usage
 2022-09-27 14:14:25 -07:00
Trevor Clinkenbeard b7c9fb2ca9
Update fdbclient/include/fdbclient/NativeAPI.actor.h 
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2022-09-27 09:45:05 -07:00
sfc-gh-tclinkenbeard ba8fbc9573 Merge remote-tracking branch 'origin/main' into read-write-fungible 2022-09-26 23:26:14 -07:00
sfc-gh-tclinkenbeard 6b094c122f Make throttling cost functions round up to nearest page size 2022-09-26 23:24:32 -07:00
Jingyu Zhou 2711bcb5e6
Merge pull request #8326 from halfprice/zhewu/update-gray-failure-knob 
Update default PEER_DEGRADATION_CONNECTION_FAILURE_COUNT value
 2022-09-26 20:33:22 -07:00
Jon Fu 0fa462fca9 initial commit to trace metacluster metrics 2022-09-26 15:56:45 -07:00
A.J. Beamon 1b8512e949
Merge pull request #8314 from sfc-gh-ajbeamon/metacluster-management-workload-expansion 
Improve Metacluster Management Workload
 2022-09-26 15:51:28 -07:00
Zhe Wu 0188f2712b Update default PEER_DEGRADATION_CONNECTION_FAILURE_COUNT value 2022-09-26 14:45:47 -07:00
Andrew Noyes 7827535ffd
Initialize memory in new DoAppendIfFits unit test (#8324) 2022-09-26 13:46:20 -07:00
A.J. Beamon 093db25d11 Add tenant group support to the tenant management workload and test configure and rename tenant operations. Fix a few error types picked up by the test. 2022-09-26 13:14:24 -07:00
Jon Fu 2d1a6b4434 Merge branch 'main' of github.com:apple/foundationdb into metacluster-assigned-cluster 2022-09-26 09:59:20 -07:00