wbtiger
/
foundationdb
mirror of https://github.com/apple/foundationdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
24,719 Commits 34 Branches 304 Tags 265 MiB
Commit Graph

24719 Commits

Author SHA1 Message Date
Nim Wijetunga 86f3665514
Handle EKP Tenant Not Found Errors (#9261) 
handle EKP tenant not found errors
 2023-02-01 19:15:38 -08:00
Xiaoxi Wang de670b7129
Merge pull request #9285 from sfc-gh-xwang/fix/main/testTimeout 
Remove test timeout
 2023-02-01 12:00:23 -08:00
Josh Slocum 6dbb96312b
move feed cleanup check to after data is guaranteed to be available for granule (#9283) 2023-02-01 13:19:08 -06:00
Jingyu Zhou 5122cb3499
Merge pull request #9282 from jzhou77/fix 
Reduce logging level for verbose events
 2023-02-01 11:16:41 -08:00
Xiaoxi Wang 8f7a98fb9d remove test timeout 2023-02-01 11:04:39 -08:00
Jingyu Zhou dbea7877cf
Merge pull request #9278 from sfc-gh-satherton/consistencyscan-updates 
Documentation for consistencyscan CLI command.
 2023-02-01 10:46:54 -08:00
Jingyu Zhou 982ce105e5 Reduce logging level for verbose events 
From one of nightly failure due to too many log lines, these are top 3:

  60100 FastRestoreLoaderDispatchRequests
  79655 FastRestoreGetVersionSize
  93888 FastRestoreSplitMutation
 2023-02-01 09:51:16 -08:00
Steve Atherton b1ed45688e Added documentation for consistencyscan CLI command. 2023-01-31 23:09:28 -08:00
Jingyu Zhou b911bcfce2
Merge pull request #9240 from jzhou77/fix 
Fix UpgradeAndBackupRestore test failures
 2023-01-31 19:15:58 -08:00
Jingyu Zhou 9c5ce68d98 Merge branch 'main' of https://github.com/apple/foundationdb into fix 2023-01-31 17:40:13 -08:00
A.J. Beamon b8592c4e09
Merge pull request #9219 from sfc-gh-jfu/tenant-object-binding-tests 
Add tenant getId to java/python binding tester
 2023-01-31 16:22:38 -08:00
He Liu eb4839d046
Fix audit_storage issues (#9265) 
* Erase audit from DD after completion.

* Use CLIENT_KNOBS->TOO_MANY as the default number of rows for
get_audit_status.
 2023-01-31 15:39:01 -08:00
Jon Fu 56bef16a87
Update bindings/bindingtester/spec/tenantTester.md 
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2023-01-31 18:01:06 -05:00
Jon Fu 2ade905a60
Update bindings/bindingtester/spec/tenantTester.md 
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2023-01-31 18:01:00 -05:00
Jon Fu e761c1c169 update bindingtester spec 2023-01-31 13:41:39 -08:00
Josh Slocum 77875d7fee
Fixing SkewedReadWrite to load its metadata in a transactionally consistent way (#9274) 2023-01-31 14:52:36 -06:00
Jingyu Zhou fcab17cfab Merge remote-tracking branch 'apple/main' into fix 2023-01-31 12:47:28 -08:00
Jon Fu c07657c592 push string onto stack when active tenant is set 2023-01-31 12:34:37 -08:00
Xiaoxi Wang 211a363131
Merge pull request #9174 from sfc-gh-xwang/feature/main/watchTenant 
Validate tenant access in applyMetadataEffect and corresponding test workload
 2023-01-31 10:32:36 -08:00
Jingyu Zhou 58fcc90783 Add comments on why custom encoding is needed 2023-01-31 10:04:27 -08:00
Xiaoxi Wang 911f849312
Apply suggestions from code review 
Co-authored-by: A.J. Beamon <aj.beamon@snowflake.com>
 2023-01-31 09:13:49 -08:00
Jingyu Zhou 0b7650e973
Merge pull request #9273 from sfc-gh-vgasiunas/vgasiunas-fix-windows-build 
Fix compilation errors in Windows
 2023-01-31 09:04:21 -08:00
Vaidas Gasiunas b7a5f6af1c Fix compilation errors in Windows 2023-01-31 16:20:31 +01:00
Marian Dvorsky dfce17f65f
Add new Blob Granule (BG) APIs for raw blob data access (#9245) 
Adds a prototype of new APIs to access the Blob Granule data:

fdb_future_readbg_get_descriptions
fdb_readbg_parse_snapshot_file
fdb_readbg_parse_delta_file
fdb_result_get_bg_mutations_array

The client first calls fdb_future_readbg_get_descriptions to get a high level description of granules, reads the files and then calls fdb_readbg_parse_snapshot_file and fdb_readbg_parse_delta_file to parse the snapshot and delta files, respectively. This allows the client to get a raw access to the blob granule data, and allows the client to process them in a way they wouldn't be efficiently able to when using previous blob granule read APIs.

We expect these APIs to evolve before a release.

---------

Co-authored-by: Josh Slocum <josh.slocum@snowflake.com>
 2023-01-31 15:22:04 +01:00
Jingyu Zhou ceeb988b29
Merge pull request #9266 from saintstack/audit_storage_help 
Edit of audit_storage usage and help message.
 2023-01-30 20:27:26 -08:00
Jingyu Zhou b4c16e9b97
Merge pull request #9217 from xis19:binding_test 
Generate bindingtester2 tarball for test
 2023-01-30 20:06:16 -08:00
Xiaoge Su 2f581fc290 fixup! Reformat source 2023-01-30 17:07:57 -08:00
Xiaoge Su 84343b3b97 fixup! fix the compile error 2023-01-30 17:07:57 -08:00
Xiaoge Su 896a9decd5 Let materializeBlobGranule accept a std::vector<StringRef> instead of StringRef[] 
The deltaFileData parameter was type StringRef[], this indirectly causes
issue #9206. By setting the incoming parameter type to be
std::vector<StringRef>, the issue might be fixed plus adding extra
memory safety.
 2023-01-30 17:07:57 -08:00
Xiaoxi Wang 3b385bbfe4 Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/watchTenant 2023-01-30 16:59:05 -08:00
Xiaoxi Wang f1c6d0ee2a fix tenantIds set condition when tenant set metadata version key 2023-01-30 16:58:57 -08:00
Hui Liu 1021878764
Merge pull request #9250 from sfc-gh-huliu/restoreurl 
Define knob url for blob manifest files
 2023-01-30 16:30:02 -08:00
Hui Liu b475121840
Merge pull request #9267 from sfc-gh-huliu/fixerror 
Fix misc blob restore error messages
 2023-01-30 15:29:42 -08:00
Xiaoxi Wang 971920e417 add tenant check trace; delete isSystemKey check; fix clear range condition bug 2023-01-30 14:21:17 -08:00
Lukas Joswiak 2b5c0ebe7b Add version epoch to status json 
Adds a new `version_epoch` object to `status json`, which includes the
status of the feature, and the current epoch if it is enabled. If the
version epoch is disabled, the `epoch` field will not be present.

```
{
    "client" : {
        ...
    },
    "cluster" : {
        ...
        "version_epoch" : {
            "enabled" : "true",
            "epoch" : "100000"
        },
        ...
    }
}
```
 2023-01-30 13:21:19 -08:00
Hui Liu 6fbabab6aa Fix misc blob restore error messages 2023-01-30 13:01:27 -08:00
Josh Slocum d7ebaad6ab
Adding fdbcli and native api force-flush and force-recompact for blob granules (#9243) 
* Adding fdbcli and native api force-flush and force-recompact for blob granule ranges

* throwing execption for faster retry instead of breaking loop

* fixing force flush at a non-committed, rolled back version

* fixing ide build
 2023-01-30 15:00:57 -06:00
Josh Slocum cd8e310464
increasing timeout since simulated recovery can take >30s, and adding comment (#9269) 2023-01-30 14:35:00 -06:00
neethuhaneesha ab42fdfec9
Removing extra logging rocksdb call, if logs are muted. (#9256) 2023-01-30 11:44:06 -08:00
Josh Slocum 1b4753a4d4
Fix chunked reads (#9246) 
* removing chunked read loop

* reducing memory overhead of async file block cache by freeing some blocks during read if no longer needed
 2023-01-30 13:43:24 -06:00
Josh Slocum ec6ae2c8c9
skipping ss validation while shutting down because of races (#9268) 2023-01-30 13:35:22 -06:00
Jingyu Zhou cd18b6ec3d
Merge pull request #9255 from sfc-gh-jslocum/bg_api_test_empty_range_fix 
add randomKeyRange to api tester with bug fix, and refactor TesterBlo…
 2023-01-30 10:09:09 -08:00
Josh Slocum 8165ec8455 Merge branch 'main' into bg_api_test_empty_range_fix 2023-01-30 11:24:41 -06:00
Jingyu Zhou 4cf20c9f7f
Merge pull request #9260 from sfc-gh-chxu/fix-grvproxy-broken-promise 
GRV Proxy converts broken_promise errors to master_failed or tlog_failed
 2023-01-30 09:14:09 -08:00
Xiaoxi Wang 04ceb2c0e9 change tenantIds initialization; format code; 2023-01-29 22:21:27 -08:00
Michael Stack f5ede397d3 Include clang-format change 2023-01-28 00:41:55 +00:00
stack 2770767255 Edit of audit_storage usage and help message. 
Add explanation of command return. Amend return format (slightly).
 2023-01-27 15:56:28 -08:00
Chunhao Xu b9e00d03df fix on GRV Proxy converts broken_promise errors to master_failed or tlog_failed 2023-01-27 15:19:44 -08:00
Josh Slocum f41b61aacf
Blobstore static connection pool, and observability improvements (#9234) 
* Adding global connection pool for multiple blobstore instances

* adding knob to enable/disable blobstore global connection pool

* Adding BlobStoreMetrics and BlobStoreRequestLatency logging for better blobstore observability
 2023-01-27 16:46:26 -06:00
Xiaoxi Wang d1d860a342 merge upstream/main 2023-01-27 12:15:04 -08:00