Commit Graph

662 Commits

Author SHA1 Message Date
Boris Korzun f7a443aea4 Add libfmt 10+ support (#11140) 2024-02-13 23:09:58 +03:00
Dimitris Apostolou a88114c222
Fix typos 2024-02-07 01:16:00 +02:00
Boris Korzun ac30cb0783 Add preinstalled doctest support 2023-10-06 13:33:30 +03:00
Zhe Wu 5c8a163c72
Update main branch to 7.4 (#10459)
* Update main branch to 7.4

* Update API version to 740

* Makes fdb_c_client_config_tests.py passing after API version update

* Remove from_7.3.0_until_7.4.0 and add from_7.3.0

* Update tests in fdb_c_client_config_tests.py
2023-06-15 10:19:39 +02:00
Sam Gwydir 6c16875c34
Add networkoption to disable non-TLS connections (#9984)
* Add networkoption to disable non-TLS connections

* add disable plaintext connection to fdbserver

* python doc

* Formatting

* Add tls disable plaintext connection to client api test

* review

* fix negative test

* formatting

* add TLS support to c client config tests

Adds support for TLS in the client and server separately

* add tests for disable_plaintext_connections

Test TLS and Plaintext Clusters and Clients

* Fix documentation

* Rename option to indicate it is client-only

* clearer formatting

* default to allowing plaintext connections

* add SetTLSDisablePlaintextConnection to go bindings
2023-05-13 00:14:11 +02:00
Zhe Wu e3490a1af8 Update API version to 730 2023-05-10 11:26:46 -07:00
Zhe Wu a6d6c70aad
Merge pull request #10103 from halfprice/zhewu/update-main-to-7.4
Bring main branch to 7.3
2023-05-10 09:53:27 -07:00
Zhe Wu 761cdbc019 Bring main to 7.3 2023-05-09 21:14:16 -07:00
A.J. Beamon 686c391281
Merge pull request #10093 from sfc-gh-ajbeamon/apply-flake8
Fix several issues found by flake8
2023-05-09 09:05:14 -07:00
Josh Slocum b5a9997217
adding DBPerTXN blob granule tests (#10095)
* adding DBPerTXN blob granule tests

* fixing comment
2023-05-08 15:43:29 +02:00
A.J. Beamon fef2967d62 Fix several issues found by flake8. Ignore E402, which detects when we have import statements not at the top of the file, since we have many files that use this pattern and are not easy to fix. 2023-05-02 10:06:40 -07:00
A.J. Beamon 182dc93ebd Apply black format to most Python files, excluding a few cases where we have Python 2 files and a few files written externally. Add external files as exclusions to the precommit checks. 2023-04-28 11:46:41 -07:00
hao fu 29161b2fda Revert matchIndex feature
It is not protocol compatible, revert it to avoid deployment issue.
Will have a new PR to have the feature if moving forward.
2023-04-17 09:39:45 -07:00
Vaidas Gasiunas 894f555e95
Test loop profiler using API Tester (#7174)
* Api Tester: Specify knobs in the toml file; Test loop profiler

* Gracefully stop the loop profiler thread

* Protect loop profiler thread by mutex

* Create loop  profiler thread only if is not stopped
2023-03-30 23:00:23 +02:00
Vaidas Gasiunas 7870479b66 Improve stability of test_external_client_not_matching_cluster_version_ignore 2023-03-20 17:30:35 -07:00
A.J. Beamon 33d501d36a Avoid potential access of empty optionals when printing errors in the api tester 2023-03-14 10:41:43 -07:00
Josh Slocum e1b620135b Merge branch 'main' into bg_latency_fixes 2023-03-06 09:23:11 -06:00
Junhyun Shim a2a29af56e
Temporarily disable client trace check for older version (#9578)
* Disable client trace check test on older version

Older version doesn't guarantee trace flush upon network::stop()
Comment it out for the time being

* Black-reformat authz and client config tester scripts
2023-03-06 15:36:28 +01:00
Josh Slocum 526134f598 review comments 2023-03-03 13:52:59 -06:00
Josh Slocum fdcb6aed5f add timestamp to bg verbose tester logs 2023-03-02 09:34:40 -06:00
Markus Pilman 5bebb5b4aa
Merge pull request #9492 from sfc-gh-vgasiunas/vgasiunas-api-version-defs
Centralize definition of API Version for Java, Python and C API
2023-02-28 12:04:02 -07:00
Russell Sears d2af157189 Add support for using pre-packaged jemalloc, doctest_proj, and fix python support for old fdbserver binaries 2023-02-27 15:38:58 -06:00
Vaidas Gasiunas ba726fac87 Replace hardcoded API version checks for 720 and 730 2023-02-27 16:18:01 +01:00
Vaidas Gasiunas 1b6ced1a09 Define latest C API version in one place 2023-02-27 15:13:56 +01:00
Josh Slocum 958f3b531b
Plumbing blob worker mapping through commit proxy like storage server (#9401)
* Plumbing blob worker mapping through commit proxy like storage server mapping

* review comments

* formatting
2023-02-21 13:21:44 -06:00
sfc-gh-tclinkenbeard 398079db3a Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration 2023-02-20 17:54:06 -08:00
Josh Slocum bfb3ffc509
added c and java apis for granule flush (#9412) 2023-02-20 10:28:11 -06:00
sfc-gh-tclinkenbeard 1aef6cb5f7 Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration 2023-02-17 20:41:59 -08:00
Josh Slocum 6c2fb13173
adding wait parameter to blobbify api (#9360)
* adding wait parameter to blobbify api

* formatting

* fixing comment style

* fixing bug and adding debugging

* adding blob ranges unit test

* testing both blobbify cases in cancel

* formatting

* switch to explicit blocking api instead of boolean flag

* remove comments

* format
2023-02-17 12:20:53 -06:00
Josh Slocum c26831ec04
adding version metadata to blob granule file pointers (#9392) 2023-02-16 17:11:11 -06:00
Josh Slocum eefc889389
Add tenant and encryption support to new bg file apis (#9315)
* Add tenant and encryption support to new bg file apis

* formatting

* fixing comment style for linter
2023-02-15 11:48:40 -06:00
Junhyun Shim 380a5aa8e8
Merge pull request #9335 from sfc-gh-jshim/fix-mako-token-reassign-after-reset
Mako: fix token not being reassigned after every reset
2023-02-09 22:14:39 +01:00
Josh Slocum 81c984e48a
adding encryption at rest support to local cluster and api tester (#9325)
* adding encryption at rest support to local cluster and api tester

* adding encrypted variant of bg tenant test and renaming ear
2023-02-09 10:23:17 -06:00
Junhyun Shim 357cec17d8 Mako: fix createNewTransation return value
Should return token instead of tenant name
2023-02-09 16:00:48 +01:00
Junhyun Shim 499a7de718 Mako: add missing token reassignment after transaction reset
Also remove redundant calls to tx.reset() in step functions after OP_COMMIT
(The step function caller is already resetting the object after OP_COMMIT is run successfully)
2023-02-09 13:35:21 +01:00
sfc-gh-tclinkenbeard 09ad864eb5 Merge remote-tracking branch 'origin/main' into expose-tag-throttled-duration 2023-02-08 11:25:14 -08:00
Junhyun Shim d7d0b658a3
Merge pull request #9314 from sfc-gh-jshim/authz-tenant-name-to-tenant-id
Make Authz use tenant ids instead of tenant names
2023-02-08 20:13:29 +01:00
Josh Slocum ebbd221fe7
formatting files (#9322)
* formatting files

* another file
2023-02-08 13:01:24 -06:00
Junhyun Shim d32d3dd085 Make token option NOT survive transaction hard resets 2023-02-08 18:12:22 +01:00
Junhyun Shim c4703b5940 Mako: lower tenant id fetch error to debug
With MVC mode, it always fails at least once for each tenant and gets spammy
2023-02-07 18:44:43 +01:00
Junhyun Shim ff61e2990b Fix clang build error and relax authz test tx timeout 2023-02-07 09:59:39 +01:00
Junhyun Shim 180ddba4ec Fix clang format 2023-02-06 23:14:39 +01:00
Junhyun Shim be225acd2a Merge remote-tracking branch 'origin/main' into authz-tenant-name-to-tenant-id 2023-02-06 23:13:43 +01:00
Josh Slocum 73c9735c21
Bg api tenant tests (#9301)
* fixing tenant creation + setup order

* removing seenReadSuccess since verifyBlobRange is called before test start

* fixing bg_desc skip tenant check
2023-02-06 13:18:27 -06:00
Junhyun Shim 56d3438ea4 Add tenant ID fetch to mako (for token signing) 2023-02-03 23:55:55 +01:00
Junhyun Shim ce652fa284 Replace AuthZ's use of tenant names in token with tenant ID
Also, to minimize audit log loss, handle token usage audit logging at each usage.
This has a side-effect of making the token use log less bursty.
This also subtly changes the dedup cache policy.
Dedup time window used to be 5 seconds (default) since the start of batch-logging.
Now it's 5 seconds from the first usage since the closing of the previous dedup window
2023-02-03 21:46:31 +01:00
Vaidas Gasiunas 6aba6ceb8a
libfdb_c_shim: Load all symbols at once in a thread-safe way; remove unused options of implib.so (#9293) 2023-02-02 19:37:24 +01:00
Vaidas Gasiunas f8b1da8bc6
An option to initialize client tracing in setupNetwork (#9209)
* client_config_tester: use a generic mechanism to set specific network options

* trace_initialize_on_setup option to initialize client traces on network setup without local IP address

* trace_initialize_on_setup: Addressing review comments

* Restore correct formatting

* trace_initialize_on_setup: Update go bindings

* Include PID for identification into trace file names by default

* Use the same naming pattern for trace files in all configurations

* Empty commit
2023-02-02 10:00:51 +01:00
Marian Dvorsky dfce17f65f
Add new Blob Granule (BG) APIs for raw blob data access (#9245)
Adds a prototype of new APIs to access the Blob Granule data:

fdb_future_readbg_get_descriptions
fdb_readbg_parse_snapshot_file
fdb_readbg_parse_delta_file
fdb_result_get_bg_mutations_array

The client first calls fdb_future_readbg_get_descriptions to get a high level description of granules, reads the files and then calls fdb_readbg_parse_snapshot_file and fdb_readbg_parse_delta_file to parse the snapshot and delta files, respectively. This allows the client to get a raw access to the blob granule data, and allows the client to process them in a way they wouldn't be efficiently able to when using previous blob granule read APIs.

We expect these APIs to evolve before a release.

---------

Co-authored-by: Josh Slocum <josh.slocum@snowflake.com>
2023-01-31 15:22:04 +01:00
Josh Slocum 30ef4f5d27 changing randomKeyRange to randomNonEmptyKeyRange 2023-01-27 10:58:39 -06:00