Commit Graph

280 Commits

Author SHA1 Message Date
Jon Fu 7a09b701cc
Merge pull request #8141 from sfc-gh-jfu/network-disable-bypass
Introduce network option for disabling mvc bypass
2022-09-21 17:33:48 -07:00
Jon Fu 937fb38dcc address review comments 2022-09-21 16:23:12 -07:00
Dennis Zhou 4ea4546cb6 blob/java: verifyBlobRange() with latestVersion 2022-09-21 14:07:16 -07:00
Jon Fu 1f778f9d76 Merge branch 'main' of github.com:apple/foundationdb into network-disable-bypass 2022-09-20 09:30:53 -07:00
Jon Fu 1abac8ea9f check on shared state ptr in native api and add to test spec in api tester 2022-09-16 15:11:33 -07:00
Jon Fu 7b0285a205 attempt to pass sim tests and api tests 2022-09-15 11:02:28 -07:00
Jon Fu a7956f42a4 add version guard and add network option to testConfig 2022-09-14 13:15:24 -07:00
A.J. Beamon 87ee0a2963
Merge pull request #8160 from sfc-gh-ajbeamon/remove-unnecessary-option
Remove unnecessary special key-space relaxed option in binding tenant management
2022-09-12 17:06:04 -07:00
A.J. Beamon 0c91336461 Remove unnecessary special key-space relaxed option in binding tenant management 2022-09-12 14:30:28 -07:00
Markus Pilman 59ce49913a
Merge pull request #8146 from sfc-gh-tclinkenbeard/improve-code-coverage
Increase the number of unit tests run in `RandomUnitTests.toml`
2022-09-12 15:10:47 -06:00
sfc-gh-tclinkenbeard 39c6989673 Remove some simulation unit tests.
These tests should not be run in simulation, because they either run too
long or break determinism.
2022-09-11 00:36:18 -07:00
Jon Fu 0f78f748f9 add assertion 2022-09-09 15:01:26 -07:00
Jon Fu f6d6aa719f initial commit to introduce network option for disabling mvc bypass 2022-09-09 14:29:38 -07:00
Dennis Zhou 802893d02b FutureBool: fix bool -> fdb_bool_t
Honestly, this might not be an issue, but it's nice to be consistent
with the conversions across c++ -> c for bool type.
2022-09-09 11:29:52 -07:00
A.J. Beamon 726d5215a0
Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (#8108)
* Remove API 720 guards for tenants (experimental feature) and the cluster ID special keys (no need to guard)

* Enable the relaxed special key access in transactions that need to use special key-space APIs introduced in 7.2
2022-09-08 17:22:36 +02:00
Dennis Zhou d9e7fc1245 ApiVersion: fix TraceFile apiversion check 2022-09-06 12:02:50 -07:00
Vaidas Gasiunas 599e57b5f2
Use distinct base trace file names for client threads (#7922)
* Save thread indexes for client instances in MVC

* An option to inlude client thread identifiers into trace file names

* Avoid using new trace file options in client versions not supporting them

* Remove redundant include generated by IDE

* Use different base trace file names for client threads by default. Provide an option to revert to using a shared base file name

* Fixing typos in the description of trace_share_among_client_threads
2022-09-06 18:46:42 +02:00
Josh Slocum 2251ae0fc2
Added summarize blob granules c api and tests (#8076)
* Added summarize blob granules c api and tests

* addressing review comments

* format
2022-09-02 17:08:51 +02:00
Dennis Zhou 80a0816157
flow: switch from hard coded to ApiVersion like ProtocolVersion (#8071)
* flow: add ApiVersion to replace hard coding api version

Instead of hard coding api value, let's rely on feature versions akin to
ProtocolVersion.

* ApiVersion: remove use of -1 for latest and use LATEST_VERSION
2022-09-02 09:28:13 +02:00
Josh Slocum 825a58880e
Reworked multi-version client readBlobGranules to not get stuck on client version changes (#8017)
* Reworked multi-version client readBlobGranules to not get stuck on version changes

* Addressing review comments
2022-08-30 09:16:09 +02:00
Dennis Zhou 3d400cff64 blob: verifyBlobRange() c api 2022-08-16 13:29:23 -07:00
Dennis Zhou 1c2109dcbd blob: add rangeLimit to getBlobGranuleRanges() 2022-08-16 13:29:23 -07:00
Dennis Zhou 96f3dd67b0 blob: add listBlobbifiedRanges() api 2022-08-16 13:29:20 -07:00
Dennis Zhou cbe9fba5e9 blob: (un)blobbifyRange() c api 2022-08-15 16:25:36 -07:00
Dennis Zhou de732c2603 bindings: add FutureBool future type 2022-08-15 16:25:36 -07:00
Vaidas Gasiunas 79571dd2b4
Testing upgrades to a future version of FDB (#7780)
* Enable configuring the next future protocol version as the current protocol version in FDB client, fdbserver, and fdbcli

* Auto format python files used in upgrade tests

* Add a test for upgrading to a future FDB version

* Emphasize that the options for using future protocol version are intended for test purposes only

* Make the global variable for current protocol version visible only locally

* Refactirng to avoid using currentProtocolVersion() in static intialization

* Update go bindings
2022-08-08 17:29:49 +02:00
A.J. Beamon c4b0f6eaae
Add an internal C API to support connection to a cluster using a connection string (#7438)
* Add an internal C API to support memory connection records

* Track shared state in the client using a unique and immutable cluster ID from the cluster

* Add missing code to store the clusterId in the database state object

* Update some arguments to pass by const&
2022-07-07 10:12:49 +02:00
A.J. Beamon 4bafe77889 Some refactoring of tenant code:
* extract tenant management into its own file and namespace
* rename the tenant management workload source file
* extract tenant special keys functions to a separate file
* extract some helper functions to GenericTransactionHelper.h
* convert StringRef -> TenantNameRef
* move some TenantMapEntry implementation into the cpp file
* add some helper functions to decode/encode a tenant mode
2022-06-27 12:32:49 -07:00
Lukas Joswiak 88557d9169 Simplify function call when transaction is null 2022-06-22 14:50:17 -07:00
Lukas Joswiak b80ed948f1 Check initialization status before accessing field 2022-06-22 14:50:17 -07:00
Josh Slocum b3597ef3a8 Added plumbing for tenant-aware purge granules 2022-06-16 13:04:34 -05:00
Hao Fu 853e6a346b
Optimization: support removing index conditionally (#7116) 2022-05-13 10:10:43 -07:00
Vaidas Gasiunas a3102c0db4 Fix updating database shared state on upgrades 2022-05-12 15:28:53 +02:00
Ray Jenkins dc9e782ccc
OpenTelemetry Tracing Perf Fixes (#6990) 2022-05-02 14:56:51 -05:00
Josh Slocum 8284ec5712 Fixing memory leak when handling FDBResult in multi version client 2022-05-02 12:56:05 -05:00
Andrew Noyes 13a388ff14
Disable leak checks for some tests that intentionally leak (#6997) 2022-04-28 12:55:34 -07:00
Ray Jenkins 1c5bf135d5
Revert "Migrate to OpenTelemetry tracing. (#6855)" (#6941)
This reverts commit 5df3bac110.
2022-04-25 09:29:56 -05:00
Vaidas Gasiunas 9897712307 Merge remote-tracking branch 'apple/main' into vgasiunas-client-tmp-dir 2022-04-22 13:00:09 +02:00
Vaidas Gasiunas 444b6c5479 Default the client temp directory to /tmp 2022-04-22 11:44:44 +02:00
Vaidas Gasiunas e48ab69c6b An option to specify a custom directory for temporary client library copies 2022-04-21 19:27:19 +02:00
Josh Slocum 40052c1394 Added Blob Granule API Tester 2022-04-20 10:15:43 -05:00
Ray Jenkins 5df3bac110
Migrate to OpenTelemetry tracing. (#6855) 2022-04-20 09:26:37 -05:00
Dan Lambright 1b3b4166c6
Merge branch 'main' into vv 2022-04-08 17:18:13 -04:00
Josh Slocum 6276cebad9
Blob integration (#6808)
* Fixing leaked stream with explicit notify failed before destructor

* better logic to prevent races in change feed fetching

* Found new race that makes assert incorrect

* handle server overloaded in initial read from fdb

* Handling more blob error types in granule retry

* Fixing rollback metadata problem, added better debugging

* Fixing version race when fetching change feed metadata

* Better racing split request handling

* fixing assert

* Handle change feed popped check in the blob worker

* fix: do not use a RYW transaction for a versionstamp because of randomize API version (#6768)

* more merge conflict issues

* Change feed destroy fixes

* Fixing change feed destroy and move race

* Check error condition in BG file req

* Using relative endpoints for blob worker interface

* Fixing bug in previous fix

* More destroy and move race fixes

* Don't update empty version on destroy in case it gets rolled back. moved() and removing will take care of ensuring it is not read

* Bug fix (#6796)

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* Fixing durability issue with moving and destroying change feeds

* Adding fix for not fully deleting files for a granule that child granules need to re-snapshot

* More destroy and move races

* Fixing change feed destroy and pop races

* Renaming bg prune to purge, and adding a C api and unit test for it

* more cleanup

* review comments

* Observability for granule purging

* better handling for change feed not registered

* Fixed purging bugs (#6815)

* fix: do not use a RYW transaction for a versionstamp because of randomize API version

* fix: if the initialSnapshotVersion was pruned, granule history was incorrect

* added a way to compress null bytes in printable()

* fixed a few purging bugs

Co-authored-by: Evan Tschannen <evan.tschannen@snowflake.com>
2022-04-08 14:15:25 -07:00
Dan Lambright c106847e3e
Merge branch 'main' into vv 2022-04-08 15:05:51 -04:00
Dan Lambright 5bdc525353
Merge branch 'main' into vv 2022-04-08 13:16:04 -04:00
Tao Lin 73ed34d2fb Fixes #6793: Upgrade from 7.0.0 to 7.1.0 fails because of missing function fdb_transaction_get_mapped_range 2022-04-08 09:32:18 -07:00
Jon Fu 1a3b5f78c4 trace sevError if no map entry is found 2022-04-08 10:32:24 -04:00
Jon Fu 68a75dd9cc additional safety checks in the upgrade scenario for GRV cache 2022-04-08 10:17:18 -04:00
Jon Fu b0ae22f16d Merge branch 'main' of github.com:apple/foundationdb into jfu-grv-cache-multi-threaded 2022-04-06 12:28:28 -04:00