Commit Graph

780 Commits

Author SHA1 Message Date
A.J. Beamon cfb36bd9c3 Support versionstamps in key backed object properties 2022-10-05 12:48:40 -07:00
Markus Pilman 550488b020 Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide
# Conflicts:
#	bindings/c/CMakeLists.txt
#	fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
#	fdbserver/BackupWorker.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/CommitProxyServer.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/PhysicalShardMove.actor.cpp
#	flow/CMakeLists.txt
2022-10-04 18:27:48 -06:00
Markus Pilman 97dfc6823f fixed build with OPEN_FOR_IDE 2022-10-04 17:01:02 -06:00
Vaidas Gasiunas 415e1aeb1f
Fix for a memory leak in database operations called after create database errors (#8334)
* ApiTester: cancel pending futures for non-transactional operations

* Clear future callback in case of abort signal to avoid memory leaks

* Abortable future: Make sure all callbacks are cleared in case of premature signal; addressing comments

* Abortable future: remove debugging asserts

* Abortable future: restore the previous method for cancelling wrapped future

* Removing debugging code
2022-10-01 11:30:52 +02:00
Josh Slocum f95c22c6a2
Blob Granule Database API tests (#8276)
* adding verify

* Adding error test cases

* more fixes to bg api tests
2022-09-26 11:15:10 +02:00
Jon Fu 7a09b701cc
Merge pull request #8141 from sfc-gh-jfu/network-disable-bypass
Introduce network option for disabling mvc bypass
2022-09-21 17:33:48 -07:00
Dennis Zhou 4ea4546cb6 blob/java: verifyBlobRange() with latestVersion 2022-09-21 14:07:16 -07:00
Jon Fu 1f778f9d76 Merge branch 'main' of github.com:apple/foundationdb into network-disable-bypass 2022-09-20 09:30:53 -07:00
Vaidas Gasiunas 97eddbc06a
Extend ApiTester with support for testing database operations (#8221)
* ApiTester: Refactoring removing unnecessary transaction actor classes

* ApiTester: enable executing non-transactional database operations

* ApiTester: a test for fdb_database_list_blobbified_ranges

* ApiTester: Fix memory ownership of the end key in randomGetRangeOp

* ApiTester: reuse get range result validation in blob granule tests
2022-09-20 16:21:38 +02:00
1inker db1f858c64
ANSI C syntax mandates declare functions without parameters with void in parameters list. K&R syntax is obsolete. (#8222) 2022-09-19 12:09:05 -07:00
Jon Fu 1abac8ea9f check on shared state ptr in native api and add to test spec in api tester 2022-09-16 15:11:33 -07:00
Jon Fu 7b0285a205 attempt to pass sim tests and api tests 2022-09-15 11:02:28 -07:00
Jon Fu a7956f42a4 add version guard and add network option to testConfig 2022-09-14 13:15:24 -07:00
Dennis Zhou c65ecb08d1 bindings/c: fix std++fs dependency
Beginning with clang 9, <filesystem> support is included without needing
to link anything else.

https://releases.llvm.org/9.0.0/projects/libcxx/docs/UsingLibcxx.html
2022-09-13 15:01:27 -07:00
Dennis Zhou 540b814446
Merge pull request #8158 from sfc-gh-dzhou/api-version
ApiVersion cleanups
2022-09-13 14:00:22 -07:00
A.J. Beamon 2d68e246fb Fix a couple other merge issues 2022-09-13 08:23:26 -07:00
Dennis Zhou 88d8242bb6 bindings: clean up ApiVersion settings
Currently ApiVersion is scattered and hardcoded across the bindings.
Let's at least clean it up so it's once per file or less and use some
redirection against FDB_API_VERSION or API_VERSION.
2022-09-12 17:08:57 -07:00
Dennis Zhou fed00e0f0c ApiTester: TMP_DIR ApiVersion gate 2022-09-12 17:08:57 -07:00
Dennis Zhou d6d69379a8 ApiVersion: tenant specific guards 2022-09-12 17:08:56 -07:00
A.J. Beamon b0f4e0b997 Fix some merge issues 2022-09-12 15:42:31 -07:00
A.J. Beamon b9c8bdeec6 Extract randomTenant() function; add back accidentally deleted line; move the new multi tenant test and fix a name error in its spec 2022-09-12 15:03:30 -07:00
A.J. Beamon 8086fc819e Some refactoring of tenants in the API tester. This mainly changes it so that tenants are stored on the ApiWorkload and any subclasses can use them. Update the setup phase to work across tenants. 2022-09-12 15:03:30 -07:00
Sagar Vemuri 00276d5700 Move the test to be multithreaded 2022-09-12 15:02:44 -07:00
Sagar Vemuri ccf405c17a Add more workload operations support 2022-09-12 15:02:44 -07:00
Sagar Vemuri 52150a5cf7 Add a new multi-tenant specific Transaction Executor 2022-09-12 15:01:39 -07:00
Sagar Vemuri ba314fbc12 Plumb tenant id from individual workload operation to transaction executor 2022-09-12 14:57:29 -07:00
Sagar Vemuri 6b17326135 Add tenant support 2022-09-12 14:40:47 -07:00
Jingyu Zhou 48a5fb9e4b
Merge pull request #8094 from sfc-gh-dadkins/sfc-gh-dadkins/fix-c-bindings
Add missing fdb_cpp ->fdb_c dependency.
2022-09-12 09:42:40 -07:00
Vaidas Gasiunas 81fff640bd
Testing with invalid cluster files, fixing update from changed cluster file (#8126)
* ApiTester: test with invalid cluster files

* More asserts in monitorProxies

* ApiTester: Test tampering the cluster file

* Fix update of connection string from the cluster file to use the new connection string only if it valid

* ApiTester: add linker dependency on std++fs

* upgrade_test: no-cleanup-on-error option

* ApiTester: use atomic operations to change and access the transaction handle
2022-09-10 09:23:00 +02:00
Vaidas Gasiunas b6fb1034a9
Fix crashes on database create errors (#8115)
* ApiTester: enable access to database instances in workloads

* ApiTester: Inject database create errors

* Fix getClusterProtocol and database shared object initialization in case of database create errors

* Check deferred error in all Thread-Safe API calls returning a future

* ApiTester: disable injection of database create errors for tests with old versions
2022-09-08 09:21:51 +02:00
Jon Fu 1c2afeff8d add raw access to create tenant api 2022-09-07 09:10:34 -07:00
Vaidas Gasiunas f60a1e488f
Fixing TSAN error in API Tests: future callback trying to accessed destroyed scheduler (#8016)
* ApiTester: More asserts on workload status; Fixing data race in the cleanup phase

* ApiTester: comments explaining assumptions behind synchronization in TransactionContext; declaring immutable fields as const

* Add build dependency between fdb_c_shim and fdb_c
2022-09-05 13:39:10 +02:00
Dan Adkins 457c5b336a Add missing fdb_cpp ->fdb_c dependency. 2022-09-02 17:41:24 -07:00
Josh Slocum 2251ae0fc2
Added summarize blob granules c api and tests (#8076)
* Added summarize blob granules c api and tests

* addressing review comments

* format
2022-09-02 17:08:51 +02:00
Vaidas Gasiunas 9fb45f349f
Make FDB C shim library static (#8040) 2022-08-31 12:31:07 +02:00
Josh Slocum 825a58880e
Reworked multi-version client readBlobGranules to not get stuck on client version changes (#8017)
* Reworked multi-version client readBlobGranules to not get stuck on version changes

* Addressing review comments
2022-08-30 09:16:09 +02:00
Josh Slocum cfc5ab550e
Update BlobGranules API tests (#7947)
* blob granule fixes from api tests

* Adding BlobGranuleErrors workload

* bug fixes and debugging improvements for blob granule api tests

* cleanup and refactoring of bg api tests

* better memory management for loadAndMaterialize
2022-08-23 17:47:02 -05:00
Dennis Zhou 3d400cff64 blob: verifyBlobRange() c api 2022-08-16 13:29:23 -07:00
Dennis Zhou 1c2109dcbd blob: add rangeLimit to getBlobGranuleRanges() 2022-08-16 13:29:23 -07:00
Dennis Zhou 96f3dd67b0 blob: add listBlobbifiedRanges() api 2022-08-16 13:29:20 -07:00
Dennis Zhou cbe9fba5e9 blob: (un)blobbifyRange() c api 2022-08-15 16:25:36 -07:00
Dennis Zhou de732c2603 bindings: add FutureBool future type 2022-08-15 16:25:36 -07:00
Vaidas Gasiunas f04ab582dd
Add documentation for API Tester (#7869)
* Add documentation for API Tester

* API Tester documentation: addressing review comments
2022-08-15 15:10:15 +02:00
Vaidas Gasiunas c387e140cd
Test more C API in upgrade tests (#7853)
* Test more C API in upgrade tests

* Fix saving results in blob granule API correctness test

* Enable multitenancy in upgrade test with API version >=720

* Disable the upgrade test with blob granule workload
2022-08-15 11:08:53 +02:00
Marian Dvorsky 35a3a33d1c
Resolve TSAN-related issues in ctests (#7864) 2022-08-12 19:23:32 +02:00
Xiaoge Su 0326d53965 Split proxy_memory_limit_exceeded to commit/grv specific exceptions
Currently GRV is reporting proxy_memory_limit_exceeded error which has
error message claiming Commit proxy failing. This split should remove
such confusion.
2022-08-12 00:45:57 -07:00
Vaidas Gasiunas d0905704c9
Disable shim library tests in sanitizer builds (#7821) 2022-08-09 11:27:52 +02:00
Vaidas Gasiunas 79571dd2b4
Testing upgrades to a future version of FDB (#7780)
* Enable configuring the next future protocol version as the current protocol version in FDB client, fdbserver, and fdbcli

* Auto format python files used in upgrade tests

* Add a test for upgrading to a future FDB version

* Emphasize that the options for using future protocol version are intended for test purposes only

* Make the global variable for current protocol version visible only locally

* Refactirng to avoid using currentProtocolVersion() in static intialization

* Update go bindings
2022-08-08 17:29:49 +02:00
Andrew Noyes db348598a1 Update Python target name to Python3 2022-07-29 14:57:29 -07:00
Vaidas Gasiunas d99d0370b1
C shim library: API for setting client library path; additional tests (#7702)
* Adding sources of the Implib.so project

* Run C unit tests and API tests with the shim library

* Reuse compilation of C test binaries with and without shim library

* Resolve client library path from an environment variable

* Refactoring: Reusable module for downloading FDB binaries

* Testing client shim library with current version and last release version

* Tests for specifying client library over an environment variable

* Enable C shim library tests on ARM

* Restore the original path for including fdb_api.hpp

* Improve cmake dependencies on doctest

* Remove unnecessary Implib.so files

* Shim library: API to specify path to the library, additional tests

* Adding shim library to Linux distribution packages

* Update link dependencies of fdb_c_shim_lib_tester

* Add dependencies on Implib.so sources

* Abort on fatal errors of shim library

* fix typo
2022-07-29 11:45:45 +02:00