Commit Graph

1720 Commits

Author SHA1 Message Date
Andrew Noyes 45c8a4e09d
Build fixes/improvements for sanitizers (#7657)
* Don't build fdb c shim with ubsan

This avoids duplicate symbols when linking. It doesn't really make sense
to assemble files with -fsanitize=undefined anyway, since it won't
insert instrumentation.

* Consolidate boost_asan with boost_target
2022-07-21 12:43:38 -07:00
A.J. Beamon 190ad8c7e9 Convert existing tuple usages to use Tuple::makeTuple() 2022-07-19 13:45:59 -07:00
A.J. Beamon 1b81e72604 Add a Tuple::makeTuple function to easily construct a tuple. Update Tuple to allow all types to be passed via .append() so they can be used with makeTuple. 2022-07-19 11:50:58 -07:00
A.J. Beamon 02ab3b05ab
Merge pull request #7590 from sfc-gh-ajbeamon/update-prefix-encoding-in-tenant-metadata-json
Use a more standard encoding in the JSON metadata reported for tenants
2022-07-15 14:32:53 -07:00
Marian Dvorsky ce4dc0fb0f
Update ctest_to_joshua to work with new ctests (#7584)
* Package inferred directories to the resulting tarball in ctest_to_joshua (except for the root source and build directories). This allows the API tests (which specify the workloads as a directory containing the workload specification files) to now work.
* Add bin/mkcert which is used by some tests.
* Rewrite fdb_c_shim_tests.py to specify its dependencies on command line.
2022-07-14 18:34:48 +02:00
A.J. Beamon 91ccdbcb03 Use a more standard encoding in the JSON metadata reported for tenants. 2022-07-13 15:22:19 -07:00
A.J. Beamon a3f970cf3a Parse integer options in fdbcli rather than passing the bytes unparsed. This allows specifing the string "1" instead of "\x01\x00\x00\x00\x00\x00\x00\x00". 2022-07-11 16:39:59 -07:00
Markus Pilman 4ece33a0a8
Merge pull request #7445 from sfc-gh-anoyes/anoyes/fix-ubsan
Fix UBSAN build when statically linking libcxx
2022-07-11 17:27:37 -06:00
Vaidas Gasiunas 0a2e5c830c
C shim library build clean up (#7560)
* Remove unnecessary Implib.so files

* Improve cmake dependencies on doctest
2022-07-11 17:57:28 +02:00
john_leach afa1120874 Changing hasIncompleteVersionstamp to use iteration rather than stream processing #7543 2022-07-08 13:27:40 -07:00
Vaidas Gasiunas 1e8feb9cb8
Generate a shim library for the FDB C API (#7506)
* Adding sources of the Implib.so project

* Run C unit tests and API tests with the shim library

* Reuse compilation of C test binaries with and without shim library

* Resolve client library path from an environment variable

* Refactoring: Reusable module for downloading FDB binaries

* Testing client shim library with current version and last release version

* Tests for specifying client library over an environment variable

* Enable C shim library tests on ARM

* Restore the original path for including fdb_api.hpp
2022-07-08 16:28:35 +02:00
Marian Dvorsky 24d7e0d28b
Add WatchAndWait API client test workload (#7524)
* Add WatchAndWait workload

* Reimplement without Notification

* Handle errors

* Finish comment

* Simplify the test by introducing commit(/*complete=*/false)

* Fix a race condition

* Refactor

* Remove unused capture

* Add assert for has_value

* Automatically adjust numClientThreads
2022-07-08 11:57:38 +02:00
Andrew Noyes 00743e4b46 Don't set linker script for ubsan 2022-07-07 10:19:20 -07:00
Andrew Noyes c3abfdc20d Use c++ as linker language for c targets with UBSAN 2022-07-07 10:18:30 -07:00
A.J. Beamon c4b0f6eaae
Add an internal C API to support connection to a cluster using a connection string (#7438)
* Add an internal C API to support memory connection records

* Track shared state in the client using a unique and immutable cluster ID from the cluster

* Add missing code to store the clusterId in the database state object

* Update some arguments to pass by const&
2022-07-07 10:12:49 +02:00
A.J. Beamon ae6d6e0e72
Merge pull request #7402 from sfc-gh-jfu/jfu-mako-active-tenants
Introduce concept of "active" versus "total" tenants in mako
2022-07-06 14:23:11 -07:00
Jon Fu 61b3e1f51e add comment about our tenant encoding 2022-07-06 13:16:24 -07:00
Jon Fu af78a2ef5f move tr options to outside loop 2022-07-06 13:07:47 -07:00
A.J. Beamon 1e8225a19e
Merge pull request #7511 from sfc-gh-ajbeamon/add-subrange-support-to-tenant-special-keys
Update the tenant special keys submodule to support multiple sub-ranges
2022-07-06 10:33:14 -07:00
Marian Dvorsky 975978d938
Add getKey support to ApiCorrectnessWorkload (#7517)
* Add getKey support to ApiCorrectnessWorkload

* Address review comments
2022-07-05 10:16:29 +02:00
A.J. Beamon aff21f062f Fix indentation and refactor prefix construction to be in forward order 2022-07-01 09:50:05 -07:00
A.J. Beamon 2f67328a0c Update the tenant special keys submodule to support multiple sub-ranges. This will enable future work that allows configuring tenants at the same time as creating them. 2022-06-30 15:03:37 -07:00
Jon Fu cd41e62a10 provide workaround for json parser and bytestring not being preserved properly 2022-06-30 13:25:29 -07:00
Marian Dvorsky 0baa76c430
Add AtomicOpsCorrectness client API workload test (#7419)
Add AtomicOpsCorrectness client API workload test
2022-06-30 18:47:16 +02:00
Marian Dvorsky 1b7b6f1b6f
Add getRange workload to apitester (#7461)
* Add getRange workload to apitester

* Run clang-format

* Clear the results vector first

* Run clang-format
2022-06-29 14:00:25 +02:00
Jon Fu 9a484f2dca attempt to issue all tenant reads first 2022-06-28 13:46:44 -07:00
Jon Fu 0c79714d70 Merge branch 'main' of github.com:apple/foundationdb into jfu-mako-active-tenants 2022-06-28 11:25:36 -07:00
Jon Fu 5f7f846388 change getTenant to read from special key space and also batch tenant clear ranges 2022-06-28 11:24:53 -07:00
Jon Fu 59fd6ac7a3 remove check for total tenants on createNewTransaction 2022-06-28 09:26:48 -07:00
Markus Pilman 1bdbd89839 Merge remote-tracking branch 'origin/main' into features/cmake-refactor-dirs 2022-06-28 08:52:03 -06:00
Markus Pilman ce30774bc1 fixed api tester build 2022-06-27 19:27:57 -06:00
Markus Pilman 03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Jon Fu 2dccbd7e70 Merge branch 'main' of github.com:apple/foundationdb into jfu-mako-active-tenants 2022-06-27 14:20:50 -07:00
A.J. Beamon 7dee9fb4fd Add a test for the new JSON output 2022-06-27 12:34:40 -07:00
Jon Fu 4bd586a8e4 introduce tenant_batch_size argument and change tenant creation/deletion to do many in one transaction. Also introduce getTenant to fdb_api.hpp 2022-06-24 11:56:15 -07:00
Markus Pilman a47ed89018 Linux fixes and addressed review comments 2022-06-23 20:52:13 -06:00
Markus Pilman 38e100ebc5 flow bindings are compiling 2022-06-23 19:06:05 -06:00
Markus Pilman d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
Jon Fu 9158d79e8e increase batch size and fix asymptomatic off-by-one error 2022-06-23 10:37:44 -07:00
Vaidas Gasiunas e28a8401fb
Update coordinator list from cluster file (#7382)
* Log failed connection attempts in monitorProxies

* Update coordinator list from the cluster file after failing to connect to all coordinators

* Wiggle and upgrade test with legacy version monitoring; updating tests to use 7.1.9

* Update coordinator list from the cluster file: addressing review comments

* Update coordinator list from the cluster file: addressing review comments

* Wait on future for all setAndPersistConnectionString calls
2022-06-23 09:22:09 +02:00
Jon Fu 67d2630507 fix off-by-1 error in batches 2022-06-22 14:26:31 -07:00
Jon Fu d0a3e64ed3 change batch tenant creation logic 2022-06-22 13:45:12 -07:00
Jon Fu e3ea9f5c73 Merge branch 'main' of github.com:apple/foundationdb into jfu-mako-active-tenants 2022-06-22 10:31:58 -07:00
Jon Fu aa8982c1fb allow only active tenants to be specified in args and use batch tenant creation in populate 2022-06-22 10:31:12 -07:00
Jon Fu c473a94250 modify changeNewTransaction logic 2022-06-17 13:56:33 -07:00
Josh Slocum 34e6a8f942
Merge pull request #7399 from sfc-gh-jslocum/bg_tenant_improvements
Bg tenant improvements
2022-06-17 11:19:41 -05:00
Josh Slocum 1cc466e068 fixes and review comments 2022-06-17 08:17:44 -05:00
sfc-gh-tclinkenbeard 111e28d0ea Merge remote-tracking branch 'origin/main' into fix-unused-var 2022-06-16 17:20:18 -07:00
Jon Fu 97115f17d5 update usage message 2022-06-16 16:03:14 -07:00
Josh Slocum b3597ef3a8 Added plumbing for tenant-aware purge granules 2022-06-16 13:04:34 -05:00