Commit Graph

623 Commits

Author SHA1 Message Date
Markus Pilman 1de37afd52
Make TEST macros C++ only (#7558)
* proof of concept

* use code-probe instead of test

* code probe working on gcc

* code probe implemented

* renamed TestProbe to CodeProbe

* fixed refactoring typo

* support filtered output

* print probes at end of simulation

* fix missed probes print

* fix deduplication

* Fix refactoring issues

* revert bad refactor

* make sure file paths are relative

* fix more wrong refactor changes
2022-07-19 13:15:51 -07:00
Ata E Husain Bohra 24b2de8de8 BlobFile Encryption and compression support
Description

Testing
2022-07-14 17:04:14 -07:00
Markus Pilman fa9e623fc1
Merge pull request #7459 from sfc-gh-ljoswiak/features/module-link-tests
Add test executables to catch missing symbols in modules
2022-07-11 17:31:11 -06:00
Andrei Gorneanu 3a0444b569 Merge remote-tracking branch 'upstream/main' into add_intel_compiler_support 2022-07-07 17:30:09 +02:00
Lukas Joswiak 618f8455c4 Add test executables to catch missing symbols
Currently, we have code in different folders like `flow/` and `fdbrpc/`
that should remain isolated. For example, `flow/` files should not
include functionality from any other modules. `fdbrpc/` files should
only be able to include functionality from itself and from `flow/`.
However, when creating a shared library, the linker doesn't complain
about undefined symbols -- this only happens when creating an
executable. Thus, for example, it is possible to forward declare an
`fdbclient` function in an `fdbrpc` file and then use it, and nothing
will break (when it should, because this is illegal).

This change adds dummy executables for a few modules (`flow`, `fdbrpc`,
`fdbclient`) that will cause a linker error if there are included
symbols which the linker can't resolve.
2022-07-06 14:49:33 -07:00
Steven Li d4273c4dda revise prerelease pkg format 2022-06-30 18:04:38 -05:00
Mohamed Oulmahdi 6fcbf5784c Formatting 2022-06-30 00:10:47 +02:00
Mohamed Oulmahdi 7af9b2fae3 Enhance TOML dependencies 2022-06-29 23:45:28 +02:00
Markus Pilman 41181cda26 fdbserver compiling (but linking still fails) 2022-06-27 18:54:58 -06:00
Markus Pilman d31fb30f57 fdbclient compiling 2022-06-27 18:31:05 -06:00
Markus Pilman 88af6b403f fdbrpc compiling 2022-06-27 18:11:55 -06:00
Markus Pilman 03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
goandrei 06004e8b25 Add Intel compiler support 2022-06-26 14:02:07 +00:00
Markus Pilman a47ed89018 Linux fixes and addressed review comments 2022-06-23 20:52:13 -06:00
Markus Pilman 38e100ebc5 flow bindings are compiling 2022-06-23 19:06:05 -06:00
Markus Pilman d35445a868 enforce include modularization in cmake 2022-06-23 14:37:35 -06:00
Andrew Noyes 849b1cd29a
Update to the latest jemalloc release (#7362)
Also remove our patch, since the fix is already present in the new
release.
2022-06-10 14:46:21 -07:00
Markus Pilman d141347500
Merge pull request #7282 from Doxense/fix-windows-tests
Fix windows tests
2022-06-08 08:18:47 -06:00
Josh Slocum 872c25148f remove BUILD_ALWAYS from awssdk.cmake 2022-06-02 13:40:56 -05:00
Mohamed Oulmahdi ae8014a181 Disable iterator debugging for Windows 2022-05-31 11:57:41 +02:00
Junhyun Shim 02b2f97e99
mTLS test helpers (#7218)
* Add TLS option to (local_cluster|tmp_cluster).py

* Add TLS-enabled C API test
2022-05-23 12:47:51 +02:00
Jingyu Zhou b5ce29ee5f
Merge pull request #7143 from sfc-gh-anoyes/anoyes/use-libcoro-valgrind
Use libcoro for valgrind
2022-05-16 13:17:48 -07:00
Markus Pilman 452315ee78
Build mockkms and add mockkms test (#7153) 2022-05-12 15:07:02 -07:00
Andrew Noyes 39e085938c Use libcoro for valgrind 2022-05-12 09:47:51 -07:00
Ata E Husain Bohra 33ae398268
REST KmsConnector implementation (#6994)
* REST KmsConnector implementation

Description
  diff-1: Address review comments.
          Add utility interface to Platform namespace to
          create and operate on tmpfile
 diff-2: Address review comments
         Link Boost::filesystem to CMake build process

Major changes includes:
1. Implement REST based KmsConnector implementation.
2. Salient features of the connector:
 2.1. Two required configuration are:
   a. Discovery KMS URLs - enable KMS discovery on bootstrap
   b. Endpoint path configuration to construct URI to fetch/refresh
      encryption keys
   c. Configuration to provide "validationTokens" to connect with
      external KMS. Patch implements file-based token validation scheme.
 2.2. On startup, RESTKmsConnector discovers KMS Urls and caches
      them in-memory. Extracts "validationTokens" based on input config.
 2.3. Expose endpoints to allow fetch/refresh of encryption keys.
 2.4. Defines JSON format to interact with external KMS - request &
      response payload format.
3. Extend Platform namespace with an interface to create and operate on
   tmp files.
4. Update Platform 'readFileBytes' and 'writeFileBytes' to leverage
   fstream supported implementation.

NOTE: KMS URLs fetched after initial discovery will be persisted using
      DynamicKnobs. It is TODO at the moment and shall be completed
      once DynamicKnobs is feature complete

Testing

Unit test to validation following:
1. Parsing on "validation tokens" logic.
2. Construction and parsing of REST JSON request and response strings.
2022-05-07 13:18:35 -07:00
sfc-gh-tclinkenbeard 8ea68154bf Remove WITH_TLS CMake variable 2022-05-02 22:45:00 -07:00
sfc-gh-tclinkenbeard ea00ae3a25 Remove DISABLE_TLS CMake argument 2022-05-02 22:31:46 -07:00
Sam Gwydir 5403a29ecb
add WolfSSL support (#6682)
remove extraneous include
2022-04-28 16:53:38 -07:00
Andrew Noyes 9f8e2a4517
Test compiling an fdb_c app in pkg_tester (#6940)
* Use execstack to test executable stack permissions

The output of readelf wasn't stable

* Tee output to stdout in pkg_tester "run" function

* Test compiling an app that includes fdb_c.h

Featuring both cmake and pkg_tester, since we intend to support those in
our packages.

* Fix packaging bug

The libdir in the pkg-config file in the packages was wrong
2022-04-26 11:54:52 -07:00
Josh Slocum 3c036300cc Addressing review comments 2022-04-20 14:34:19 -05:00
Josh Slocum 8fa25aa013 Adding blob granule client test target and local cluster setup 2022-04-20 10:15:43 -05:00
Steven Li 01234720e7 update prerelease package format (#6733) 2022-04-08 18:15:40 -05:00
Aaron Molitor 16e96e9103 attempt to fix rpm dependency issue for release candidate build 2022-04-08 18:15:40 -05:00
Jingyu Zhou 284dbdc473 Fix cmake USE_GPERFTOOLS compiling error
Otherwise, I got weird errors like pthread library is not found when checking
compilers.
2022-04-08 13:56:06 -07:00
Josh Slocum ace2750ae3 Build AWS SDK as part of FDB, and use it to acquire blob credentials. This supports IAM role-based authentication, in addition to all existing credential types. 2022-03-24 10:09:06 -05:00
Andrew Noyes 68c03a7e32
Jemalloc integration fixes (#6626)
* Set default for USE_JEMALLOC initially in ConfigureCompiler

Instead of trying to change the value later on. This fixes the valgrind
build, which was previously incorrectly getting jemalloc involved.

* Check aligned_alloc result for null

And OOM if so - don't assert

* Check that we can allocate magazines with no internal fragmentation

We may want to do this so that the jemalloc heap profiler has some
knowledge of FastAlloc

* Populate TestFile field for noSim tests in TestHarness

* Remove handling for nonexistent "ActualRun"
2022-03-17 15:17:27 -07:00
Andrew Noyes ccc11cbdb2
Apply jemalloc fix, and always build jemalloc (#6610)
* Apply jemalloc fix, and always build jemalloc

See https://github.com/jemalloc/jemalloc/pull/1924

* Move USE_JEMALLOC to flow/config.h

* Add unit test to validate fix
2022-03-16 13:19:28 -07:00
Vaidas Gasiunas f4bb82ad56
Merge pull request #6433 from sfc-gh-vgasiunas/vgasiunas-system-tester
A framework for C API tests
2022-03-16 10:49:52 +01:00
Steve Atherton 415d1958d4
Build configuration fix: USE_JEMALLOC=OFF had no effect, did not disable using jemalloc on platforms that support it. (#6590) 2022-03-14 11:00:34 -07:00
Vaidas Gasiunas bb7d26a6eb Merge remote-tracking branch 'apple/main' into vgasiunas-system-tester 2022-03-10 19:13:54 +01:00
Yi Wu e6950abae6 Use jemalloc for SQLite/Redwood page cache allocation 2022-03-09 11:48:08 -08:00
Vaidas Gasiunas ef96231db0 Merge remote-tracking branch 'apple/main' into vgasiunas-system-tester 2022-03-04 16:25:18 +01:00
Vaidas Gasiunas b8386f15d6 ApiTester: configuration for ApiCorrectness workload; Better error handing and reporting 2022-03-04 16:22:49 +01:00
zhenfeng yang 847cff72b7
Add a target to generate profile (#6396)
add a target to generate profile
2022-02-28 17:00:30 -06:00
Andrew Noyes 24f37aceb7 Avoid conflict between versioned client/server rpms
Building locally this seems to cause a conflict between the versioned
client and server rpms. Not sure why this doesn't always show up.
2022-02-28 15:24:29 -06:00
vikasgupta8 595f50ce26 indentation corrected 2022-02-15 13:03:12 +00:00
root 15159a5deb resolved format error 2022-02-11 09:57:24 +00:00
vikasgupta8 edfff755bf added support for ppc64le 2022-02-11 06:17:15 +00:00
Kao Makino 886ee280d8
Merge pull request #6297 from sfc-gh-zyang/zyang/fdo
support Profile Guided Optimization
2022-02-09 16:03:41 -08:00
Andrew Noyes 863449c983
Allow access to transaction options through a go `ReadTransaction` interface (#6318)
* Run go unit tests in ctest

* Remove unnecessary cgo directives

* go: Allow access to TransactionOptions from ReadTransaction

Closes #6265

* Try go get instead of go install

My understanding is that this should download dependencies (e.g. xerrors)

* Add -d flag to match go command for building libraries

* Avoid concurrent calls to `go get`
2022-02-09 14:41:21 -08:00