Commit Graph

642 Commits

Author SHA1 Message Date
Steven Li 01234720e7 update prerelease package format (#6733) 2022-04-08 18:15:40 -05:00
Aaron Molitor 16e96e9103 attempt to fix rpm dependency issue for release candidate build 2022-04-08 18:15:40 -05:00
Jingyu Zhou 284dbdc473 Fix cmake USE_GPERFTOOLS compiling error
Otherwise, I got weird errors like pthread library is not found when checking
compilers.
2022-04-08 13:56:06 -07:00
Josh Slocum ace2750ae3 Build AWS SDK as part of FDB, and use it to acquire blob credentials. This supports IAM role-based authentication, in addition to all existing credential types. 2022-03-24 10:09:06 -05:00
Andrew Noyes 68c03a7e32
Jemalloc integration fixes (#6626)
* Set default for USE_JEMALLOC initially in ConfigureCompiler

Instead of trying to change the value later on. This fixes the valgrind
build, which was previously incorrectly getting jemalloc involved.

* Check aligned_alloc result for null

And OOM if so - don't assert

* Check that we can allocate magazines with no internal fragmentation

We may want to do this so that the jemalloc heap profiler has some
knowledge of FastAlloc

* Populate TestFile field for noSim tests in TestHarness

* Remove handling for nonexistent "ActualRun"
2022-03-17 15:17:27 -07:00
Andrew Noyes ccc11cbdb2
Apply jemalloc fix, and always build jemalloc (#6610)
* Apply jemalloc fix, and always build jemalloc

See https://github.com/jemalloc/jemalloc/pull/1924

* Move USE_JEMALLOC to flow/config.h

* Add unit test to validate fix
2022-03-16 13:19:28 -07:00
Vaidas Gasiunas f4bb82ad56
Merge pull request #6433 from sfc-gh-vgasiunas/vgasiunas-system-tester
A framework for C API tests
2022-03-16 10:49:52 +01:00
Steve Atherton 415d1958d4
Build configuration fix: USE_JEMALLOC=OFF had no effect, did not disable using jemalloc on platforms that support it. (#6590) 2022-03-14 11:00:34 -07:00
Vaidas Gasiunas bb7d26a6eb Merge remote-tracking branch 'apple/main' into vgasiunas-system-tester 2022-03-10 19:13:54 +01:00
Yi Wu e6950abae6 Use jemalloc for SQLite/Redwood page cache allocation 2022-03-09 11:48:08 -08:00
Vaidas Gasiunas ef96231db0 Merge remote-tracking branch 'apple/main' into vgasiunas-system-tester 2022-03-04 16:25:18 +01:00
Vaidas Gasiunas b8386f15d6 ApiTester: configuration for ApiCorrectness workload; Better error handing and reporting 2022-03-04 16:22:49 +01:00
zhenfeng yang 847cff72b7
Add a target to generate profile (#6396)
add a target to generate profile
2022-02-28 17:00:30 -06:00
Andrew Noyes 24f37aceb7 Avoid conflict between versioned client/server rpms
Building locally this seems to cause a conflict between the versioned
client and server rpms. Not sure why this doesn't always show up.
2022-02-28 15:24:29 -06:00
vikasgupta8 595f50ce26 indentation corrected 2022-02-15 13:03:12 +00:00
root 15159a5deb resolved format error 2022-02-11 09:57:24 +00:00
vikasgupta8 edfff755bf added support for ppc64le 2022-02-11 06:17:15 +00:00
Kao Makino 886ee280d8
Merge pull request #6297 from sfc-gh-zyang/zyang/fdo
support Profile Guided Optimization
2022-02-09 16:03:41 -08:00
Andrew Noyes 863449c983
Allow access to transaction options through a go `ReadTransaction` interface (#6318)
* Run go unit tests in ctest

* Remove unnecessary cgo directives

* go: Allow access to TransactionOptions from ReadTransaction

Closes #6265

* Try go get instead of go install

My understanding is that this should download dependencies (e.g. xerrors)

* Add -d flag to match go command for building libraries

* Avoid concurrent calls to `go get`
2022-02-09 14:41:21 -08:00
Aaron Molitor 96dd86ebf8 update RocskDB and Boost
add Finduring, and include into fdbserver
add BOOST asio/uring settings to fdbserver compile
move portable rocks, liburing up to be configurable at build time.
2022-02-09 10:48:18 -06:00
Mohamed Oulmahdi 23fa27cfde
Merge pull request #6333 from Doxense/add-missing-tests
Add missing tests
2022-02-09 01:03:08 +01:00
sfc-gh-zyang 0968935665 rename params and add disallow users to set both of PROFILE_INSTR_GENERATE and PROFILE_INSTR_USE 2022-02-07 12:34:27 -08:00
sfc-gh-zyang ddb05b6e8e fix typo during merge 2022-02-04 17:03:15 -08:00
sfc-gh-zyang 93f22ea381 Merge https://github.com/sfc-gh-zyang/foundationdb into zyang/fdo 2022-02-04 16:59:16 -08:00
sfc-gh-zyang 2ec6633ef3 support fdo 2022-02-04 16:51:45 -08:00
sfc-gh-zyang 58034d2554 support fdo 2022-02-04 16:46:17 -08:00
Andrew Noyes 84d26c7c13 Don't run downgrade tests for valgrind. Closes #6322 2022-02-03 16:53:56 -08:00
Mohamed Oulmahdi d50ec42c87 Enhance the detection of missing test files 2022-02-03 22:26:10 +01:00
sfc-gh-zyang a263191387 support instrumented build 2022-01-26 16:52:47 -08:00
Andrew Noyes 03b4a3a74a Add TRACE_PC_GUARD_INSTRUMENTATION_LIB 2022-01-20 13:24:23 -08:00
Mohamed Oulmahdi 0ca3d96506 Reintroduce HAVE_OPENSSL parameter 2022-01-17 19:10:20 +01:00
Mohamed Oulmahdi b0f4c65388 Enable TLS Support for Windows (main) 2022-01-17 09:46:33 +01:00
Markus Pilman 8b77c8d79a Remove unused OpenSSL code 2022-01-11 10:34:37 -08:00
Andrew Noyes a3f37df94a
Merge pull request #6175 from sfc-gh-anoyes/anoyes/delete-non-virtual-destructor
Enable -Wdelete-non-virtual-dtor for clang build
2022-01-05 15:41:59 -08:00
Andrew Noyes 38a97a2e8f Increase default timeout to 5 minutes for add_fdbclient_test 2021-12-22 15:23:05 -08:00
Andrew Noyes fd33d31ff5 Enable -Wdelete-non-virtual-dtor for clang build
We had been disabling -Wdelete-non-virtual-dtor, because this seems to be done intentionally in the generated code of the actor compiler. I spent some time trying to rewrite it in a way that doesn't literally delete/destroy through a pointer to a base class without a virtual destructor, but I was unable to come up with something that passes correctness. My best guess is that we do this so that we can destroy actor state classes, call callbacks registered on the actor SAV, and then destroy the SAV.

Anyway now we'll detect new usages of deleting through a pointer to a base class without a virtual destructor.
2021-12-20 16:19:31 -08:00
Aaron Molitor 95d33cb363 copy packaging/docker to PROJECT_BINARY_DIR (undoing part of #5994),
fetch commit_sha from source_code_directory (don't assume we're in the source tree anymore),
allow custom tag (if a parameter is passed in as $1)
update README.md
2021-12-15 15:23:17 -08:00
A.J. Beamon 496000477c
Merge pull request #6144 from sfc-gh-ajbeamon/unify-flags
Convert command line arguments to use hyphens rather than underscores
2021-12-15 10:47:32 -08:00
zhenfeng yang 76974605c1
support lto (#6140)
* support lto

* use relative path

* add another variable to control lto

* remove unnecessary code
2021-12-14 15:45:07 -08:00
A.J. Beamon f29f487823
Unify flags (#25)
* Unify flags implementation and change help text in backup.actor.cpp
Description

Testing

* Keep LOG_GROUP unchanged

Description

Testing

* Transfer the hyphens to underscores for internal options and user's input, EXCEPT leading hyphens

Description

Testing

* Use a deep copy of the user's input flag to do the match

Description

Testing

* Convert the _ to - in Option arrays of backup.actor.cpp

Description

Testing

* Transter _ to - for files:
        TLSConfig.actor.h, fdbcli.actor.cpp, fdbserver.actor.cpp, FileConverter.h, FileConverter.cpp

Description

Testing

* Change another way to unify flag: using SO_O_ICASE_HYPHEN_AND_UNDERSCORE to determine whether we do the conversion in function IsEqual

Description

Testing

* Change the config command's name from SO_O_ICASE_HYPHEN_AND_UNDERSCORE to SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Update the comment for the SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Fix left underscore in SOption arrays

Description

Testing

* Convert _ to - in several files for commands

Description

Testing

* Make the FDBService and fdbmonitor backward compatible

Description

Testing

* Fix bugs about pointers

Description

Testing

* Check underscore and hyphen at the same time for --knob_, --localily_ and --test_
And fix bugs in fdbmonitor and FDBService
Description

Testing

* Simplify the function in fdbmonitor and FDBService about retrieving arguments.
And fix some documents in masterserver.actor.cpp

Description

Testing

* Convert _ to - for knob in the setKnob functions

Description

Testing

* Convert - to _ in the setKnob functions

Description
Since key in the knob related maps only contain _

Testing

* Rename varialbe name in the fdbmonitor and FDBService for clarification

Description

Testing

Co-authored-by: Chang Liu <chang.liu@snowflake.com>
2021-12-14 08:44:39 -08:00
Andrew Noyes 79116ea887 Use boost coroutines by default for sanitizers 2021-12-08 16:44:35 -08:00
Aaron Molitor 9f5197b54c use FDB_VERSION in lieu of PROJECT_VERSION (package scripts) 2021-11-29 15:11:20 -08:00
Aaron Molitor 77db63274b use FDB_VERSION in lieu of PROJECT_VERSION or CMAKE_PROJECT_VERSION
write FDB_VERSION to a file to avoid regex golf in other build scripts
2021-11-29 15:11:20 -08:00
Aaron Molitor 08b635d405 rename prerelease_string, replace PRERELEASE with SNAPSHOT 2021-11-29 15:11:20 -08:00
Andrew Noyes b6fd402a3c Add option to use boost or libcoro
By default, use boost everywhere except windows and linux x86 (for
performance reasons)
2021-11-29 13:14:15 -08:00
sfc-gh-tclinkenbeard c745002607 Merge remote-tracking branch 'origin/master' into add-format-warning 2021-11-18 11:13:03 -08:00
Aaron Molitor 124e7eb4f2 refactor docker directory
- make release images ALL based from centos:7
- keep eks images that are based from amazonlinux:2, but create strong alignment with release (centos7) images
- use multi-stage Dockerfile(s)
- have a single build-images script
- remove per image directories (which would only contain scripts that are copied into images)
- remove empty requirements.txt file for sidecar.py (the pip command, package and version are in the Dockerfile)
- don't copy docker into PROJECT_BINARY_DIR
- incorporate latest changes already in master
2021-11-17 16:29:07 -08:00
Tao Lin 9a07a117d2
Not warn on constant-logical-operand when compile with clang (#5990) 2021-11-17 16:23:20 -08:00
sfc-gh-tclinkenbeard 766a05d33c Merge remote-tracking branch 'origin/master' into add-format-warning 2021-11-17 12:14:01 -08:00
sfc-gh-tclinkenbeard 62efeb6812 Merge remote-tracking branch 'origin/master' into add-format-warning 2021-11-12 11:50:36 -08:00
Markus Pilman 5af465aa29 FDB compiles on Apple Sillicon 2021-11-10 20:05:38 -07:00
sfc-gh-tclinkenbeard 7f09bdbda4 Remove -Wclass-memaccess for clang 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard 27db99a77f Enable clang comment warnings 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard c7b28abaf0 Enable unknown-pragmas warning for clang 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard 8a69aa08a2 Enable tautological-pointer-compare clang warning 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard d0c9cf4fb0 Enable mismatched-tags clang warning 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard ebcc023b6f Enable missing-field-initializers clang warning 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard 25257f6f87 Enable unused-function warning for clang 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard b0cec29849 Enable unused-local-typedef clang warning 2021-11-01 14:18:31 -07:00
sfc-gh-tclinkenbeard 13bb7838aa Enable clang -Wformat warning 2021-10-30 21:07:38 -07:00
Renxuan Wang f39e34cabe Use --enable-prof switch when building jemalloc。
I got errors like `<jemalloc>: Invalid conf pair: prof:true` when trying to use jemalloc. Referring to https://stackoverflow.com/questions/27422508/heap-dump-fails-with-jemalloc-mcllctl, seems that we are missing out this flag. https://github.com/jeffgriffith/native-jvm-leaks#building.
2021-10-21 10:08:55 -07:00
A.J. Beamon de863d170d Add some tests for timeout behavior with disconnected clusters 2021-09-29 13:16:17 -06:00
Neethu Haneesha Bingi c0e6b09fe3 Enforcing 6.22.1 rocksdb version and enabling readRange deadline option. 2021-09-17 08:53:23 -07:00
Markus Pilman 548b9d75f6 Fix filename 2021-09-16 15:15:06 -07:00
Markus Pilman 1ad8ca7dda Fix naming and change filename 2021-09-16 15:15:06 -07:00
Markus Pilman 2c39f4b2f0 Make naming consistent with Fedora guidelines 2021-09-16 15:15:06 -07:00
Markus Pilman 56aff2bc14 change package name to make yum happy 2021-09-16 15:15:06 -07:00
Mohamed Oulmahdi e93f6e323a Fix test paths for on Windows 2021-09-15 16:30:58 -06:00
Andrew Noyes 0d362aaf42 Fix the versioned packages so that they don't modify the source dir 2021-08-25 15:14:08 -07:00
Andrew Noyes 92c52d44d8 make_directory is deprecated. Use file(MAKE_DIRECTORY ...) 2021-08-25 11:30:26 -07:00
Andrew Noyes 3b4fed4f18
Merge pull request #4818 from sfc-gh-anoyes/anoyes/pkg_tester
Add pkg_tester
2021-08-24 12:22:50 -07:00
Andrew Noyes 8bbcfca8a0 Run all tests for versioned packages 2021-08-19 16:32:22 +00:00
Markus Pilman 2e8a469b27 Incomplete implementation 2021-08-19 16:32:22 +00:00
Pierre Zemb b1dce03e0d
Add warning on CMake when bindings requirements are not reached 2021-08-19 17:39:03 +02:00
Eric Dong c6ed0f94bf Fix the wording of a CMake message for SSL on Windows 2021-08-13 13:02:49 -07:00
Lukas Joswiak a605fb3852
Merge pull request #5026 from sfc-gh-ljoswiak/fixes/alp6
Actor sampling
2021-08-11 13:44:17 -07:00
Markus Pilman eb908546bb
Merge pull request #5318 from PierreZ/cmake/options
add CMake's option for bindings
2021-08-06 13:06:24 -06:00
Pierre Zemb 415bf2afc0
add CMake's option for bindings 2021-08-05 21:55:45 +02:00
Markus Pilman defa41b502 Don't use -latomic with clang 2021-08-05 11:51:44 -06:00
Xiaoge Su f6beda6623 When using non-Apple clang, link to libatomic 2021-08-05 07:32:41 -07:00
Xiaoge Su 38ac3481ff fixup! Fix the indentation 2021-08-05 07:32:41 -07:00
Xiaoge Su 1d8068d48f Ensure CompileBoost work on Mac
This is tested on a local Mac with most recent upgrades of xcode.
2021-08-05 07:32:41 -07:00
Xiaoge Su 1d685524db Let CMake use /opt/boost_1_72_0_clang/ boost when the compiler is clang
For boost::context, clang abi and gcc abi are not compatible. In docker
environment, we have different builds, which are located at

  gcc version:    /opt/boost_1_72_0
  clang version:  /opt/boost_1_72_0_clang

In this patch, when clang compiler is detected, the clang version of
boost::context is used.
2021-08-05 07:32:41 -07:00
Lukas Joswiak 5dc9a97230 Merge branch 'master' into fixes/alp6 2021-08-01 20:42:52 -07:00
Andrew Noyes 353efe7db2
Merge pull request #5264 from sfc-gh-tclinkenbeard/fix-more-clang-warnings
Enable more warnings for `clang`
2021-07-29 15:43:54 -07:00
Daniel Smith d5174b2d98 Update RocksDB version 2021-07-29 12:58:31 -07:00
sfc-gh-tclinkenbeard 94a65865d9 Merge remote-tracking branch 'origin/master' into fix-clang-warnings 2021-07-28 12:29:27 -07:00
sfc-gh-tclinkenbeard c74047c665 Merge remote-tracking branch 'origin/master' into fix-more-clang-warnings 2021-07-28 11:51:02 -07:00
Lukas Joswiak 3eed4084e2 Merge branch 'master' into fixes/alp6 2021-07-27 11:26:53 -07:00
Markus Pilman e1ec5f9aa4 Fix bug where tests wouldn't be added to correctness package 2021-07-27 11:38:25 -06:00
Markus Pilman 91e5498357 disable simulation in ctest by default 2021-07-27 11:17:55 -06:00
Lukas Joswiak 59d535149e Merge branch 'master' into fixes/alp6 2021-07-27 10:07:18 -07:00
sfc-gh-tclinkenbeard cedaa72c0f Enable reorder and reorder-ctor warnings 2021-07-24 23:23:52 -07:00
sfc-gh-tclinkenbeard 13fe89ff06 Enable self-assign warning for clang 2021-07-23 17:32:01 -07:00
sfc-gh-tclinkenbeard 8d4569b63c Enable dangling-else warning for clang 2021-07-23 17:26:37 -07:00
sfc-gh-tclinkenbeard 8fe3e45fc6 Enable more clang warnings 2021-07-23 10:07:04 -07:00
Markus Pilman 35f835548d Fix Java integration tests 2021-07-21 13:46:39 -06:00
Chaoguang Lin 5e83281742 Add the option to customize add_fdbclient_test timeout time 2021-07-15 06:55:54 +00:00
Chaoguang Lin 41a9470058 Add comments, remove debugging info 2021-07-14 20:24:16 +00:00
Chaoguang Lin 1e3d3811cc Merge branch 'master' of github.com:apple/foundationdb into extend-fdbclient-test-for-multiple-processes 2021-07-14 20:20:39 +00:00