Commit Graph

22610 Commits

Author SHA1 Message Date
Xiaoxi Wang e7a280ec03 format code 2022-09-21 20:49:39 -07:00
Xiaoxi Wang aaed0c8b7e Actor to DDQueue methods 2022-09-21 17:57:40 -07:00
Xiaoxi Wang 141bd99133 Merge branch 'main' of https://github.com/apple/foundationdb into feature/main/txnProcessor_team 2022-09-21 15:35:26 -07:00
Jingyu Zhou a8d3282250
Merge pull request #8137 from xis19/main
Reads and reports cpu.stat
2022-09-21 15:33:57 -07:00
Josh Slocum 4794ebabcc
Merge pull request #8262 from sfc-gh-dzhou/purge-fixes
blob: java api fixes (purge + verify)
2022-09-21 17:25:54 -05:00
Xiaoxi Wang 5500ec8126 replace BgDDMountainChopper and BgDDValleyFiller with BgDDLoadRebalance 2022-09-21 15:11:04 -07:00
Xiaoxi Wang 9358aea097 fix busy loop with correct error handling in valley filler 2022-09-21 14:58:34 -07:00
neethuhaneesha c4f59ba654
Merge pull request #8261 from neethuhaneesha/rocksdb-cachesize
RocksDB block cache size increase.
2022-09-21 14:50:31 -07:00
Markus Pilman 1c71f5c778
Merge pull request #8253 from sfc-gh-mpilman/documentation/testharness-and-codeprobes
Upgrade sphinx and document test harness and code probes
2022-09-21 15:22:18 -06:00
Andrew Noyes e7c171737e
Swap the order of curl and zlib for awssdk (#8268)
According to
https://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_node/ld_3.html,
dependencies should appear after. Since curl depends on zlib, put zlib
after curl.
2022-09-21 14:20:36 -07:00
Dennis Zhou d5623766d2 blob: fix verifyBlobRange() MVC api 2022-09-21 14:07:16 -07:00
Dennis Zhou 4ea4546cb6 blob/java: verifyBlobRange() with latestVersion 2022-09-21 14:07:16 -07:00
Dennis Zhou 86eac7386f blob/java: fix verifyBlobRange() calling wrong function
verifyBlobRange() accidentally called list_blobbified_ranges().
2022-09-21 14:04:58 -07:00
Dennis Zhou e353169a50 blob: teach purge about latestVersion
This teaches purgeBlobGranules about latestVersion and rejects
versions <= 0.
2022-09-21 14:04:58 -07:00
Josh Slocum b7be356a98
removing overhead of local change feed stream copy (#8205) 2022-09-21 12:44:51 -07:00
Josh Slocum 6270016bed
Seq insert perf fixes main (#8264)
* Force flushing granules post-split to guarantee parent feeds get cleaned up

* fixing bug and cleaning up split finalize code
2022-09-21 12:36:02 -07:00
Trevor Clinkenbeard 999e20a7a4
Merge pull request #8258 from sfc-gh-tclinkenbeard/remove-applyHmacKeyDerivationFunc
Remove dead code for `applyHmacKeyDerivationFunc`
2022-09-21 11:53:35 -07:00
neethuhaneesha bc9e806f1c RocksDB block cache size increase. 2022-09-21 11:41:27 -07:00
Xiaoxi Wang f9e0230b86 DDQueue constructor with ITxnProcessor 2022-09-21 10:56:22 -07:00
Ata E Husain Bohra 1f015b7e1a
Lookup encryption cipher key while writing RAW_ACCESS mutation (#8256)
* Lookup encryption cipher key while writing RAW_ACCESS mutation

Description

Patch update writeMutation handling of RAW_ACCESS mutations by
looking up Encryption keys if not already part of 'cipherKeys'
map.

Testing

FuzzApiCorrectnessClean.toml -s 3194376849 - issue was found
devCorrectnesRun - 100k
2022-09-21 09:59:09 -07:00
Dennis Zhou 2caf0c7726 blob/java: add purge at latestVersion 2022-09-21 09:46:55 -07:00
Markus Pilman 9c062a220e clarify statistics of TestHarness 2022-09-21 09:29:49 -06:00
Markus Pilman a8cec4f05c clarify how code probes are reported 2022-09-21 09:27:28 -06:00
Markus Pilman 90b48e862e
Apply suggestions from code review
Co-authored-by: Trevor Clinkenbeard <trevor.clinkenbeard@snowflake.com>
Co-authored-by: Bharadwaj V.R <bharadwaj.vr@snowflake.com>
2022-09-21 08:28:01 -06:00
Junhyun Shim a5fd42990f
Merge pull request #8250 from sfc-gh-jshim/fix-python-fdb_c-lookup
Fix Python binding's fdb_c lookup
2022-09-21 10:43:24 +02:00
Markus Pilman 833a1950b1
Merge pull request #8257 from sfc-gh-tclinkenbeard/fix-compound-workload-check
Fix `CompoundWorkload::check` method
2022-09-20 22:49:18 -06:00
sfc-gh-tclinkenbeard 436cdff912 Remove dead code for applyHmacKeyDerivationFunc 2022-09-20 21:08:36 -07:00
sfc-gh-tclinkenbeard 715b58686d Fix CompoundWorkload::check method 2022-09-20 20:55:36 -07:00
Xiaoge Su 970463223c
Merge branch 'main' into main 2022-09-20 16:56:56 -07:00
Xiaoge Su 6befa98937 fixup! Remove the Windows trial fix 2022-09-20 16:48:48 -07:00
Hui Liu c237186ec3
Merge pull request #8252 from sfc-gh-huliu/fixtest
Fix correctness test: skip lock check in MoveKeysWorkload
2022-09-20 16:34:22 -07:00
Trevor Clinkenbeard 3b5117ca92
Merge pull request #8233 from sfc-gh-mpilman/features/disable-failure-injection-for-workload
All workloads to disable some failures
2022-09-20 16:17:35 -07:00
Ata E Husain Bohra 8b0f6e2d24
Encryption: Remove memcpy due to computeToken (#8211)
Description

Major changes include:
1. Eliminate memcpy due to authToken
2. Update TraceEvent message format
3. Remove redundant code.

Testing

flow/BlobCipher unit test
EncryptionOps.toml
devRunCorrectness - 100k
2022-09-20 15:58:53 -07:00
Nim Wijetunga eadb769cfa
Encrypt Backup Mutation Log (#8159)
* encrypt backup mutation log

* format

* address pr comments

* format

* fix bug

* revert knobs

* address pr comments
2022-09-20 15:43:39 -07:00
Markus Pilman 24aced6d4a Upgrade sphinx and document test harness and code probes 2022-09-20 16:43:02 -06:00
Chaoguang Lin 9628561235
Add DataDistributionMetrics workload into correctness packages, (#8237)
which makes the code probe hit in nightly tests.
2022-09-20 15:33:15 -07:00
Hui Liu 52f1bef8ec Fix correctness test: skip lock check in MoveKeysWorkload 2022-09-20 15:16:56 -07:00
Junhyun Shim 2e51a110f2 Fix Python binding's fdb_c lookup
All python documentations indicate you shouldn't put prefixes like 'lib' or
suffixes like '.so', '.dll', or '.dylib' in argument to ctypes.util.find_library(),
unlike ctypes.CDLL() which demands an exact file name.
I suppose this case hasn't been tested.
2022-09-20 22:47:04 +02:00
Xiaoxi Wang 97fd5878d9 change DDTeamCollection constructor 2022-09-20 13:00:28 -07:00
Andrew Noyes 65ec5ef37f
Statically link libcurl for the aws sdk (#8226) 2022-09-20 11:23:18 -07:00
Markus Pilman 57806dda91
Merge pull request #8245 from sfc-gh-anoyes/anoyes/initialize-verificationMode
Fix a UBSAN diagnostic caused by use of uninitialized memory
2022-09-20 11:32:32 -06:00
Vaidas Gasiunas 118d693783
Reduce the size of debug symbols in FDB binaries (#8244)
* Compress debug symbols; generate less debug symbols by default

* Generate full debug symbols for CMAKE_BUILD_TYPE=Debug builds
2022-09-20 19:07:44 +02:00
Markus Pilman 56a6c685c5
Merge pull request #8239 from sfc-gh-huliu/fix
pass dbInfo to compond workload
2022-09-20 10:37:30 -06:00
Jingyu Zhou 887c6ab27a
Merge pull request #8240 from jzhou77/fix-tuple
Added 7.1.22, 7.1.23 release notes
2022-09-20 08:57:03 -07:00
Andrew Noyes 4ac5498f2e Fix a UBSAN diagnostic caused by use of uninitialized memory
The diagnostic is

fdbserver/workloads/DiskFailureInjection.actor.cpp:57:64: runtime error: load of value 208, which is not a valid value for type 'bool'

The fix is simply to initialize the default value of verificationMode
2022-09-20 08:54:45 -07:00
Hui Liu 428ecc679f pass dbInfo to compond workload 2022-09-20 08:31:14 -07:00
Lukas Joswiak c97377f48e
Remove durable cluster ID write (#8238) 2022-09-20 07:56:07 -07:00
Vaidas Gasiunas 97eddbc06a
Extend ApiTester with support for testing database operations (#8221)
* ApiTester: Refactoring removing unnecessary transaction actor classes

* ApiTester: enable executing non-transactional database operations

* ApiTester: a test for fdb_database_list_blobbified_ranges

* ApiTester: Fix memory ownership of the end key in randomGetRangeOp

* ApiTester: reuse get range result validation in blob granule tests
2022-09-20 16:21:38 +02:00
Junhyun Shim 2e2dd119fe
Increase test coverage of TLS server-client combination (#8057)
Explore all combinations where the server and the client
each takes one of the following TLS/cert-chain configuration:
  No TLS, no chain, valid chain, expired chain, valid self-signed, expired self-signed

Negative cases are indirectly tested using timeouts:
  i.e. if it times out, it means it's not connecting

+ Fix TSAN data race error in original test
  (Calling FlowTransport::bind() with an active network thread running in background)
+ Properly redirect-and-capture subprocess stdout instead of just printing
  (CTest tends to truncate subprocess console outputs)
2022-09-20 13:47:35 +02:00
Jingyu Zhou 890681a52f Added 7.1.22, 7.1.23 release notes 2022-09-19 20:35:32 -07:00