Commit Graph

58 Commits

Author SHA1 Message Date
Junhyun Shim eb0bc70ea8
Fix flaky ArenaBlock wipe test (#9960) 2023-04-14 20:40:56 +02:00
Yanqin Jin 0faa6538d1
Remove an assertion for UB (#9865)
Original code comment says
"there's no hard guarantee about the above equality and the result could vary by platform, malloc implementation, and tooling instrumentation (e.g. ASAN, valgrind)"

Therefore, we should remove the assertion on UB. Since removal of the UB makes the rest of the test case
less meaningful, we commented out the entire test case.

Furthermore, we have a case in which, even without ASAN or other instrumentation, e.g. Valgrind, etc, the equality does not hold.

Profile: team
Commit hash: 3164cadc6f
Command: devRetryCorrectnessTest bin/fdbserver -r simulation -f tests/fast/RandomUnitTests.toml -s 3939330597 -b on  --crash --trace_format json
2023-04-03 12:01:25 -07:00
A.J. Beamon 64b6a5d257 Allow boolean parameters to be nested inside of namespaces or classes 2023-03-30 15:09:59 -07:00
Junhyun Shim b6f0d2095a
Cases where newBuf == buf assertion fails have been observed in Valgrind (#9528) 2023-03-01 15:49:11 +01:00
A.J. Beamon 4a38bb4c3f Allow performing assert comparisons (e.g. ASSERT_EQ) with any traceable type 2023-02-24 12:53:01 -08:00
Junhyun Shim 06ee5f1e76
Disable client memory wipe test for all XSAN builds (#9409) 2023-02-21 14:10:44 +01:00
Junhyun Shim d9c126a2d9
Introduce WipedString for Arena block holding AuthZ tokens (#9381)
* Enable secure allocation mode in Arena

This mode allows zeroing out blocks holding sensitive data after use

* Introduce WipedString to all token-holding memory

Also introduce a option flag "sensitive"

* Make pointer equivalency a hard requirement for non-ASAN builds

So that we can detect when Arena/malloc/memory-wipe behavior changes
2023-02-16 10:44:32 +01:00
A.J. Beamon 985bbe66d4 Add support for flatMap in optional and erroror. Allow calling map with a lambda without specifying the return type template parameter. 2022-12-15 15:32:02 -08:00
A.J. Beamon f1c5eb55ba Add overloads of the Optional::map and ErrorOr::map functions that allow you to more easily convert to a value's members. Add a similar mapRef function if the wrapped type is a pointer-type. 2022-12-14 15:22:54 -08:00
Dan Adkins fc1d29f429
Arena: avoid null-pointer reference in ASSERT (#8875)
Narrow the self-reference ASSERT in ArenaBlock::dependOn to only
the cases where we're trying to make a reference from self -> other.

It's allowed for self.getPtr() to be null, in which case we create
the ArenaBlock anew and there's no danger of self-reference.

Fixes Issue #8869.
2022-11-18 18:08:58 -05:00
Dan Adkins a864216d97 Make Arena::dependOn(Arena) a no-op in case they're referring to the same ArenaBlock.
The intent is to have one ArenaBlock live at least at long as another. If they're
the same, that's accomplished without doing any additional bookkeeping.
2022-11-10 19:36:59 -08:00
Dan Adkins 2a8f5c7c3c Assert that we don't try to have an ArenaBlock depend on itself.
It introduces a cycle in the `next` pointers in the ArenaBlockRef's,
which results in infinite recursion as we follow them in `getTotalSize`.

This doesn't fix the underlying problem, but gives a better failure
than a stack overflow.
2022-11-10 19:36:59 -08:00
A.J. Beamon 7324f28441 Add the ability for StringRef.eat to return whether a separator was found. Add a serialize function and equality operators to ClusterConnectionString. Remove unused variable. 2022-07-19 11:55:58 -07:00
Markus Pilman 03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Andrew Noyes 38db712e7a
Make ASAN arena aware (#7336) 2022-06-13 13:24:02 -07:00
Andrew Noyes 966402a7a2
Allocate at least sizeof(ArenaBlock) for an ArenaBlock (#6770)
* Allocate at least sizeof(ArenaBlock) for an ArenaBlock

* Fix message pack unit test

Previously we were using only the 4 least significan bits as the length
of a message pack string, but it should be 5 according to https://github.com/msgpack/msgpack/blob/master/spec.md#str-format-family
2022-04-05 18:14:10 -07:00
Andrew Noyes 3b7389fbc5 Use malloc for > 256 in FastAllocated, allocateFast, Arena 2022-03-22 11:01:33 -07:00
sfc-gh-tclinkenbeard a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard 44b0e05538 Merge remote-tracking branch 'origin/main' into arena-get-size-boolean-param 2022-03-16 22:53:30 -07:00
Steve Atherton 6cef85e268 Change assertions to not rely on the first argument being evaluated before the second. 2022-03-16 19:00:26 -07:00
sfc-gh-tclinkenbeard 0ae9ba5fd0 Add FastInaccurateEstimate boolean parameter for Arena::getSize 2022-03-16 16:54:27 -07:00
Steve Atherton 6f16702042 Merge branch 'main' of https://github.com/apple/foundationdb into redwood-bug-fixes-and-memory-leak 2022-02-26 18:00:18 -08:00
Andrew Noyes 7a9217a392
Add contrib/debug_determinism (#6389)
* Add contrib/debug_determinism

Add an instrumentation-based technique for debugging unseen mismatches. Also guard a few existing sources of nondeterminism that don't affect unseen with the DEBUG_DETERMINISM macro.

Also change the simulated run loop to not run as the only task inside the real run loop, since that was a source of nondeterminism.

Also fix nondeterminism from calling timer_int

* Add StorageMetadataType::currentTime

Basically a deterministic-in-simulation version of timer_int that we can
use instead of timer_int for StorageMetadataType::createdTime
2022-02-25 12:54:31 -08:00
Steve Atherton 735c5697d0 Added fast size estimates to Arena which are O(1) and are usually accurate. When inaccurate, a full size scan will update all estimates in the tree. 2022-02-18 03:25:28 -08:00
A.J. Beamon 39a7dc7a2d Add two unit tests for the new hashing functionality 2022-01-28 15:56:58 -08:00
Andrew Noyes eb15746e41 Dereference to get result before marking "no access" 2021-06-02 16:07:04 -07:00
Steve Atherton a31e4f622f Changed ArenaBlockRef to use 32 bit aligned4kBuffer size. 2021-05-16 03:58:05 -07:00
Steve Atherton 2298567c2b Use of aligned_alloc() for 4k pages causes too much wasted virtual memory. Added new 4k-aligned fast allocator, and changed Arena::allocatedAlignedBuffer() to be 4k-specific, now called Arena::allocate4kAlignedBuffer(). 2021-05-14 23:12:00 -07:00
Steve Atherton 16f9f6e75c Added a test of Arena dependencies and destruction when using the new aligned buffer feature. 2021-05-06 02:05:09 -07:00
Steve Atherton f8a8bf315b Added Arena::allocateAlignedBuffer() to get an aligned memory block owned by an Arena, and ArenaPage uses this. 2021-05-05 15:00:12 -07:00
Andrew Noyes a63dc3cfd6 Inform LSAN that we have unaligned pointers 2021-03-13 00:33:47 +00:00
FDB Formatster df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes f91079dbe7 Use camelCase 2020-10-22 21:51:00 +00:00
Andrew Noyes dc5e09ba62 Fix normal build 2020-10-22 16:34:37 +00:00
Andrew Noyes ab4eb627c2 Replace with environment variable
We need to change the behavior of the allocator before main is called,
so a knob won't work.
2020-10-22 16:31:48 +00:00
Andrew Noyes 79163947a6 Add FLOW_KNOBS->VALGRIND_PRECISE 2020-10-22 16:31:48 +00:00
sfc-gh-tclinkenbeard 0ac08f6a9b Replace NULL with nullptr in flow 2020-09-20 11:31:49 -07:00
Jingyu Zhou cb582e58c9 Avoid iterator+2 or -2 passes over the boundary 2020-07-30 13:42:36 -07:00
Jingyu Zhou 9a63882cd4 Fix unit test failure 2020-07-30 11:01:52 -07:00
Andrew Noyes d6d374818c Actually test copy 2020-07-22 19:57:47 +00:00
Andrew Noyes 4195d3ab3b Fix compilation, test copies 2020-07-22 19:57:47 +00:00
Andrew Noyes a25689e6e5 Add append tests, fix append 2020-07-22 19:57:47 +00:00
Andrew Noyes 82b70a0fa3 Fix test bug 2020-07-22 19:57:47 +00:00
Andrew Noyes 03a45e979d Add unit tests for VectorRef and SmallVectorRef 2020-07-22 19:57:47 +00:00
Andrew Noyes 6caa3e470c Fix valgrind build 2020-07-09 22:05:48 +00:00
Russell Sears 4ce51c4cce
Merge pull request #2768 from atn34/atn34/arena-cpp
Check for memory errors in arena-allocated memory
2020-07-06 09:43:28 -07:00
Andrew Noyes 10e094c6d7 Address review comments 2020-06-11 15:43:31 +00:00
Andrew Noyes e772f3a3f1 Separate knob to control valgrind arena instrumentation 2020-05-07 22:07:33 +00:00
A.J. Beamon 96187618a0 Fix condition to check whether allocation tracing is enabled 2020-03-16 15:12:50 -07:00
A.J. Beamon d8cfabe73b Extend the allocation tracing disabling flag to cover more parts of trace logging as a precaution. Make it possible to disable via knob. 2020-03-16 13:59:31 -07:00