Summary:
When ASan reports an issue the contents of the system log buffer
(`error_message_buffer`) get flushed to the system log (via
`LogFullErrorReport()`). After this happens the buffer is not cleared
but this is usually fine because the process usually exits soon after
reporting the issue.
However, when ASan runs in `halt_on_error=0` mode execution continues
without clearing the buffer. This leads to problems if more ASan
issues are found and reported.
1. Duplicate ASan reports in the system log. The Nth (start counting from 1)
ASan report will be duplicated (M - N) times in the system log if M is the
number of ASan issues reported.
2. Lost ASan reports. Given a sufficient
number of reports the buffer will fill up and consequently cannot be appended
to. This means reports can be lost.
The fix here is to reset `error_message_buffer_pos` to 0 which
effectively clears the system log buffer.
A test case is included but unfortunately it is Darwin specific because
querying the system log is an OS specific activity.
rdar://problem/55986279
Reviewers: kubamracek, yln, vitalybuka, kcc, filcab
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76749
Disable symbolization of results, since llvm-symbolizer cannot start
due to restricted readlink(), causing the test to die with SIGPIPE.
Author: Ilya Leoshkevich
Reviewed By: Evgenii Stepanov
Differential Revision: https://reviews.llvm.org/D76576
Temporarily revert "tsan: fix leak of ThreadSignalContext for fibers"
because it breaks the LLDB bot on GreenDragon.
This reverts commit 93f7743851.
This reverts commit d8a0f76de7.
When creating and destroying fibers in tsan a thread state
is created and destroyed. Currently, a memory mapping is
leaked with each fiber (in __tsan_destroy_fiber).
This causes applications with many short running fibers
to crash or hang because of linux vm.max_map_count.
The root of this is that ThreadState holds a pointer to
ThreadSignalContext for handling signals. The initialization
and destruction of it is tied to platform specific events
in tsan_interceptors_posix and missed when destroying a fiber
(specifically, SigCtx is used to lazily create the
ThreadSignalContext in tsan_interceptors_posix). This patch
cleans up the memory by inverting the control from the
platform specific code calling the generic ThreadFinish to
ThreadFinish calling a platform specific clean-up routine
after finishing a thread.
The relevant code causing the leak with fibers is the fiber destruction:
void FiberDestroy(ThreadState *thr, uptr pc, ThreadState *fiber) {
FiberSwitchImpl(thr, fiber);
ThreadFinish(fiber);
FiberSwitchImpl(fiber, thr);
internal_free(fiber);
}
I would appreciate feedback if this way of fixing the leak is ok.
Also, I think it would be worthwhile to more closely look at the
lifecycle of ThreadState (i.e. it uses no constructor/destructor,
thus requiring manual callbacks for cleanup) and how OS-Threads/user
level fibers are differentiated in the codebase. I would be happy to
contribute more if someone could point me at the right place to
discuss this issue.
Reviewed-in: https://reviews.llvm.org/D76073
Author: Florian (Florian)
struct stack_t on Linux x86_64 has internal padding which may be left
uninitialized. The check should be replaced with multiple checks for
individual fields of the struct. For now, remove the check altogether.
Summary:
Move interceptor from msan to sanitizer_common_interceptors.inc, so that
other sanitizers could benefit.
Adjust FixedCVE_2016_2143() to deal with the intercepted uname().
Patch by Ilya Leoshkevich.
Reviewers: eugenis, vitalybuka, uweigand, jonpa
Reviewed By: eugenis, vitalybuka
Subscribers: dberris, krytarowski, #sanitizers, stefansf, Andreas-Krebbel
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76578
This test case fails due to different handling of weak items between
LLD and LD on PPC. The issue only occurs when the default linker is LLD
and the test case is run on a system where ASLR is enabled.
A recent change to MemorySSA caused LLVM to start optimizing the call to
'f(x)' into just 'x', despite the 'noinline' attribute. So try harder to
prevent this optimization from firing.
After a first attempt to fix the test-suite failures, my first recommit
caused the same failures again. I had updated CMakeList.txt files of
tests that needed -fcommon, but it turns out that there are also
Makefiles which are used by some bots, so I've updated these Makefiles
now too.
See the original commit message for more details on this change:
0a9fc9233e
This includes fixes for:
- test-suite: some benchmarks need to be compiled with -fcommon, see D75557.
- compiler-rt: one test needed -fcommon, and another a change, see D75520.
and follow-ups:
a2ca1c2d "build: disable zlib by default on Windows"
2181bf40 "[CMake] Link against ZLIB::ZLIB"
1079c68a "Attempt to fix ZLIB CMake logic on Windows"
This changed the output of llvm-config --system-libs, and more
importantly it broke stand-alone builds. Instead of piling on more fix
attempts, let's revert this to reduce the risk of more breakages.
After the format change from D69471, there can be more than one section
in an object that contains coverage function records. Look up each of
these sections and concatenate all the records together.
This re-enables the instrprof-merging.cpp test, which previously was
failing on OSes which use comdats.
Thanks to Jeremy Morse, who very kindly provided object files from the
bot I broke to help me debug.
An execution count goes missing for a constructor, this needs
investigation:
http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/45132/
```
/home/buildbots/ppc64be-clang-test/clang-ppc64be/llvm/compiler-rt/test/profile/instrprof-merging.cpp:28:16:
error: V1: expected string not found in input
A() {} // V1: [[@LINE]]{{ *}}|{{ *}}1
<stdin>:28:32: note: possible intended match here
28| | A() {} // V1: [[@LINE]]{{ *}}|{{ *}}1
```
Hope this fixes:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-android/builds/27977/steps/run%20lit%20tests%20%5Bi686%2Ffugu-userdebug%2FN2G48C%5D/logs/stdio
```
: 'RUN: at line 8'; UBSAN_OPTIONS=suppressions=/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/compiler_rt_build_android_i686/test/ubsan/Standalone-i386/TestCases/Misc/Output/nullability.c.tmp.supp /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/compiler_rt_build_android_i686/test/ubsan/Standalone-i386/TestCases/Misc/Output/nullability.c.tmp 2>&1 | count 0
--
Exit Code: 1
Command Output (stderr):
--
Expected 0 lines, got 2.
```
Not sure what this would be printing though, a sanitizer initialization message?
Summary:
When -dfsan-event-callbacks is specified, insert a call to
__dfsan_mem_transfer_callback on every memcpy and memmove.
Reviewers: vitalybuka, kcc, pcc
Reviewed By: kcc
Subscribers: eugenis, hiraditya, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75386
Summary:
For now just insert the callback for stores, similar to how MSan tracks
origins. In the future we may want to add callbacks for loads, memcpy,
function calls, CMPs, etc.
Reviewers: pcc, vitalybuka, kcc, eugenis
Reviewed By: vitalybuka, kcc, eugenis
Subscribers: eugenis, hiraditya, #sanitizers, llvm-commits, kcc
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75312
Summary:
Sanitizer tests don't entirely pass on an R device. Fix up all the
incompatibilities with the new system.
Reviewers: eugenis, pcc
Reviewed By: eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75303
Generally we ignore interceptors coming from called_from_lib-suppressed libraries.
However, we must not ignore critical interceptors like e.g. pthread_create,
otherwise runtime will lost track of threads.
pthread_detach is one of these interceptors we should not ignore as it affects
thread states and behavior of pthread_join which we don't ignore as well.
Currently we can produce very obscure false positives. For more context see:
https://groups.google.com/forum/#!topic/thread-sanitizer/ecH2P0QUqPs
The added test captures this pattern.
While we are here rename ThreadTid to ThreadConsumeTid to make it clear that
it's not just a "getter", it resets user_id to 0. This lead to confusion recently.
Reviewed in https://reviews.llvm.org/D74828
Like was done before in D67999 for `logbf`, this patch fixes the tests for
the internal compiler-rt implementations of `logb` and `logbl` to consider
all NaNs equivalent. Not doing so was resulting in test failures for
riscv64, since the the NaNs had different signs, but the spec doesn't
specify the NaN signedness or payload.
Fixes bug 44244.
Reviewers: rupprecht, delcypher
Reviewed By: rupprecht, delcypher
Differential Revision: https://reviews.llvm.org/D74826
Summary:
This substitution expands to the appropriate minimum deployment target
flag where thread local storage (TLS) was first introduced on Darwin
platforms. For all other platforms the substitution expands to an empty
string.
E.g. for macOS the substitution expands to `-mmacosx-version-min=10.12`
This patch adds support for the substitution (and future substitutions)
by doing a minor refactor and then uses the substitution in the relevant
TSan tests.
rdar://problem/59568956
Reviewers: yln, kubamracek, dvyukov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74802
Summary:
The number of "inputs have the Data Flow Trace" cannot be greater than
the number of inputs touching the focus function. The existing message is rather
confusing as the same log would mention a greater total number of traces a few
lines above.
Reviewers: kcc, metzman
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74779
This change makes the following lit substitutions expand to the correct
thing for macOS, iOS, tvOS, and watchOS.
%darwin_min_target_with_full_runtime_arc_support
%macos_min_target_10_11
rdar://problem/59463146
This patch defines `config.apple_platform_min_deployment_target_flag`
in the ASan, LibFuzzer, TSan, and UBSan lit test configs.
rdar://problem/59463146
Summary:
A number of testcases in TSAN are designed to deal with intermittent problems
not exist in all executions of the tested program. A script called deflake.bash
runs the executable up to 10 times to deal with the intermittent nature of the tests.
The purpose of this patch is to parameterize the hard-coded threshold above via
--cmake_variables=-DTSAN_TEST_DEFLAKE_THRESHOLD=SomeIntegerValue
When this cmake var is not set, the default value of 10 will be used.
Reviewer: dvyukov (Dmitry Vyukov), eugenis (Evgenii Stepanov), rnk (Reid Kleckner), hubert.reinterpretcast (Hubert Tong), vitalybuka (Vitaly Buka)
Reviewed By: vitalybuka (Vitaly Buka)
Subscribers: mgorny (Michal Gorny), jfb (JF Bastien), steven.zhang (qshanz), llvm-commits (Mailing List llvm-commits), Sanitizers
Tag: LLVM, Sanitizers
Differential Revision: https://reviews.llvm.org/D73707
EXCLUDE_FROM_ALL means something else for add_lit_testsuite as it does
for something like add_executable. Distinguish between the two by
renaming the variable and making it an argument to add_lit_testsuite.
Differential revision: https://reviews.llvm.org/D74168
Summary:
Forewarning: This patch looks big in #LOC changed. I promise it's not that bad, it just moves a lot of content from one file to another. I've gone ahead and left inline comments on Phabricator for sections where this has happened.
This patch:
1. Introduces the crash handler API (crash_handler_api.h).
2. Moves information required for out-of-process crash handling into an AllocatorState. This is a trivially-copied POD struct that designed to be recovered from a deceased process, and used by the crash handler to create a GWP-ASan report (along with the other trivially-copied Metadata struct).
3. Implements the crash handler API using the AllocatorState and Metadata.
4. Adds tests for the crash handler.
5. Reimplements the (now optionally linked by the supporting allocator) in-process crash handler (i.e. the segv handler) using the new crash handler API.
6. Minor updates Scudo & Scudo Standalone to fix compatibility.
7. Changed capitalisation of errors (e.g. /s/Use after free/Use After Free).
Reviewers: cryptoad, eugenis, jfb
Reviewed By: eugenis
Subscribers: merge_guards_bot, pcc, jfb, dexonsmith, mgorny, cryptoad, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73557
Summary:
An implementation for `sigaltstack` to make its side effect be visible to MSAN.
```
ninja check-msan
```
Reviewers: vitalybuka, eugenis
Reviewed By: eugenis
Subscribers: dberris, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73816
Patch by Igor Sugak.
Summary: `.cmd` is interpreted as script in windows console.
Reviewers: davidxl, rnk
Reviewed By: davidxl
Differential Revision: https://reviews.llvm.org/D73327
Summary:
In order to do this `FUZZER_SUPPORTED_OS` had to be pulled out of
`lib/fuzzer/CMakeLists.txt` and into the main config so we can use it
from the `test/fuzzer/CMakeList.txt`. `FUZZER_SUPPORTED_OS` currently
has the same value of `SANITIZER_COMMON_SUPPORTED_OS` which preserves
the existing behaviour but this allows us in the future to adjust the
supported platforms independent of `SANITIZER_COMMON_SUPPORTED_OS`. This
mirrors the other sanitizers.
For non-Apple platforms `FUZZER_SUPPORTED_OS` is not defined and
surprisingly this was the behaviour before this patch because
`SANITIZER_COMMON_SUPPORTED_OS` was actually empty. This appears to
not matter right now because the functions that take an `OS` as an
argument seem to ignore it on non-Apple platforms.
While this change tries to be NFC it is technically not because we
now generate an iossim config whereas previously we didn't. This seems
like the right thing to do because the build system was configured to
compile LibFuzzer for iossim but previously we weren't generating a lit
test config for it. The device/simulator testing configs don't run by
default anyway so this shouldn't break testing.
This change relies on the get_capitalized_apple_platform() function
added in a previous commit.
rdar://problem/58798733
Reviewers: kubamracek, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73243
Summary:
The previous code hard-coded platform names but compiler-rt's CMake
build system actually already knows which Apple platforms TSan supports.
This change uses this information to enumerate the different Apple
platforms.
This change relies on the `get_capitalized_apple_platform()` function
added in a previous commit.
rdar://problem/58798733
Reviewers: kubamracek, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73238
Summary:
The previous code hard-coded platform names but compiler-rt's CMake
build system actually already knows which Apple platforms ASan supports.
This change uses this information to enumerate the different Apple
platforms.
rdar://problem/58798733
Reviewers: kubamracek, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73232
Summary:
there is an ongoing work on interchangeable custom mutators
(https://github.com/google/clusterfuzz/pull/1333/files#r367706283)
and having some sort of signalling from libFuzzer that it has loaded
a custom mutator would be helpful.
The initial idea was to make the mutator to print something, but given
the anticipated variety of different mutators, it does not seem possible
to make all of them print the same message to signal their execution.
Reviewers: kcc, metzman
Reviewed By: metzman
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73136
Fix clear_cache_test to work on NetBSD with PaX MPROTECT enabled, that
is when creating W+X mmaps is prohibited. Use the recommended solution:
create two mappings for the same memory area, make one of them RW, while
the other RX. Copy the function into the RW area but run it from the RX
area.
In order to implement this, I've split the pointer variables to
'write_buffer' and 'execution_buffer'. Both are separate pointers
on NetBSD, while they have the same value on other systems.
I've also split the memcpy_f() into two: new memcpy_f() that only takes
care of copying memory and discards the (known) result of memcpy(),
and realign_f() that applies ARM realignment to the given pointer.
Again, there should be no difference on non-NetBSD systems but on NetBSD
copying is done on write_buffer, while realignment on pointer
to the execution_buffer.
I have tested this change on NetBSD and Linux.
Differential Revision: https://reviews.llvm.org/D72578
Dan Liew