Summary:
Port libFuzzer's fork mode to Windows.
Implement Windows versions of MkDir, RmDir, and IterateDirRecursive to do this.
Don't print error messages under new normal uses of FileSize (on a non-existent file).
Implement portable way of piping output to /dev/null.
Fix test for Windows and comment fork-sigusr.test on why it won't be ported to Win.
Reviewers: zturner
Reviewed By: zturner
Subscribers: kcc, zturner, jdoerfert, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58513
llvm-svn: 355019
Underlying condition for throttling is "has large mmap'd regions" (i.e.,
shadow memory) and not sanitizers in general (e.g., UBSan does not need
to be throttled).
Rename parallelism group `darwin-64bit-sanitizer` to `shadow-memory` and
apply it unconditionally to all tests which require it. We can then have
all the Darwin throttling logic in one place in the commen lit config.
Throttle sanitizer_common unit tests. Configuration was previously
missing from sanitizer_common/Unit/lit.site.cfg.
Reviewed by: kubamracek
Differential Revision: https://reviews.llvm.org/D58677
llvm-svn: 355018
Summary:
This CL adds a standalone vector class that will be used by the scoped
strings when they land. We reimplement our own vector class because we
can't use the std library one.
It's mostly borrowed from the current sanitizer_common one, with LLVM
code style changes.
Additionnally a casing change in a function name that slipped through
the previous review (the function isn't used yet).
Reviewers: vitalybuka, eugenis, flowerhack, dmmoore415, mcgrathr, morehouse
Reviewed By: vitalybuka
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58689
llvm-svn: 354999
As discussed elsewhere: LLVM uses cpp as its C++ source extension; the
sanitizers should too. This updates files in hwasan.
Patch generated by
for f in lib/hwasan/*.cc ; do svn mv $f ${f%.cc}.cpp; done
followed by
for f in lib/hwasan/*.cpp ; do sed -i '' -e '1s/\.cc -/.cpp /' $f; done
CMakeLists.txt updated manually.
Differential Revision: https://reviews.llvm.org/D58620
llvm-svn: 354989
For 'cascadelake' this is adding a 'avx512vnni' feature check to the 0x55 skylake-avx512 model check. These CPUs use the same model number and only differ in the stepping number. But the feature flag is simpler than collecting all the stepping numbers.
For 'znver2' this is just syncing with LLVM's Host.cpp.
llvm-svn: 354927
Summary:
When running the standalone main on a large corpus, I eventually get a
EMFILE error ("Too many open files").
Patch by Paul Chaignon
Reviewers: kcc, vitalybuka
Reviewed By: vitalybuka
Subscribers: lebedev.ri, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58622
llvm-svn: 354918
Summary:
The current error message can cause confusion if llvm-config can't be
executed for reasons other than "not found". In my example, cross
compiling generated an llvm-config binary for aarch64 which couldn't be
executed natively. Instead of telling me that the error was with the
file architecture, it reports the file as not being present.
Reviewers: pcc
Subscribers: dberris, javed.absar, kristof.beyls, jdoerfert, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58647
llvm-svn: 354907
Summary:
This CL adds the platform specific code for Fuchsia, Linux & Android,
as well as some tests related to those (more tests to come later).
While some of it is pretty much a straight port of the existing scudo &
sanitizer_common code, the memory mapping functions have been reworked
a bit to fit the limited usage scenario that Scudo has for them.
For Fuchsia, I can now track the Vmar/Vmo pair for memory mappings if
there is an intent to grow or decommit some mapping (that will be
useful for the Primary).
Reviewers: eugenis, vitalybuka, mcgrathr, phosek, flowerhack, morehouse, dmmoore415
Reviewed By: vitalybuka, morehouse
Subscribers: kcc, dvyukov, srhines, mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58184
llvm-svn: 354895
/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/projects/compiler-rt/test/asan/TestCases/Posix/bcmp_test.cc:14:12: error: CHECK: expected string not found in input
// CHECK: {{#1.*bcmp}}
^
<stdin>:2:57: note: scanning from here
==34677==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffee93777c4 at pc 0x0001068a7285 bp 0x7ffee9377770 sp 0x7ffee9376ef8
^
<stdin>:6:20: note: possible intended match here
#2 0x106888e77 in main bcmp_test.cc:12
^
llvm-svn: 354888
Summary:
pthread_sigmask is just like sigprocmask, except that its behavior in
multithreaded programs is explicitly specified. Sanitizers were lacking
a common interceptor for pthread_sigmask (although some specific
sanitizers defined custom version), which lead to false positives
(at least in msan) when using this function.
The interceptor implementation, and its test are based on the equivalent
code for sigprocmask.
Reviewers: eugenis, vitalybuka
Subscribers: kubamracek, delcypher, jfb, jdoerfert, llvm-commits, #sanitizers
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58382
llvm-svn: 354874
Summary:
I have not introduced a separate hook for `bcmp()` as I don't think there
should be any reason for a sanitizer to treat it differently from `memcmp()`.
This is only enabled when building on POSIX with GNU extensions.
Context: this is to avoid losing coverage when emitting `bcmp() == 0` instead
of `memcmp() == 0` in llvm, see https://reviews.llvm.org/D56593.
Reviewers: mgorny, krytarowski, vitalybuka, dvyukov
Subscribers: kubamracek, dberris, delcypher, jdoerfert, #sanitizers, llvm-commits, jyknight
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58379
llvm-svn: 354851
Summary:
The llvm-cov tool needs to be able to find coverage names in the
executable, so the .lprfn and .lcovmap sections cannot be merged into
.rdata.
Also, the linker merges .lprfn$M into .lprfn, so llvm-cov needs to
handle that when looking up sections. It has to support running on both
relocatable object files and linked PE files.
Lastly, when loading .lprfn from a PE file, llvm-cov needs to skip the
leading zero byte added by the profile runtime.
Reviewers: vsk
Subscribers: hiraditya, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58661
llvm-svn: 354840
This is a second attempt at r342652 using a TLS callback instead of an
interceptor.
In long-running builds we've seen some ASan complaints during thread creation
that we suspect are due to leftover poisoning from previous threads whose
stacks occupied that memory. This patch adds a callback that unpoisons the
stack memory when a thread exits.
Differential Revision: https://reviews.llvm.org/D58641
llvm-svn: 354836
Summary:
ASan and Scudo tests are adding "-android" to test arch.
There are no tests that depend on it as far as I can see.
If necessary, do this instead:
REQUIRES: aarch64-target-arch && android
Reviewers: pcc, vitalybuka
Subscribers: srhines, kubamracek, mgorny, javed.absar, kristof.beyls, cryptoad, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58532
llvm-svn: 354829
FastUnwindStack -> UnwindFast
SlowUnwindStack -> UnwindSlow
Stack is redundant, verb should come first.
SlowUnwindStackWithContext(uptr pc, void *context, u32 max_depth) ->
SlowUnwindStack
WithContext is redundant, since it is a required parameter.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58551
llvm-svn: 354696
__asan_handle_vfork was unpoisoning the wrong part of the stack.
Adjust the test to catch this reliably (current failure is
non-deterministic).
llvm-svn: 354627
Summary:
The goal of the test to check that msan does not crash when code is racy on __cxa_atexit. Original crash was caused by race condition in the glibc. With
the msan patch the msan does not crashes however the race is still there and the test triggers it.
Because the test relies on triggering of undefined behavior results are not
very predictable and it may occasionally crashes or hangs.
I don't see how to reasonably improve the test, so I remove it.
Reviewers: eugenis, peter.smith
Subscribers: jfb, jdoerfert, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58396
llvm-svn: 354377
This is another follow up to r354212 which is broken on Darwin when
cross-compiling runtimes to Linux when it ignores the -fuse-ld=lld
linker flag and attempts to use the host linker when performing the
compiler identification. Upon investigation, I noticed that setting
the project with appropriate list of languages makes the error go
away and it shouldn't hurt either.
Differential Revision: https://reviews.llvm.org/D58372
llvm-svn: 354350
In compiler-rt we have the notion of a `fast` and a `slow` stack
unwinder. Darwin currently only supports the fast unwinder.
From reading the code, my understanding is that
`BufferedStackTrace::Unwind` can be called with `bp=0, stack_top=0,
stack_bottom=0, request_fast_unwind=false`. If
`request_fast_unwind=true`, then we alos need to supply bp, stack_top,
and stack_bottom.
However, `BufferedStackTrace::Unwind` uses
`StackTrace::WillUseFastUnwind` which will adapt `request_fast_unwind`
if the requested unwinder is not supported. On Darwin, the result is
that we don't pass actual values for bp, stack_top, and stack_bottom,
but end up using the fast unwinder. The tests then fail because we only
print the topmost stack frame.
This patch adds a check to `WillUseFastUnwind` at the point of usage to
avoid the mismatch between `request_fast_unwind` and what `Unwind`
actually does. I am also interested in cleaning up the
`request_fast_unwind` machinery so this patch just the simplest thing
possible so I can enable the tests.
Reviewers: vitalybuka, vsk
Differential Revision: https://reviews.llvm.org/D58156
llvm-svn: 354282
According to the logs and local debugging there were two issues:
1) tsan tests listed libc++.a before the source file. That's usually
ok for shared libraries, but the linker will not add symbols from
a static library unless needed at that time. As a result the tests
that rely upon symbols from the library (and not only include the
headers) had undefined references.
To solve this I'm adding a new substitution %link_libcxx_tsan which
expands to libc++.a if available.
2) The target Fuzzer-x86_64-Test linked in SANITIZER_TEST_CXX_LIBRARIES
which defaults to -lstdc++. This resulted in error messages like
hidden symbol '_ZdlPv' is not defined locally
hidden symbol '_Znwm' is not defined locally
when using GNU gold (ld.bfd and lld are fine). Removing the linkage
is fine because we build a custom libc++ for that purpose.
llvm-svn: 354231
This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:
1) The resulting libFuzzer doesn't expose C++ internals and looks
like a plain C library.
2) We don't have to manually link in libstdc++ to provide cxxabi.
3) The sanitizer tests cannot interfere with an installed version
of libc++.so in LD_LIBRARY_PATH.
Differential Revision: https://reviews.llvm.org/D58013
llvm-svn: 354212
Add missed value "libcxxabi" and introduce SANITIZER_TEST_CXX for linking
unit tests. This needs to be a full C++ library and cannot be libcxxabi.
Recommit r354132 which I reverted in r354153 because it broke a sanitizer
bot. This was because of the "fixes" for pthread linking, so I've removed
these changes.
Differential Revision: https://reviews.llvm.org/D58012
llvm-svn: 354198
Serial execution on iOS devices is not specific to sanitizers. We want
to throttle all on-device tests. Pull the setting of the
parallelism_group up into the common lit configuration file.
Rename `darwin-ios-device-sanitizer` to `ios-device`. This group is not
specific to sanitizers and (theoretically) independent from the host OS.
Note that we don't support running unit tests on-device (there are no
configurations generated for that). If that ever changes, we also need
this configuration in `unittests/lit.common.unit.cfg`.
Reviewers: delcypher
Differential Revision: https://reviews.llvm.org/D58209
llvm-svn: 354179
There is an ambiguity between ::SizeClassMap (the typedef declared near
the start of this file) and __sanitizer::SizeClassMap (found by the
'using namespace __sanitizer;' near the start of this file).
Historically a Clang bug has meant that the error was not diagnosed, but
soon Clang will start diagnosing it. Explicitly qualify this use of
SizeClassMap so that it finds __sanitizer::SizeClassMap rather than
being ill-formed due to ambiguity.
llvm-svn: 354174
Summary:
When symbols are unavailable, the current code prints
sp: ... pc: ... (null) (null)
instead of module name + offset.
Change the output to include module name and offset, and also to match
the regular sanitizer stack trace format so that it is recognized by
symbolize.py out of the box.
Reviewers: kcc, pcc
Subscribers: kubamracek, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D58267
llvm-svn: 354157
Summary:
With tag_in_free=1, malloc() can not assume that the memory is untagged,
and needs to retag is to 0.
Reviewers: pcc, kcc
Subscribers: kubamracek, jfb, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D58158
llvm-svn: 354155
Add missed value "libcxxabi" and introduce SANITIZER_TEST_CXX for linking
unit tests. This needs to be a full C++ library and cannot be libcxxabi.
Differential Revision: https://reviews.llvm.org/D58012
llvm-svn: 354132
Summary:
The changes in https://reviews.llvm.org/D44847 cause load time failure
due to lprofMergeValueProfData in Android libs enabled with profile generation:
"dlopen failed: cannot locate symbol "lprofMergeValueProfData" referenced by..."
Marking lprofMergeValueProfData as hidden so the correct in-module definition
is picked by the linker.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: efriedma, xur, davidxl, llvm-commits
Differential Revision: https://reviews.llvm.org/D55893
llvm-svn: 354064
Summary:
Pre 2.27 libc can run same atexit handler twice
We will keep MSanAtExitRecord and reset fun to mark it as executed.
Fix PR40162
Reviewers: eugenis
Subscribers: jfb, jdoerfert, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58221
llvm-svn: 354005
This patch adds functions for managing fibers:
__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()
See the added tests for use examples.
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889
[The previous commit of this change was reverted,
this is a resubmit with a squashed fix for check_analyze.sh
and COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED]
llvm-svn: 353947
This function doesn't use anything MSVC specific but works fine
for any _WIN32 target.
Differential Revision: https://reviews.llvm.org/D58106
llvm-svn: 353918
Summary:
A link error was encountered when using the Red Hat Developer Toolset.
In the RHDTS, `libstdc++.so` is a linker script that may resolve symbols
to a static library. This patch places `-lstdc++` later in the ordering.
Reviewers: sfertile, nemanjai, tstellar, dberris
Reviewed By: dberris
Subscribers: dberris, mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58144
llvm-svn: 353905
buildgo.sh puts most of sanitizer_common together in a single source
file. These single-letter macros end up affecting a lot of unrelated
code; #undef them as early as possible.
llvm-svn: 353902
This patch adds functions for managing fibers:
__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()
See the added tests for use examples.
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889
llvm-svn: 353817
Otherwise this propagates all the way to CMake and results in an error
during configuration. We check and handle the result and report warning
separately so this is not changing the behavior.
Differential Revision: https://reviews.llvm.org/D58086
llvm-svn: 353784
Summary:
This test instruments the following code with coverage, runs the fuzzer
once, and asserts that there are uncovered PCs. The ARM64 backend
optimizes this code using the `csel` (Conditional select) instruction,
which removes all branching from the resulting machine code. The test
then fails because we do not have any uncovered PCs. The easiest
solution for now is to turn off optimization for the DSOs used in this
test.
```
int DSO1(int a) {
if (a < 123456)
return 0;
return 1;
}
```
rdar://47646400
Reviewers: kcc
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58087
llvm-svn: 353780
We shouldn't be treating runtimes builds as standalone builds since
we have enough of the context loaded into the runtimes environment.
Differential Revision: https://reviews.llvm.org/D57992
llvm-svn: 353601
The __sanitizer_acquire_crash_state function has int return type, but the
fuzzer's external function definitions give it bool.
Places where __sanitizer_acquire_crash_state is declared:
include/sanitizer_common/sanitizer_interface_defs.h
lib/sanitizer_common/sanitizer_interface_internal.h
lib/sanitizer_common/sanitizer_common.cc
lib/fuzzer/FuzzerExtFunctions.def (this is the only bool)
llvm-svn: 353596
Jonathan Metzman