Summary:
It's needed to use __GLIBC_PREREQ from <features.h>
tsan didn't let us to include <features.h> by using --sysroot=. to disable system includes on
anything that is not named as "tsan*posix*", "tsan*mac*", "tsan*linux*".
See compiler-rt/lib/tsan/CMakeLists.txt
Reviewers: eugenis, dvyukov, kcc
Reviewed By: kcc
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D68176
llvm-svn: 373282
Summary:
This interceptor is useful on its own, but the main purpose of this
change is to intercept libpthread initialization on linux/glibc in
order to run __msan_init before any .preinit_array constructors.
We used to trigger on pthread_initialize_minimal -> getrlimit(), but
that call has changed to __getrlimit at some point.
Reviewers: vitalybuka, pcc
Subscribers: jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D68168
llvm-svn: 373239
We can't use short granules with stack instrumentation when targeting older
API levels because the rest of the system won't understand the short granule
tags stored in shadow memory.
Moreover, we need to be able to let old binaries (which won't understand
short granule tags) run on a new system that supports short granule
tags. Such binaries will call the __hwasan_tag_mismatch function when their
outlined checks fail. We can compensate for the binary's lack of support
for short granules by implementing the short granule part of the check in
the __hwasan_tag_mismatch function. Unfortunately we can't do anything about
inline checks, but I don't believe that we can generate these by default on
aarch64, nor did we do so when the ABI was fixed.
A new function, __hwasan_tag_mismatch_v2, is introduced that lets code
targeting the new runtime avoid redoing the short granule check. Because tag
mismatches are rare this isn't important from a performance perspective; the
main benefit is that it introduces a symbol dependency that prevents binaries
targeting the new runtime from running on older (i.e. incompatible) runtimes.
Differential Revision: https://reviews.llvm.org/D68059
llvm-svn: 373035
Adding annotation function variants __tsan_write_range_pc and
__tsan_read_range_pc to annotate ranged access to memory while providing a
program counter for the access.
Differential Revision: https://reviews.llvm.org/D66885
llvm-svn: 372730
Differential Revision: https://reviews.llvm.org/D67937
M lib/builtins/fp_add_impl.inc
M lib/builtins/fp_lib.h
M lib/builtins/fp_trunc_impl.inc
llvm-svn: 372684
This fixes buildbot failures for https://reviews.llvm.org/rL372459.
(at least on PowerPC/Z )
The fix is generated by running clang-format on the error lines only.
llvm-svn: 372511
Summary:
strerror(3) on NetBSD uses internally TSD with a destructor that is never
fired for exit(3). It's correctly called for pthread_exit(3) scenarios.
This is a case when a leak on exit(3) is expected, unavoidable and harmless.
Reviewers: joerg, vitalybuka, dvyukov, mgorny
Reviewed By: vitalybuka
Subscribers: dmgreen, kristof.beyls, jfb, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D67337
llvm-svn: 372461
Summary:
The atexit(3) and __cxa_atexit() calls allocate internally memory and free on exit,
after executing all callback. This causes false positives as DoLeakCheck() is called
from the atexit handler. In the LSan/ASan tests there are strict checks triggering
false positives here.
Intercept all atexit(3) and __cxa_atexit() calls and disable LSan when calling the
real functions.
Stop tracing allocations in pthread_atfork(3) funtions, as there are performed
internal allocations that are not freed for the time of running StopTheWorld()
code. This avoids false-positives.
The same changes have to be replicated in the ASan and LSan runtime.
Non-NetBSD OSs are not tested and this code is restricted to NetBSD only.
Reviewers: dvyukov, joerg, mgorny, vitalybuka, eugenis
Reviewed By: vitalybuka
Subscribers: jfb, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D67331
llvm-svn: 372459
Summary:
Do not grab the allocator lock before calling dl_iterate_phdr. This may
cause a lock order inversion with (valid) user code that uses malloc
inside a dl_iterate_phdr callback.
Reviewers: vitalybuka, hctim
Subscribers: jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D67738
llvm-svn: 372348
Fuchsia doesn't have /proc/id/maps, so it relies on the kernel logging system
to provide the DSO map to be able to symbolize in the context of ASLR. The DSO
map is logged automatically on Fuchsia when encountering a crash or writing to
the sanitizer log for the first time in a process. There are several cases
where libFuzzer doesn't encounter a crash, e.g. on timeouts, OOMs, and when
configured to print new PCs as they become covered, to name a few. Therefore,
this change always writes to the sanitizer log on startup to ensure the DSO map
is available in the log.
Author: aarongreen
Differential Revision: https://reviews.llvm.org/D66233
llvm-svn: 372056
Summary:
.init_array gets optimized away when building with -O2 and as a result,
check_cxx_section_exists failed to pass -DCOMPILER_RT_HAS_INITFINI_ARRAY
when building crtbegin.o and crtend.o, which causes binaries linked with
them encounter segmentation fault. See https://crbug.com/855759 for
details. This change prevents .init_array section to be optimized away
even with -O2 or higher optimization level.
Subscribers: dberris, mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D67628
llvm-svn: 372038
Summary:
getauxval() is not available on NetBSD and there is no a direct equivalent.
Add a function that implements the same semantics with NetBSD internals.
Reorder the GetPageSize() functions to prefer the sysctl approach for NetBSD.
It no longer makes a difference which approach is better. Avoid changing
conditional code path.
Reviewers: vitalybuka, dvyukov, mgorny, joerg
Reviewed By: vitalybuka
Subscribers: llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D67329
llvm-svn: 371758
Summary:
This changes a few things to improve memory footprint and performances
on Android, and fixes a test compilation error:
- add `stdlib.h` to `wrappers_c_test.cc` to address
https://bugs.llvm.org/show_bug.cgi?id=42810
- change Android size class maps, based on benchmarks, to improve
performances and lower the Svelte memory footprint. Also change the
32-bit region size for said configuration
- change the `reallocate` logic to reallocate in place for sizes larger
than the original chunk size, when they still fit in the same block.
This addresses patterns from `memory_replay` dumps like the following:
```
202: realloc 0xb48fd000 0xb4930650 12352
202: realloc 0xb48fd000 0xb48fd000 12420
202: realloc 0xb48fd000 0xb48fd000 12492
202: realloc 0xb48fd000 0xb48fd000 12564
202: realloc 0xb48fd000 0xb48fd000 12636
202: realloc 0xb48fd000 0xb48fd000 12708
202: realloc 0xb48fd000 0xb48fd000 12780
202: realloc 0xb48fd000 0xb48fd000 12852
202: realloc 0xb48fd000 0xb48fd000 12924
202: realloc 0xb48fd000 0xb48fd000 12996
202: realloc 0xb48fd000 0xb48fd000 13068
202: realloc 0xb48fd000 0xb48fd000 13140
202: realloc 0xb48fd000 0xb48fd000 13212
202: realloc 0xb48fd000 0xb48fd000 13284
202: realloc 0xb48fd000 0xb48fd000 13356
202: realloc 0xb48fd000 0xb48fd000 13428
202: realloc 0xb48fd000 0xb48fd000 13500
202: realloc 0xb48fd000 0xb48fd000 13572
202: realloc 0xb48fd000 0xb48fd000 13644
202: realloc 0xb48fd000 0xb48fd000 13716
202: realloc 0xb48fd000 0xb48fd000 13788
...
```
In this situation we were deallocating the old chunk, and
allocating a new one for every single one of those, but now we can
keep the same chunk (we just updated the header), which saves some
heap operations.
Reviewers: hctim, morehouse, vitalybuka, eugenis, cferris, rengolin
Reviewed By: morehouse
Subscribers: srhines, delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D67293
llvm-svn: 371628
Summary:
This change allows to perform corpus merging in two steps. This is useful when
the user wants to address the following two points simultaneously:
1) Get trustworthy incremental stats for the coverage and corpus size changes
when adding new corpus units.
2) Make sure the shorter units will be preferred when two or more units give the
same unique signal (equivalent to the `REDUCE` logic).
This solution was brainstormed together with @kcc, hopefully it looks good to
the other people too. The proposed use case scenario:
1) We have a `fuzz_target` binary and `existing_corpus` directory.
2) We do fuzzing and write new units into the `new_corpus` directory.
3) We want to merge the new corpus into the existing corpus and satisfy the
points mentioned above.
4) We create an empty directory `merged_corpus` and run the first merge step:
`
./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus
`
this provides the initial stats for `existing_corpus`, e.g. from the output:
`
MERGE-OUTER: 3 new files with 11 new features added; 11 new coverage edges
`
5) We recreate `merged_corpus` directory and run the second merge step:
`
./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus ./new_corpus
`
this provides the final stats for the merged corpus, e.g. from the output:
`
MERGE-OUTER: 6 new files with 14 new features added; 14 new coverage edges
`
Alternative solutions to this approach are:
A) Store precise coverage information for every unit (not only unique signal).
B) Execute the same two steps without reusing the control file.
Either of these would be suboptimal as it would impose an extra disk or CPU load
respectively, which is bad given the quadratic complexity in the worst case.
Tested on Linux, Mac, Windows.
Reviewers: morehouse, metzman, hctim, kcc
Reviewed By: morehouse
Subscribers: JDevlieghere, delcypher, mgrang, #sanitizers, llvm-commits, kcc
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D66107
llvm-svn: 371620
This reverts commit r371584. It introduced a dependency from compiler-rt
to llvm/include/ADT, which is problematic for multiple reasons.
One is that it is a novel dependency edge, which needs cross-compliation
machinery for llvm/include/ADT (yes, it is true that right now
compiler-rt included only header-only libraries, however, if we allow
compiler-rt to depend on anything from ADT, other libraries will
eventually get used).
Secondly, depending on ADT from compiler-rt exposes ADT symbols from
compiler-rt, which would cause ODR violations when Clang is built with
the profile library.
llvm-svn: 371598
This patch contains the basic functionality for reporting potentially
incorrect usage of __builtin_expect() by comparing the developer's
annotation against a collected PGO profile. A more detailed proposal and
discussion appears on the CFE-dev mailing list
(http://lists.llvm.org/pipermail/cfe-dev/2019-July/062971.html) and a
prototype of the initial frontend changes appear here in D65300
We revised the work in D65300 by moving the misexpect check into the
LLVM backend, and adding support for IR and sampling based profiles, in
addition to frontend instrumentation.
We add new misexpect metadata tags to those instructions directly
influenced by the llvm.expect intrinsic (branch, switch, and select)
when lowering the intrinsics. The misexpect metadata contains
information about the expected target of the intrinsic so that we can
check against the correct PGO counter when emitting diagnostics, and the
compiler's values for the LikelyBranchWeight and UnlikelyBranchWeight.
We use these branch weight values to determine when to emit the
diagnostic to the user.
A future patch should address the comment at the top of
LowerExpectIntrisic.cpp to hoist the LikelyBranchWeight and
UnlikelyBranchWeight values into a shared space that can be accessed
outside of the LowerExpectIntrinsic pass. Once that is done, the
misexpect metadata can be updated to be smaller.
In the long term, it is possible to reconstruct portions of the
misexpect metadata from the existing profile data. However, we have
avoided this to keep the code simple, and because some kind of metadata
tag will be required to identify which branch/switch/select instructions
are influenced by the use of llvm.expect
Patch By: paulkirth
Differential Revision: https://reviews.llvm.org/D66324
llvm-svn: 371584
Summary:
This option is true by default in sanitizer common. The default
false value was added a while ago without any reasoning in
524e934112
so, presumably it's safe to remove for consistency.
Reviewers: hctim, samsonov, morehouse, kcc, vitalybuka
Reviewed By: hctim, samsonov, vitalybuka
Subscribers: delcypher, #sanitizers, llvm-commits, kcc
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D67193
llvm-svn: 371442
I verified that the test is red without the interceptors.
rdar://40334350
Reviewed By: kubamracek, vitalybuka
Differential Revision: https://reviews.llvm.org/D66616
llvm-svn: 371439
Most builtins accepting or returning long double use the fp_t typedef.
Change the remaining few cases to do so.
Differential Revision: https://reviews.llvm.org/D35034
llvm-svn: 371400
Use attribute flag `POSIX_SPAWN_CLOEXEC_DEFAULT` in the call to
`posix_spawn`.
If this flag is set, then only file descriptors explicitly described by
the file_actions argument are available in the spawned process; all of
the other file descriptors are automatically closed in the spawned
process.
POSIX_SPAWN_CLOEXEC_DEFAULT is an Apple-specific extension.
llvm-svn: 370121
Summary:
@eugenis to approve addition of //compiler-rt/tools.
@pree-jackie please confirm that this WFY.
D66494 introduced the GWP-ASan stack_trace_compressor_fuzzer. Building fuzz
targets in compiler-rt is a new affair, and has some challenges:
- If the host compiler doesn't have compiler-rt, the -fsanitize=fuzzer may not
be able to link against `libclang_rt.fuzzer*`.
- Things in compiler-rt generally aren't built when you want to build with
sanitizers using `-DLLVM_USE_SANITIZER`. This tricky to work around, so
we create the new tools directory so that we can build fuzz targets with
sanitizers. This has the added bonus of fixing the problem above as well, as
we can now just guard the fuzz target build to only be done with
`-DLLVM_USE_SANITIZE_COVERAGE=On`.
Reviewers: eugenis, pree-jackie
Reviewed By: eugenis, pree-jackie
Subscribers: dberris, mgorny, #sanitizers, llvm-commits, eugenis, pree-jackie, lebedev.ri, vitalybuka, morehouse
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D66776
llvm-svn: 370094
I D65322 I added a check for BrokenPipeError. However, python 2.7 doesn't
have BrokenPipeError. To be python 2.7 and 3 compatible we need to catch
IOError instead and check for errno == errno.EPIPE.
llvm-svn: 370025
- Unless explicit configuration, using FreeBSD super pages feature for shadow mapping.
- asan only for now.
Reviewers: dim, emaste, vitalybuka
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D65851
llvm-svn: 370008
Summary:
Currently, llvm-symbolizer will print -1 when presented with -1 and not
print a second line. In that case we will block for ever trying to read
the file name. This also happens for non-existent files, in which case GNU
addr2line exits immediate, but llvm-symbolizer does not (see
https://llvm.org/PR42754). While touching these lines, I also added some
more debug logging to help diagnose this and potential future issues.
Reviewers: kcc, eugenis, glider, samsonov
Reviewed By: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D65322
llvm-svn: 369924
This is a fixup for r369823 which introduced the use of
VERSION_GREATER_EQUAL in the cmake config for gwp_asan.
Minimum supported version of cmake in LLVM is 3.4.3 and
VERSION_GREATER_EQUAL was not introduced until later
versions of cmake.
llvm-svn: 369891
Vitaly Buka