Commit Graph

13153 Commits

Author SHA1 Message Date
Matt Morehouse 10670bdf54 Revert "[fuzzer] Create user provided fuzzer writeable directories when requested if they dont exist"
This reverts commit cb8912799d, since the
test fails on Windows.
2020-09-01 12:05:46 -07:00
Anatoly Trosinenko 93eed63d2f [builtins] Make __div[sdt]f3 handle denormal results
This patch introduces denormal result support to soft-float division
implementation unified by D85031.

Reviewed By: sepavloff

Differential Revision: https://reviews.llvm.org/D85032
2020-09-01 21:52:34 +03:00
Matt Morehouse cb8912799d [fuzzer] Create user provided fuzzer writeable directories when requested if they dont exist
Currently, libFuzzer will exit with an error message if a non-existent
directory is provided for any of the appropriate arguments. For cases
where libFuzzer is used in a specialized embedded environment, it would
be much easier to have libFuzzer create the directories for the user.

This patch accommodates for this scenario by allowing the user to provide
the argument `-create_missing_dirs=1` which makes libFuzzer attempt to
create the `artifact_prefix`, `exact_artifact_path`,
`features_dir` and/or corpus directory if they don't already exist rather
than throw an error and exit.

Split off from D84808 as requested [here](https://reviews.llvm.org/D84808#2208546).

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86733
2020-09-01 11:50:47 -07:00
mhl 2665425908 [libfuzzer] Reduce default verbosity when printing large mutation sequences
When using a custom mutator (e.g. thrift mutator, similar to LPM)
that calls back into libfuzzer's mutations via `LLVMFuzzerMutate`, the mutation
sequences needed to achieve new coverage can get prohibitively large.

Printing these large sequences has two downsides:

1) It makes the logs hard to understand for a human.
2) The performance cost slows down fuzzing.

In this patch I change the `PrintMutationSequence` function to take a max
number of entries, to achieve this goal. I also update `PrintStatusForNewUnit`
to default to printing only 10 entries, in the default verbosity level (1),
requiring the user to set verbosity to 2 if they want the full mutation
sequence.

For our use case, turning off verbosity is not an option, as that would also
disable `PrintStats()` which is very useful for infrastructure that analyzes
the logs in realtime. I imagine most users of libfuzzer always want those logs
in the default.

I built a fuzzer locally with this patch applied to libfuzzer.

When running with the default verbosity, I see logs like this:

    #65 NEW    cov: 4799 ft: 10443 corp: 41/1447Kb lim: 64000 exec/s: 1 rss: 575Mb L: 28658/62542 MS: 196 Custom-CrossOver-ChangeBit-EraseBytes-ChangeBit-ChangeBit-ChangeBit-CrossOver-ChangeBit-CrossOver- DE: "\xff\xff\xff\x0e"-"\xfe\xff\xff\x7f"-"\xfe\xff\xff\x7f"-"\x17\x00\x00\x00\x00\x00\x00\x00"-"\x00\x00\x00\xf9"-"\xff\xff\xff\xff"-"\xfa\xff\xff\xff"-"\xf7\xff\xff\xff"-"@\xff\xff\xff\xff\xff\xff\xff"-"E\x00"-
    #67 NEW    cov: 4810 ft: 10462 corp: 42/1486Kb lim: 64000 exec/s: 1 rss: 577Mb L: 39823/62542 MS: 135 Custom-CopyPart-ShuffleBytes-ShuffleBytes-ChangeBit-ChangeBinInt-EraseBytes-ChangeBit-ChangeBinInt-ChangeBit- DE: "\x01\x00\x00\x00\x00\x00\x01\xf1"-"\x00\x00\x00\x07"-"\x00\x0d"-"\xfd\xff\xff\xff"-"\xfe\xff\xff\xf4"-"\xe3\xff\xff\xff"-"\xff\xff\xff\xf1"-"\xea\xff\xff\xff"-"\x00\x00\x00\xfd"-"\x01\x00\x00\x05"-

Staring hard at the logs it's clear that the cap of 10 is applied.

When running with verbosity level 2, the logs look like the below:

    #66    NEW    cov: 4700 ft: 10188 corp: 37/1186Kb lim: 64000 exec/s: 2 rss: 509Mb L: 47616/61231 MS: 520 Custom-CopyPart-ChangeBinInt-ChangeBit-ChangeByte-EraseBytes-PersAutoDict-CopyPart-ShuffleBytes-ChangeBit-ShuffleBytes-CopyPart-EraseBytes-CopyPart-ChangeBinInt-CopyPart-ChangeByte-ShuffleBytes-ChangeBinInt-ShuffleBytes-ChangeBit-CMP-ShuffleBytes-ChangeBit-CrossOver-ChangeBinInt-ChangeByte-ShuffleBytes-CrossOver-EraseBytes-ChangeBinInt-InsertRepeatedBytes-PersAutoDict-InsertRepeatedBytes-InsertRepeatedBytes-CrossOver-ChangeByte-ShuffleBytes-CopyPart-ShuffleBytes-CopyPart-CrossOver-ChangeBit-ShuffleBytes-CrossOver-PersAutoDict-ChangeByte-ChangeBit-ShuffleBytes-CrossOver-ChangeByte-EraseBytes-CopyPart-ChangeBinInt-PersAutoDict-CrossOver-ShuffleBytes-CrossOver-CrossOver-EraseBytes-CrossOver-EraseBytes-CrossOver-ChangeBit-ChangeBinInt-ChangeByte-EraseBytes-ShuffleBytes-ShuffleBytes-ChangeBit-EraseBytes-ChangeBinInt-ChangeBit-ChangeBinInt-CopyPart-EraseBytes-PersAutoDict-EraseBytes-CopyPart-ChangeBinInt-ChangeByte-CrossOver-ChangeBinInt-ShuffleBytes-PersAutoDict-PersAutoDict-ChangeBinInt-CopyPart-ChangeBinInt-CrossOver-ChangeBit-ChangeBinInt-CopyPart-ChangeByte-ChangeBit-CopyPart-CrossOver-ChangeByte-ChangeBit-ChangeByte-ShuffleBytes-CMP-ChangeBit-CopyPart-ChangeBit-ChangeByte-ChangeBinInt-PersAutoDict-ChangeBinInt-CrossOver-ChangeBinInt-ChangeBit-ChangeBinInt-ChangeBinInt-PersAutoDict-ChangeBinInt-ChangeBinInt-ChangeByte-CopyPart-ShuffleBytes-ChangeByte-ChangeBit-ChangeByte-ChangeByte-EraseBytes-CrossOver-ChangeByte-ChangeByte-EraseBytes-EraseBytes-InsertRepeatedBytes-ShuffleBytes-CopyPart-CopyPart-ChangeBit-ShuffleBytes-PersAutoDict-ShuffleBytes-ChangeBit-ChangeByte-ChangeBit-ShuffleBytes-ChangeByte-ChangeBinInt-CrossOver-ChangeBinInt-ChangeBit-EraseBytes-CopyPart-ChangeByte-CrossOver-EraseBytes-CrossOver-ChangeByte-ShuffleBytes-ChangeByte-ChangeBinInt-CrossOver-ChangeByte-InsertRepeatedBytes-InsertByte-ShuffleBytes-PersAutoDict-ChangeBit-ChangeByte-ChangeBit-ShuffleBytes-ShuffleBytes-CopyPart-ShuffleBytes-EraseBytes-ShuffleBytes-ShuffleBytes-CrossOver-ChangeBinInt-CopyPart-CopyPart-CopyPart-EraseBytes-EraseBytes-ChangeByte-ChangeBinInt-ShuffleBytes-CMP-InsertByte-EraseBytes-ShuffleBytes-CopyPart-ChangeBit-CrossOver-CopyPart-CopyPart-ShuffleBytes-ChangeByte-ChangeByte-ChangeBinInt-EraseBytes-ChangeByte-ChangeBinInt-ChangeBit-ChangeBit-ChangeByte-ShuffleBytes-PersAutoDict-PersAutoDict-CMP-ChangeBit-ShuffleBytes-PersAutoDict-ChangeBinInt-EraseBytes-EraseBytes-ShuffleBytes-ChangeByte-ShuffleBytes-ChangeBit-EraseBytes-CMP-ShuffleBytes-ChangeByte-ChangeBinInt-EraseBytes-ChangeBinInt-ChangeByte-EraseBytes-ChangeByte-CrossOver-ShuffleBytes-EraseBytes-EraseBytes-ShuffleBytes-ChangeBit-EraseBytes-CopyPart-ShuffleBytes-ShuffleBytes-CrossOver-CopyPart-ChangeBinInt-ShuffleBytes-CrossOver-InsertByte-InsertByte-ChangeBinInt-ChangeBinInt-CopyPart-EraseBytes-ShuffleBytes-ChangeBit-ChangeBit-EraseBytes-ChangeByte-ChangeByte-ChangeBinInt-CrossOver-ChangeBinInt-ChangeBinInt-ShuffleBytes-ShuffleBytes-ChangeByte-ChangeByte-ChangeBinInt-ShuffleBytes-CrossOver-EraseBytes-CopyPart-CopyPart-CopyPart-ChangeBit-ShuffleBytes-ChangeByte-EraseBytes-ChangeByte-InsertRepeatedBytes-InsertByte-InsertRepeatedBytes-PersAutoDict-EraseBytes-ShuffleBytes-ChangeByte-ShuffleBytes-ChangeBinInt-ShuffleBytes-ChangeBinInt-ChangeBit-CrossOver-CrossOver-ShuffleBytes-CrossOver-CopyPart-CrossOver-CrossOver-CopyPart-ChangeByte-ChangeByte-CrossOver-ChangeBit-ChangeBinInt-EraseBytes-ShuffleBytes-EraseBytes-CMP-PersAutoDict-PersAutoDict-InsertByte-ChangeBit-ChangeByte-CopyPart-CrossOver-ChangeByte-ChangeBit-ChangeByte-CopyPart-ChangeBinInt-EraseBytes-CrossOver-ChangeBit-CrossOver-PersAutoDict-CrossOver-ChangeByte-CrossOver-ChangeByte-ChangeByte-CrossOver-ShuffleBytes-CopyPart-CopyPart-ShuffleBytes-ChangeByte-ChangeByte-ChangeBinInt-ChangeBinInt-ChangeBinInt-ChangeBinInt-ShuffleBytes-CrossOver-ChangeBinInt-ShuffleBytes-ChangeBit-PersAutoDict-ChangeBinInt-ShuffleBytes-ChangeBinInt-ChangeByte-CrossOver-ChangeBit-CopyPart-ChangeBit-ChangeBit-CopyPart-ChangeByte-PersAutoDict-ChangeBit-ShuffleBytes-ChangeByte-ChangeBit-CrossOver-ChangeByte-CrossOver-ChangeByte-CrossOver-ChangeBit-ChangeByte-ChangeBinInt-PersAutoDict-CopyPart-ChangeBinInt-ChangeBit-CrossOver-ChangeBit-PersAutoDict-ShuffleBytes-EraseBytes-CrossOver-ChangeByte-ChangeBinInt-ShuffleBytes-ChangeBinInt-InsertRepeatedBytes-PersAutoDict-CrossOver-ChangeByte-Custom-PersAutoDict-CopyPart-CopyPart-ChangeBinInt-ShuffleBytes-ChangeBinInt-ChangeBit-ShuffleBytes-CrossOver-CMP-ChangeByte-CopyPart-ShuffleBytes-CopyPart-CopyPart-CrossOver-CrossOver-CrossOver-ShuffleBytes-ChangeByte-ChangeBinInt-ChangeBit-ChangeBit-ChangeBit-ChangeByte-EraseBytes-ChangeByte-ChangeBit-ChangeByte-ChangeByte-CopyPart-PersAutoDict-ChangeBinInt-PersAutoDict-PersAutoDict-PersAutoDict-CopyPart-CopyPart-CrossOver-ChangeByte-ChangeBinInt-ShuffleBytes-ChangeBit-CopyPart-EraseBytes-CopyPart-CopyPart-CrossOver-ChangeByte-EraseBytes-ShuffleBytes-ChangeByte-CopyPart-EraseBytes-CopyPart-CrossOver-ChangeBinInt-ChangeBinInt-InsertByte-ChangeBinInt-ChangeBit-ChangeByte-CopyPart-ChangeByte-EraseBytes-ChangeByte-ChangeBit-ChangeByte-ShuffleBytes-CopyPart-ChangeBinInt-EraseBytes-CrossOver-ChangeBit-ChangeBit-CrossOver-EraseBytes-ChangeBinInt-CopyPart-CopyPart-ChangeBinInt-ChangeBit-EraseBytes-InsertRepeatedBytes-EraseBytes-ChangeBit-CrossOver-CrossOver-EraseBytes-EraseBytes-ChangeByte-CopyPart-CopyPart-ShuffleBytes-ChangeByte-ChangeBit-ChangeByte-EraseBytes-ChangeBit-ChangeByte-ChangeByte-CrossOver-CopyPart-EraseBytes-ChangeByte-EraseBytes-ChangeByte-ShuffleBytes-ShuffleBytes-ChangeByte-CopyPart-ChangeByte-ChangeByte-ChangeBit-CopyPart-ChangeBit-ChangeBinInt-CopyPart-ShuffleBytes-ChangeBit-ChangeBinInt-ChangeBit-EraseBytes-CMP-CrossOver-CopyPart-ChangeBinInt-CrossOver-CrossOver-CopyPart-CrossOver-CrossOver-InsertByte-InsertByte-CopyPart-Custom- DE: "warn"-"\x00\x00\x00\x80"-"\xfe\xff\xff\xfb"-"\xff\xff"-"\x10\x00\x00\x00"-"\xfe\xff\xff\xff"-"\xff\xff\xff\xf6"-"U\x01\x00\x00\x00\x00\x00\x00"-"\xd9\xff\xff\xff"-"\xfe\xff\xff\xea"-"\xf0\xff\xff\xff"-"\xfc\xff\xff\xff"-"warn"-"\xff\xff\xff\xff"-"\xfe\xff\xff\xfb"-"\x00\x00\x00\x80"-"\xfe\xff\xff\xf1"-"\xfe\xff\xff\xea"-"\x00\x00\x00\x00\x00\x00\x012"-"\xe2\x00"-"\xfb\xff\xff\xff"-"\x00\x00\x00\x00"-"\xe9\xff\xff\xff"-"\xff\xff"-"\x00\x00\x00\x80"-"\x01\x00\x04\xc9"-"\xf0\xff\xff\xff"-"\xf9\xff\xff\xff"-"\xff\xff\xff\xff\xff\xff\xff\x12"-"\xe2\x00"-"\xfe\xff\xff\xff"-"\xfe\xff\xff\xea"-"\xff\xff\xff\xff"-"\xf4\xff\xff\xff"-"\xe9\xff\xff\xff"-"\xf1\xff\xff\xff"-
    #48    NEW    cov: 4502 ft: 9151 corp: 27/750Kb lim: 64000 exec/s: 2 rss: 458Mb L: 50772/50772 MS: 259 ChangeByte-ShuffleBytes-ChangeBinInt-ChangeByte-ChangeByte-ChangeByte-ChangeByte-ChangeBit-CopyPart-CrossOver-CopyPart-ChangeByte-CrossOver-CopyPart-ChangeBit-ChangeByte-EraseBytes-ChangeByte-CopyPart-CopyPart-CopyPart-ChangeBit-EraseBytes-ChangeBinInt-CrossOver-CopyPart-CrossOver-CopyPart-ChangeBit-ChangeByte-ChangeBit-InsertByte-CrossOver-InsertRepeatedBytes-InsertRepeatedBytes-InsertRepeatedBytes-ChangeBinInt-EraseBytes-InsertRepeatedBytes-InsertByte-ChangeBit-ShuffleBytes-ChangeBit-ChangeBit-CopyPart-ChangeBit-ChangeByte-CrossOver-ChangeBinInt-ChangeByte-CrossOver-CMP-ChangeByte-CrossOver-ChangeByte-ShuffleBytes-ShuffleBytes-ChangeByte-ChangeBinInt-CopyPart-EraseBytes-CrossOver-ChangeBit-ChangeBinInt-InsertByte-ChangeBit-CopyPart-ChangeBinInt-ChangeByte-CrossOver-ChangeBit-EraseBytes-CopyPart-ChangeBinInt-ChangeBit-ChangeBit-ChangeByte-CopyPart-ChangeBinInt-CrossOver-PersAutoDict-ChangeByte-ChangeBit-ChangeByte-ChangeBinInt-ChangeBinInt-EraseBytes-CopyPart-CopyPart-ChangeByte-ChangeByte-EraseBytes-PersAutoDict-CopyPart-ChangeByte-ChangeByte-EraseBytes-CrossOver-CopyPart-CopyPart-CopyPart-ChangeByte-ChangeBit-CMP-CopyPart-ChangeBinInt-ChangeBinInt-CrossOver-ChangeBit-ChangeBit-EraseBytes-ChangeByte-ShuffleBytes-ChangeBit-ChangeBinInt-CMP-InsertRepeatedBytes-CopyPart-Custom-ChangeByte-CrossOver-EraseBytes-ChangeBit-CopyPart-CrossOver-CMP-ShuffleBytes-EraseBytes-CrossOver-PersAutoDict-ChangeByte-CrossOver-CopyPart-CrossOver-CrossOver-ShuffleBytes-ChangeBinInt-CrossOver-ChangeBinInt-ShuffleBytes-PersAutoDict-ChangeByte-EraseBytes-ChangeBit-CrossOver-EraseBytes-CrossOver-ChangeBit-ChangeBinInt-EraseBytes-InsertByte-InsertRepeatedBytes-InsertByte-InsertByte-ChangeByte-ChangeBinInt-ChangeBit-CrossOver-ChangeByte-CrossOver-EraseBytes-ChangeByte-ShuffleBytes-ChangeBit-ChangeBit-ShuffleBytes-CopyPart-ChangeByte-PersAutoDict-ChangeBit-ChangeByte-InsertRepeatedBytes-CMP-CrossOver-ChangeByte-EraseBytes-ShuffleBytes-CrossOver-ShuffleBytes-ChangeBinInt-ChangeBinInt-CopyPart-PersAutoDict-ShuffleBytes-ChangeBit-CopyPart-ShuffleBytes-CopyPart-EraseBytes-ChangeByte-ChangeBit-ChangeBit-ChangeBinInt-ChangeByte-CopyPart-EraseBytes-ChangeBinInt-EraseBytes-EraseBytes-PersAutoDict-CMP-PersAutoDict-CrossOver-CrossOver-ChangeBit-CrossOver-PersAutoDict-CrossOver-CopyPart-ChangeByte-EraseBytes-ChangeByte-ShuffleBytes-ChangeByte-ChangeByte-CrossOver-ChangeBit-EraseBytes-ChangeByte-EraseBytes-ChangeBinInt-CrossOver-CrossOver-EraseBytes-ChangeBinInt-CrossOver-ChangeBit-ShuffleBytes-ChangeBit-ChangeByte-EraseBytes-ChangeBit-CrossOver-CrossOver-CrossOver-ChangeByte-ChangeBit-ShuffleBytes-ChangeBit-ChangeBit-EraseBytes-CrossOver-CrossOver-CopyPart-ShuffleBytes-ChangeByte-ChangeByte-CopyPart-CrossOver-CopyPart-CrossOver-CrossOver-EraseBytes-EraseBytes-ShuffleBytes-InsertRepeatedBytes-ChangeBit-CopyPart-Custom- DE: "\xfe\xff\xff\xfc"-"\x00\x00\x00\x00"-"F\x00"-"\xf3\xff\xff\xff"-"St9exception"-"_\x00\x00\x00"-"\xf6\xff\xff\xff"-"\xfe\xff\xff\xff"-"\x00\x00\x00\x00"-"p\x02\x00\x00\x00\x00\x00\x00"-"\xfe\xff\xff\xfb"-"\xff\xff"-"\xff\xff\xff\xff"-"\x01\x00\x00\x07"-"\xfe\xff\xff\xfe"-

These are prohibitively large and of limited value in the default case (when
someone is running the fuzzer, not debugging it), in my opinion.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86658
2020-09-01 11:14:36 -07:00
Alex Richardson 7be8682921 [tsan] Fix "failed to intercept sysctlnametomib" on FreeBSD
The sysctlnametomib function is called from __tsan::Initialize via
__sanitizer::internal_sysctlbyname (see stack trace below). This results
in a fatal error since sysctlnametomib has not been intercepted yet.
This patch allows internal_sysctlbyname to be called before
__tsan::Initialize() has completed. On FreeBSD >= 1300045 sysctlbyname()
is a real syscall, but for older versions it calls sysctlnametomib()
followed by sysctl(). To avoid calling the intercepted version, look up
the real sysctlnametomib() followed by internal_sysctl() if the
syscall is not available.

This reduces check-sanitizer failures from 62 to 11 for me.

==34433==FATAL: ThreadSanitizer: failed to intercept sysctlnametomib
    at /exports/users/alr48/sources/upstream-llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_termination.cpp:51
    name=0x7fffffffce10, namelenp=0x7fffffffce08)
    at /exports/users/alr48/sources/upstream-llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:7908
    oldp=0x7fffffffcf2c, oldlenp=0x7fffffffcf20, newp=0x0, newlen=0)
    at /exports/users/alr48/sources/upstream-llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp:803
    at /exports/users/alr48/sources/upstream-llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp:2152
    at /exports/users/alr48/sources/upstream-llvm-project/compiler-rt/lib/tsan/rtl/tsan_rtl.cpp:367
    fname=0x21c731 "readlink", pc=34366042556)
    at /exports/users/alr48/sources/upstream-llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:255
    bufsiz=1024)
    at /exports/users/alr48/sources/upstream-llvm-project/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:7151

Reviewed By: #sanitizers, vitalybuka

Differential Revision: https://reviews.llvm.org/D85292
2020-09-01 18:25:22 +01:00
Anatoly Trosinenko 0e90d8d4fe [builtins] Unify the softfloat division implementation
This patch replaces three different pre-existing implementations of
__div[sdt]f3 LibCalls with a generic one - like it is already done for
many other LibCalls.

Reviewed By: sepavloff

Differential Revision: https://reviews.llvm.org/D85031
2020-09-01 19:05:50 +03:00
Vitaly Buka bd6ee51150 [NFC][Asan] Format includes 2020-09-01 05:30:07 -07:00
Petr Hosek 3c7bfbd683 [CMake] Use find_library for ncurses
Currently it is hard to avoid having LLVM link to the system install of
ncurses, since it uses check_library_exists to find e.g. libtinfo and
not find_library or find_package.

With this change the ncurses lib is found with find_library, which also
considers CMAKE_PREFIX_PATH. This solves an issue for the spack package
manager, where we want to use the zlib installed by spack, and spack
provides the CMAKE_PREFIX_PATH for it.

This is a similar change as https://reviews.llvm.org/D79219, which just
landed in master.

Patch By: haampie

Differential Revision: https://reviews.llvm.org/D85820
2020-08-31 20:06:21 -07:00
Anatoly Trosinenko 11cf6346fd [NFC][compiler-rt] Factor out __div[sdt]i3 and __mod[dt]i3 implementations
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D86400
2020-08-30 16:14:08 +03:00
Marco Vanotti 6760f7ee6f [compiler-rt][tsan] Remove unnecesary typedefs
These typedefs are not used anywhere else in this compilation unit.

Differential Revision: https://reviews.llvm.org/D86826
2020-08-28 18:43:54 -07:00
Vitaly Buka 6cf72ad3b3 [NFC][asan] Don't unwind stack before pool check 2020-08-28 02:00:35 -07:00
JF Bastien 82d29b397b Add an unsigned shift base sanitizer
It's not undefined behavior for an unsigned left shift to overflow (i.e. to
shift bits out), but it has been the source of bugs and exploits in certain
codebases in the past. As we do in other parts of UBSan, this patch adds a
dynamic checker which acts beyond UBSan and checks other sources of errors. The
option is enabled as part of -fsanitize=integer.

The flag is named: -fsanitize=unsigned-shift-base
This matches shift-base and shift-exponent flags.

<rdar://problem/46129047>

Differential Revision: https://reviews.llvm.org/D86000
2020-08-27 19:50:10 -07:00
Justin Cady 1d3ef5f122 [MSAN] Add fiber switching APIs
Add functions exposed via the MSAN interface to enable MSAN within
binaries that perform manual stack switching (e.g. through using fibers
or coroutines).

This functionality is analogous to the fiber APIs available for ASAN and TSAN.

Fixes google/sanitizers#1232

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D86471
2020-08-27 19:30:40 -07:00
Harmen Stoppels cdcb9ab10e Revert "Use find_library for ncurses"
The introduction of find_library for ncurses caused more issues than it solved problems. The current open issue is it makes the static build of LLVM fail. It is better to revert for now, and get back to it later.

Revert "[CMake] Fix an issue where get_system_libname creates an empty regex capture on windows"
This reverts commit 1ed1e16ab8.

Revert "Fix msan build"
This reverts commit 34fe9613dd.

Revert "[CMake] Always mark terminfo as unavailable on Windows"
This reverts commit 76bf26236f.

Revert "[CMake] Fix OCaml build failure because of absolute path in system libs"
This reverts commit 8e4acb82f7.

Revert "[CMake] Don't look for terminfo libs when LLVM_ENABLE_TERMINFO=OFF"
This reverts commit 495f91fd33.

Revert "Use find_library for ncurses"
This reverts commit a52173a3e5.

Differential revision: https://reviews.llvm.org/D86521
2020-08-27 17:57:26 -07:00
Dokyung Song c10e63677f Recommit "[libFuzzer] Fix arguments of InsertPartOf/CopyPartOf calls in CrossOver mutator."
The CrossOver mutator is meant to cross over two given buffers (referred to as
the first/second buffer henceforth). Previously InsertPartOf/CopyPartOf calls
used in the CrossOver mutator incorrectly inserted/copied part of the second
buffer into a "scratch buffer" (MutateInPlaceHere of the size
CurrentMaxMutationLen), rather than the first buffer. This is not intended
behavior, because the scratch buffer does not always (i) contain the content of
the first buffer, and (ii) have the same size as the first buffer;
CurrentMaxMutationLen is typically a lot larger than the size of the first
buffer. This patch fixes the issue by using the first buffer instead of the
scratch buffer in InsertPartOf/CopyPartOf calls.

A FuzzBench experiment was run to make sure that this change does not
inadvertently degrade the performance. The performance is largely the same; more
details can be found at:
https://storage.googleapis.com/fuzzer-test-suite-public/fixcrossover-report/index.html

This patch also adds two new tests, namely "cross_over_insert" and
"cross_over_copy", which specifically target InsertPartOf and CopyPartOf,
respectively.

- cross_over_insert.test checks if the fuzzer can use InsertPartOf to trigger
  the crash.

- cross_over_copy.test checks if the fuzzer can use CopyPartOf to trigger the
  crash.

These newly added tests were designed to pass with the current patch, but not
without the it (with 790878f291 these tests do not
pass). To achieve this, -max_len was intentionally given a high value. Without
this patch, InsertPartOf/CopyPartOf will generate larger inputs, possibly with
unpredictable data in it, thereby failing to trigger the crash.

The test pass condition for these new tests is narrowed down by (i) limiting
mutation depth to 1 (i.e., a single CrossOver mutation should be able to trigger
the crash) and (ii) checking whether the mutation sequence of "CrossOver-" leads
to the crash.

Also note that these newly added tests and an existing test (cross_over.test)
all use "-reduce_inputs=0" flags to prevent reducing inputs; it's easier to
force the fuzzer to keep original input string this way than tweaking
cov-instrumented basic blocks in the source code of the fuzzer executable.

Differential Revision: https://reviews.llvm.org/D85554
2020-08-27 21:48:45 +00:00
Dokyung Song 52f1df0923 Recommit "[libFuzzer] Fix value-profile-load test."
value-profile-load.test needs adjustment with a mutator change in
bb54bcf849, which reverted as of now, but will be
recommitted after landing this patch.

This patch makes value-profile-load.test more friendly to (and aware of) the
current value profiling strategy, which is based on the hamming as well as the
absolute distance. To this end, this patch adjusts the set of input values that
trigger an expected crash. More specifically, this patch now uses a single value
0x01effffe as a crashing input, because this value is close to values like
{0x1ffffff, 0xffffff, ...}, which are very likely to be added to the corpus per
the current hamming- and absolute-distance-based value profiling strategy. Note
that previously the crashing input values were {1234567 * {1, 2, ...}, s.t. <
INT_MAX}.

Every byte in the chosen value 0x01effeef is intentionally different; this was
to make it harder to find the value without the intermediate inputs added to the
corpus by the value profiling strategy.

Also note that LoadTest.cpp now uses a narrower condition (Size != 8) for
initial pruning of inputs, effectively preventing libFuzzer from generating
inputs longer than necessary and spending time on mutating such long inputs in
the corpus - a functionality not meant to be tested by this specific test.

Differential Revision: https://reviews.llvm.org/D86247
2020-08-27 19:12:30 +00:00
Raul Tambre 45344cf7ac [CMake][compiler-rt][libunwind] Compile assembly files as ASM not C, unify workarounds
It isn't very wise to pass an assembly file to the compiler and tell it to compile as a C file and hope that the compiler recognizes it as assembly instead.
Simply don't mark the file as C and CMake will recognize the rest.

This was attempted earlier in https://reviews.llvm.org/D85706, but reverted due to architecture issues on Apple.
Subsequent digging revealed a similar change was done earlier for libunwind in https://reviews.llvm.org/rGb780df052dd2b246a760d00e00f7de9ebdab9d09.
Afterwards workarounds were added for MinGW and Apple:
* https://reviews.llvm.org/rGb780df052dd2b246a760d00e00f7de9ebdab9d09
* https://reviews.llvm.org/rGd4ded05ba851304b26a437896bc3962ef56f62cb

The workarounds in libunwind and compiler-rt are unified and comments added pointing to each other.
The workaround is updated to only be used for MinGW for CMake versions before 3.17, which fixed the issue (https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4287).

Additionally fixed Clang not being passed as the assembly compiler for compiler-rt runtime build.

Example error:
[525/634] Building C object lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
FAILED: lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
/opt/tooling/drive/host/bin/clang --target=aarch64-linux-gnu -I/opt/tooling/drive/llvm/compiler-rt/lib/tsan/.. -isystem /opt/tooling/drive/toolchain/opt/drive/toolchain/include -x c -Wall -Wno-unused-parameter -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fPIE -fno-rtti -Wframe-larger-than=530 -Wglobal-constructors --sysroot=. -MD -MT lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -MF lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o.d -o lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -c /opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S
/opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S:29:1: error: expected identifier or '('
.section .text
^
1 error generated.

Differential Revision: https://reviews.llvm.org/D86308
2020-08-27 15:40:15 +03:00
Anatoly Trosinenko fce035eae9 [NFC][compiler-rt] Factor out __mulo[sdt]i4 implementations to .inc file
The existing implementations are almost identical except for width of the
integer type.

Factor them out to int_mulo_impl.inc for better maintainability.

This patch is almost identical to D86277.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D86289
2020-08-27 14:33:48 +03:00
Anatoly Trosinenko 182d14db07 [NFC][compiler-rt] Factor out __mulv[sdt]i3 implementations to .inc file
The existing implementations are almost identical except for width of the
integer type.

Factor them out to int_mulv_impl.inc for better maintainability.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D86277
2020-08-27 14:33:48 +03:00
Vedant Kumar 1f47f89a90 [profile] Add InstrProfilingVersionVar.c.o to Darwin kext builtins
Fixes a build failure in the Darwin kernel. Tested with:

% nm -mU lib/libclang_rt.cc_kext_x86_64h_osx.a | grep __llvm_profile_raw_version

rdar://67809173
2020-08-26 10:02:13 -07:00
Matt Morehouse 2392ff093a [libFuzzer] Error and exit if user supplied fuzzer writeable directories don't exist
Currently, libFuzzer will exit with an error message if a non-existent
corpus directory is provided. However, if a user provides a non-existent
directory for the `artifact_prefix`, `exact_artifact_path`, or
`features_dir`, libFuzzer will continue execution but silently fail to
write artifacts/features.

To improve the user experience, this PR adds validation for the existence of
all user supplied directories before executing the main fuzzing loop. If they
don't exist, libFuzzer will exit with an error message.

Patch By: dgg5503

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D84808
2020-08-26 09:27:07 -07:00
Hans Wennborg 8421503300 Bump -len_control value in fuzzer-custommutator.test (PR47286)
to make the test more stable, as suggested by mmoroz.
2020-08-26 16:45:51 +02:00
Kuba Mracek e713b0ecbc [tsan] On arm64e, strip out ptrauth bits from incoming PCs
Differential Revision: https://reviews.llvm.org/D86378
2020-08-25 11:59:36 -07:00
David Tenty f8454d60b8 [AIX][compiler-rt][builtins] Don't add ppc builtin implementations that require __int128 on AIX
since __int128 currently isn't supported on AIX.

Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D85972
2020-08-25 11:35:38 -04:00
Kostya Kortchinsky bd5ca4f0ed [scudo][standalone] Skip irrelevant regions during release
With the 'new' way of releasing on 32-bit, we iterate through all the
regions in between `First` and `Last`, which covers regions that do not
belong to the class size we are working with. This is effectively wasted
cycles.

With this change, we add a `SkipRegion` lambda to `releaseFreeMemoryToOS`
that will allow the release function to know when to skip a region.
For the 64-bit primary, since we are only working with 1 region, we never
skip.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D86399
2020-08-25 07:41:02 -07:00
Anatoly Trosinenko b9f49d13fd [compiler-rt][builtins] Add more test cases for __div[sdt]f3 LibCalls
* Make the three tests look more uniformly
* Explicitly specify types of integer and floating point literals
* Add more test cases (mostly inspired by divtf3_test.c)
  - tests are added for obviously special cases such as +/-Inf, +/-0.0 and some
    more implementation-specific cases such as divisor being almost 1.0
* Make NaN in the second test case of `divtf3` to be `sNaN` instead of
  testing for `qNaN` again

Reviewed By: sepavloff

Differential Revision: https://reviews.llvm.org/D84932
2020-08-25 16:19:23 +03:00
Ilya Leoshkevich 151f603199 [libFuzzer] Un-XFAIL msan.test on SystemZ
After https://reviews.llvm.org/D86382 it works.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86184
2020-08-25 15:01:55 +02:00
Alex Richardson 39d2506461 Fix crypt.cpp sanitizer test on FreeBSD
FreeBSD doesn't provide a crypt.h header but instead defines the functions
in unistd.h. Use __has_include() to handle that case.

Reviewed By: #sanitizers, vitalybuka

Differential Revision: https://reviews.llvm.org/D85406
2020-08-25 12:20:33 +01:00
Alex Richardson 5695fa9190 [asan] Also allow for SIGBUS in high-address-dereference.c
FreeBSD delivers a SIGBUS signal for bad addresses rather than SIGSEGV.

Reviewed By: #sanitizers, vitalybuka, yln

Differential Revision: https://reviews.llvm.org/D85409
2020-08-25 12:20:33 +01:00
Rainer Orth e3585ff7af [compiler-rt][asan][test] Set LD_LIBRARY_PATH_{32,64} on Solaris
The dynamically linked ASan tests rely on `LD_LIBRARY_PATH` to find
`libclang_rt.asan-*.so` at runtime.

However, the Solaris runtime linker `ld.so.1` also supports more specific
variables: `LD_LIBRARY_PATH_32` and `LD_LIBRARY_PATH_64` respectively.  If
those happen to be set, `LD_LIBRARY_PATH` is ignored.  In such a case, all
dynamically linked ASan tests `FAIL`.  For i386 alone, this affects about
200 tests.

The following patch fixes that by also setting `LD_LIBRARY_PATH_{32,64}` on
Solaris.

Tested on `amd64-pc-solaris2.11` both with only `LD_LIBRARY_PATH` set and
with `LD_LIBRARY_PATH_{32,64}` set too.

Differential Revision: https://reviews.llvm.org/D86333
2020-08-25 09:36:51 +02:00
Mateusz Mikuła 879c1db5d2 [Compiler-RT] Fix profiler building with MinGW GCC
Differential Revision: https://reviews.llvm.org/D86405
2020-08-25 10:22:53 +03:00
Freddy Ye e02d081f2b [X86] Support -march=sapphirerapids
Support -march=sapphirerapids for x86.
Compare with Icelake Server, it includes 14 more new features. They are
amxtile, amxint8, amxbf16, avx512bf16, avx512vp2intersect, cldemote,
enqcmd, movdir64b, movdiri, ptwrite, serialize, shstk, tsxldtrk, waitpkg.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D86503
2020-08-25 14:21:21 +08:00
Julian Lettner c1d25e9a82 [TSan][libdispatch] Add interceptors for dispatch_barrier_async_and_wait(), pt. 2
Also intercept these target functions on Linux.
2020-08-24 14:37:34 -07:00
Julian Lettner fa0a855c73 [TSan][libdispatch] Add interceptors for dispatch_barrier_async_and_wait()
Add interceptors for the barrier variants of the "async and wait"
dispatch API family.
2020-08-24 14:01:04 -07:00
Shoaib Meenai 2c80e2fe51 [runtimes] Use llvm-libtool-darwin for runtimes build
It's full featured now and we can use it for the runtimes build instead
of relying on an external libtool, which means the CMAKE_HOST_APPLE
restriction serves no purpose either now. Restrict llvm-lipo to Darwin
targets while I'm here, since it's only needed there.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D86367
2020-08-24 13:48:30 -07:00
Shoaib Meenai 26c1d689ae [compiler-rt] Disable ranlib when using libtool
libtool already produces a table of contents, and ranlib just gives
spurious errors because it doesn't understand universal binaries.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D86365
2020-08-24 13:48:28 -07:00
Julian Lettner 2b7a2cbb15 [TSan][Darwin] Handle NULL argument in interceptor
Handle NULL address argument in the `mach_vm_[de]allocate()`
interceptors and fix test: `Assignment 2` is not valid if we weren't
able to re-allocate memory.

rdar://67680613
2020-08-24 11:36:02 -07:00
Dokyung Song 9659b81b2a [libFuzzer] Make msan.test:SimpleCmpTest succeed with less trials.
Currently SimpleCmpTest passes after 9,831,994 trials on x86_64/Linux
when the number of given trials is 10,000,000, just a little bigger than
that. This patch modifies SimpleCmpTest.cpp so that the test passes with less
trials, reducing its chances of future failures as libFuzzer evolves. More
specifically, this patch changes a 32-bit equality check to a 8-bit equality
check, making this test pass at 4,635,303 trials.

Differential Revision: https://reviews.llvm.org/D86382
2020-08-24 14:45:39 +00:00
Julian Lettner cc62373915 [ASan][Darwin] Enable test on additional platforms 2020-08-21 11:13:09 -07:00
Azharuddin Mohammed 8831e34771 Revert "[libFuzzer] Fix arguments of InsertPartOf/CopyPartOf calls in CrossOver mutator."
This reverts commit bb54bcf849.

It is causing the value-profile-load.test test to fail on macOS.
2020-08-21 09:58:50 -07:00
Azharuddin Mohammed 7ed3286a51 [LibFuzzer] [tests] [Darwin] Use the pthread library from the SDK 2020-08-21 09:22:13 -07:00
Luís Marques 57903cf093 [compiler-rt][RISCV] Use muldi3 builtin assembly implementation
D80465 added an assembly implementation of muldi3 for RISC-V but it didn't
add it to the cmake `*_SOURCES` list, so the C implementation was being used
instead. This patch fixes that.

Differential Revision: https://reviews.llvm.org/D86036
2020-08-21 13:06:35 +01:00
Craig Topper df9a9bb7be [X86] Correct the implementation of the testFeature macro in getIntelProcessorTypeAndSubtype to do a proper bit test.
Instead of ANDing with a one hot mask representing the bit to
be tested, we were ANDing with just the bit number. This tests
multiple bits none of them the correct one.

This caused skylake-avx512, cascadelake and cooperlake to all
be misdetected. Based on experiments with the Intel SDE, it seems
that all of these CPUs are being detected as being cooperlake.
This is bad since its the newest CPU of the 3.
2020-08-20 23:50:45 -07:00
Vitaly Buka 66c882e529 Revert "[libFuzzer] Fix value-profile-load test."
D86247 fails on Windows.

This reverts commit 428bebaf10.
2020-08-20 17:49:12 -07:00
Vitaly Buka ea9bf460a8 [NFC][libFuzzer] Try to fix test on Windows
Broken after D86247
2020-08-20 17:33:51 -07:00
Julian Lettner 53aff8d864 [Darwin][iOS] Enable test on non-macOS platforms
We are now using a properly-substituted minimal deployment target
compiler flag (`%min_macos_deployment_target=10.11`).  Enable test on
iOS and watchOS plus simulators.  We are also not testing on very old
platforms anymore, so we can remove some obsolete lit infrastructure.
2020-08-20 16:40:32 -07:00
Julian Lettner 1f3c92f968 [compiler-rt][Darwin] Refactor minimum deployment target substitutions
* Support macOS 11+ version scheme
* Standardize substitution name `%min_deployment_target=x.y`
* Remove unneeded error cases (the input version is hard-coded)
* Specify version as tuple instead of string; no need to parse it

These changes should also facilitate a future addition of a substitution
that expands to "set deployment target to current target version"
(https://reviews.llvm.org/D70151).

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D85925
2020-08-20 16:22:56 -07:00
Julian Lettner d9b062ad87 [TSan][Darwin] Remove unnecessary lit substitution
We don't test on very old versions of Apple platforms anymore.  The
following lit substitution concerning the minimum deployment target for
ARC support can be removed.

```
%darwin_min_target_with_full_runtime_arc_support -> 10.11
```

Differential Revision: https://reviews.llvm.org/D85803
2020-08-20 13:00:32 -07:00
Julian Lettner 6222a28db5 [TSan][Darwin] Enable test on non-macOS platforms
After removing the unnecessary `-mmacosx-version-min=10.12` compiler
flag this test can run on all platforms.  I confirmed that this test is
green for iOS, iOS simulator, and watchOS simulator.

Differential Revision: https://reviews.llvm.org/D85952
2020-08-20 12:28:05 -07:00
Raphael Isemann adf0b8cc70 Revert "[compiler-rt] Compile assembly files as ASM not C"
This reverts commit d58fd4e521. This broke
compiler-rt compilation on macOS:

codesign --sign - /Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/clang/12.0.99/lib/darwin/libclang_rt.tsan_ios_dynamic.dylib
ld: warning: ignoring file projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_ios_dynamic.dir/rtl/tsan_rtl_amd64.S.o, building for iOS-arm64 but attempting to link with file built for iOS Simulator-x86_64
ld: warning: ignoring file projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_ios_dynamic.dir/rtl/tsan_rtl_aarch64.S.o, building for iOS-arm64 but attempting to link with file built for iOS Simulator-x86_64
Undefined symbols for architecture arm64:
  "_wrap__setjmp", referenced from:
      substitution__setjmp in tsan_interceptors_posix.cpp.o
  "_wrap_setjmp", referenced from:
      substitution_setjmp in tsan_interceptors_posix.cpp.o
  "_wrap_sigsetjmp", referenced from:
      substitution_sigsetjmp in tsan_interceptors_posix.cpp.o
ld: symbol(s) not found for architecture arm64
2020-08-20 15:25:22 +02:00
Petr Hosek d58fd4e521 [compiler-rt] Compile assembly files as ASM not C
It isn't very wise to pass an assembly file to the compiler and tell it to compile as a C file and hope that the compiler recognizes it as assembly instead.
Instead enable the ASM language and mark the files as being ASM.

[525/634] Building C object lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
FAILED: lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
/opt/tooling/drive/host/bin/clang --target=aarch64-linux-gnu -I/opt/tooling/drive/llvm/compiler-rt/lib/tsan/.. -isystem /opt/tooling/drive/toolchain/opt/drive/toolchain/include -x c -Wall -Wno-unused-parameter -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fPIE -fno-rtti -Wframe-larger-than=530 -Wglobal-constructors --sysroot=. -MD -MT lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -MF lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o.d -o lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -c /opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S
/opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S:29:1: error: expected identifier or '('
.section .text
^
1 error generated.

Fixed Clang not being passed as the assembly compiler for compiler-rt runtime build.

Patch By: tambre

Differential Revision: https://reviews.llvm.org/D85706
2020-08-20 00:34:59 -07:00
Dokyung Song 428bebaf10 [libFuzzer] Fix value-profile-load test.
The behavior of the CrossOver mutator has changed with
bb54bcf849. This seems to affect the
value-profile-load test on Darwin. This patch provides a wider margin for
determining success of the value-profile-load test, by testing the targeted
functionality (i.e., GEP index value profile) more directly and faster. To this
end, LoadTest.cpp now uses a narrower condition (Size != 8) for initial pruning
of inputs, effectively preventing libFuzzer from generating inputs longer than
necessary and spending time on mutating such long inputs in the corpus - a
functionality not meant to be tested by this specific test.

Previously, on x86/Linux, it required 6,597,751 execs with -use_value_profile=1
and 19,605,575 execs with -use_value_profile=0 to hit the crash. With this
patch, the test passes with 174,493 execs, providing a wider margin from the
given trials of 10,000,000. Note that, without the value profile (i.e.,
-use_value_profile=0), the test wouldn't pass as it still requires 19,605,575
execs to hit the crash.

Differential Revision: https://reviews.llvm.org/D86247
2020-08-19 22:14:43 +00:00
Matt Morehouse 4deda57106 [DFSan] Handle mmap() calls before interceptors are installed.
InitializeInterceptors() calls dlsym(), which calls calloc().  Depending
on the allocator implementation, calloc() may invoke mmap(), which
results in a segfault since REAL(mmap) is still being resolved.

We fix this by doing a direct syscall if interceptors haven't been fully
resolved yet.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D86168
2020-08-19 15:07:41 -07:00
Peter Collingbourne a208ad5ddb sanitizer_common: Use void* for madvise first argument on Solaris.
Differential Revision: https://reviews.llvm.org/D86166
2020-08-19 10:55:55 -07:00
Petr Hosek 495f91fd33 [CMake] Don't look for terminfo libs when LLVM_ENABLE_TERMINFO=OFF
D85820 introduced a bug where LLVM_ENABLE_TERMINFO was set to true when
the library was found, even when the user had set
-DLLVM_ENABLE_TERMINFO=OFF.

Patch By: haampie

Differential Revision: https://reviews.llvm.org/D86173
2020-08-19 10:31:42 -07:00
Julian Lettner 40ae296bc3 [TSan][libdispatch] Guard test execution on old platforms
`dispatch_async_and_wait()` was introduced in macOS 10.14.  Let's
forward declare it to ensure we can compile the test with older SDKs and
guard execution by checking if the symbol is available.  (We can't use
`__builtin_available()`, because that itself requires a higher minimum
deployment target.)  We also need to specify the `-undefined
dynamic_lookup` compiler flag.

Differential Revision: https://reviews.llvm.org/D85995
2020-08-18 18:34:14 -07:00
Julian Lettner 686fe293e6 [TSan][libdispatch] Ensure TSan dylib works on old systems
`dispatch_async_and_wait()` was introduced in macOS 10.14, which is
greater than our minimal deployment target.  We need to forward declare
it as a "weak import" to ensure we generate a weak reference so the TSan
dylib continues to work on older systems.  We cannot simply `#include
<dispatch.h>` or use the Darwin availability macros since this file is
multi-platform.

In addition, we want to prevent building these interceptors at all when
building with older SDKs because linking always fails.

Before:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA  __interpose      0x000F5E68    pointer      0 libSystem        _dispatch_async_and_wait_f
```

After:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA  __got            0x000EC0A8    pointer      0 libSystem        _dispatch_async_and_wait (weak import)
__DATA  __interpose      0x000F5E78    pointer      0 libSystem        _dispatch_async_and_wait (weak import)
```

This is a follow-up to D85854 and should fix:
https://reviews.llvm.org/D85854#2221529

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D86103
2020-08-18 18:34:14 -07:00
Julian Lettner 0c4863a253 Reland "[TSan][libdispatch] Add interceptors for dispatch_async_and_wait()"
The linker errors caused by this revision have been addressed.

Add interceptors for `dispatch_async_and_wait[_f]()` which was added in
macOS 10.14.  This pair of functions is similar to `dispatch_sync()`,
but does not force a context switch of the queue onto the caller thread
when the queue is active (and hence is more efficient).  For TSan, we
can apply the same semantics as for `dispatch_sync()`.

From the header docs:
> Differences with dispatch_sync()
>
> When the runtime has brought up a thread to invoke the asynchronous
> workitems already submitted to the specified queue, that servicing
> thread will also be used to execute synchronous work submitted to the
> queue with dispatch_async_and_wait().
>
> However, if the runtime has not brought up a thread to service the
> specified queue (because it has no workitems enqueued, or only
> synchronous workitems), then dispatch_async_and_wait() will invoke the
> workitem on the calling thread, similar to the behaviour of functions
> in the dispatch_sync family.

Additional context:
> The guidance is to use `dispatch_async_and_wait()` instead of
> `dispatch_sync()` when it is necessary to mix async and sync calls on
> the same queue. `dispatch_async_and_wait()` does not guarantee
> execution on the caller thread which allows to reduce context switches
> when the target queue is active.
> https://gist.github.com/tclementdev/6af616354912b0347cdf6db159c37057

rdar://35757961

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D85854
2020-08-18 18:34:14 -07:00
Arthur Eubanks 501a078cbb Revert "[TSan][libdispatch] Add interceptors for dispatch_async_and_wait()"
This reverts commit d137db8029.

Breaks builds on older SDKs.
2020-08-18 09:49:05 -07:00
Dokyung Song bb54bcf849 [libFuzzer] Fix arguments of InsertPartOf/CopyPartOf calls in CrossOver mutator.
The CrossOver mutator is meant to cross over two given buffers (referred to as
the first/second buffer henceforth). Previously InsertPartOf/CopyPartOf calls
used in the CrossOver mutator incorrectly inserted/copied part of the second
buffer into a "scratch buffer" (MutateInPlaceHere of the size
CurrentMaxMutationLen), rather than the first buffer. This is not intended
behavior, because the scratch buffer does not always (i) contain the content of
the first buffer, and (ii) have the same size as the first buffer;
CurrentMaxMutationLen is typically a lot larger than the size of the first
buffer. This patch fixes the issue by using the first buffer instead of the
scratch buffer in InsertPartOf/CopyPartOf calls.

A FuzzBench experiment was run to make sure that this change does not
inadvertently degrade the performance. The performance is largely the same; more
details can be found at:
https://storage.googleapis.com/fuzzer-test-suite-public/fixcrossover-report/index.html

This patch also adds two new tests, namely "cross_over_insert" and
"cross_over_copy", which specifically target InsertPartOf and CopyPartOf,
respectively.

- cross_over_insert.test checks if the fuzzer can use InsertPartOf to trigger
  the crash.

- cross_over_copy.test checks if the fuzzer can use CopyPartOf to trigger the
  crash.

These newly added tests were designed to pass with the current patch, but not
without the it (with 790878f291 these tests do not
pass). To achieve this, -max_len was intentionally given a high value. Without
this patch, InsertPartOf/CopyPartOf will generate larger inputs, possibly with
unpredictable data in it, thereby failing to trigger the crash.

The test pass condition for these new tests is narrowed down by (i) limiting
mutation depth to 1 (i.e., a single CrossOver mutation should be able to trigger
the crash) and (ii) checking whether the mutation sequence of "CrossOver-" leads
to the crash.

Also note that these newly added tests and an existing test (cross_over.test)
all use "-reduce_inputs=0" flags to prevent reducing inputs; it's easier to
force the fuzzer to keep original input string this way than tweaking
cov-instrumented basic blocks in the source code of the fuzzer executable.

Differential Revision: https://reviews.llvm.org/D85554
2020-08-18 16:09:18 +00:00
Rainer Orth 13080ca1f0 [compiler-rt][test] XFAIL two tests on 32-bit sparc
Two tests `FAIL` on 32-bit sparc:

  Profile-sparc :: Posix/instrprof-gcov-parallel.test
  UBSan-Standalone-sparc :: TestCases/Float/cast-overflow.cpp

The failure mode is similar:

  Undefined                       first referenced
   symbol                             in file
  __atomic_store_4                    /var/tmp/instrprof-gcov-parallel-6afe8d.o
  __atomic_load_4                     /var/tmp/instrprof-gcov-parallel-6afe8d.o

  Undefined                       first referenced
   symbol                             in file
  __atomic_load_1                     /var/tmp/cast-overflow-72a808.o

This is a known bug: `clang` doesn't inline atomics on 32-bit sparc, unlike
`gcc`.

The patch therefore `XFAIL`s the tests.

Tested on `sparcv9-sun-solaris2.11` and `amd64-pc-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D85346
2020-08-18 11:32:51 +02:00
Harmen Stoppels a52173a3e5 Use find_library for ncurses
Currently it is hard to avoid having LLVM link to the system install of
ncurses, since it uses check_library_exists to find e.g. libtinfo and
not find_library or find_package.

With this change the ncurses lib is found with find_library, which also
considers CMAKE_PREFIX_PATH. This solves an issue for the spack package
manager, where we want to use the zlib installed by spack, and spack
provides the CMAKE_PREFIX_PATH for it.

This is a similar change as https://reviews.llvm.org/D79219, which just
landed in master.

Differential revision: https://reviews.llvm.org/D85820
2020-08-17 19:52:52 -07:00
Dávid Bolvanský 0f14b2e6cb Revert "[BPI] Improve static heuristics for integer comparisons"
This reverts commit 50c743fa71. Patch will be split to smaller ones.
2020-08-17 20:44:33 +02:00
Matt Morehouse 69721fc9d1 [DFSan] Support fast16labels mode in dfsan_union.
While the instrumentation never calls dfsan_union in fast16labels mode,
the custom wrappers do.  We detect fast16labels mode by checking whether
any labels have been created.  If not, we must be using fast16labels
mode.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D86012
2020-08-17 11:27:28 -07:00
Amy Huang 51c152ca2a Revert "Make compiler-rt/asan tests run with llvm-lit."
This reverts commit 7f84f62ef0.

Seems to be causing a bunch of compiler-rt test failures on
ppc64-linux bots.
2020-08-17 10:00:45 -07:00
Amy Huang 7f84f62ef0 Make compiler-rt/asan tests run with llvm-lit.
This sets some config parameters so we can run the asan tests with
llvm-lit,
e.g. `./bin/llvm-lit [...]/compiler-rt/test/asan`

Differential Revision: https://reviews.llvm.org/D83821
2020-08-17 09:24:19 -07:00
Dimitry Andric 3aecf4bdf3 On FreeBSD, add -pthread to ASan dynamic compile flags for tests
Otherwise, lots of these tests fail with a CHECK error similar to:

==12345==AddressSanitizer CHECK failed: compiler-rt/lib/asan/asan_posix.cpp:120 "((0)) == ((pthread_key_create(&tsd_key, destructor)))" (0x0, 0x4e)

This is because the default pthread stubs in FreeBSD's libc always
return failures (such as ENOSYS for pthread_key_create) in case the
pthread library is not linked in.

Reviewed By: arichardson

Differential Revision: https://reviews.llvm.org/D85082
2020-08-15 13:05:31 +02:00
Gui Andrade 97de0188dd [MSAN] Reintroduce libatomic load/store instrumentation
Have the front-end use the `nounwind` attribute on atomic libcalls.
This prevents us from seeing `invoke __atomic_load` in MSAN, which
is problematic as it has no successor for instrumentation to be added.
2020-08-14 20:31:10 +00:00
Matt Morehouse bb3a3da38d [DFSan] Don't unmap during dfsan_flush().
Unmapping and remapping is dangerous since another thread could touch
the shadow memory while it is unmapped.  But there is really no need to
unmap anyway, since mmap(MAP_FIXED) will happily clobber the existing
mapping with zeroes.  This is thread-safe since the mmap() is done under
the same kernel lock as page faults are done.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D85947
2020-08-14 11:43:49 -07:00
Julian Lettner d137db8029 [TSan][libdispatch] Add interceptors for dispatch_async_and_wait()
Add interceptors for `dispatch_async_and_wait[_f]()` which was added in
macOS 10.14.  This pair of functions is similar to `dispatch_sync()`,
but does not force a context switch of the queue onto the caller thread
when the queue is active (and hence is more efficient).  For TSan, we
can apply the same semantics as for `dispatch_sync()`.

From the header docs:
> Differences with dispatch_sync()
>
> When the runtime has brought up a thread to invoke the asynchronous
> workitems already submitted to the specified queue, that servicing
> thread will also be used to execute synchronous work submitted to the
> queue with dispatch_async_and_wait().
>
> However, if the runtime has not brought up a thread to service the
> specified queue (because it has no workitems enqueued, or only
> synchronous workitems), then dispatch_async_and_wait() will invoke the
> workitem on the calling thread, similar to the behaviour of functions
> in the dispatch_sync family.

Additional context:
> The guidance is to use `dispatch_async_and_wait()` instead of
> `dispatch_sync()` when it is necessary to mix async and sync calls on
> the same queue. `dispatch_async_and_wait()` does not guarantee
> execution on the caller thread which allows to reduce context switches
> when the target queue is active.
> https://gist.github.com/tclementdev/6af616354912b0347cdf6db159c37057

rdar://35757961

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D85854
2020-08-14 09:39:57 -07:00
Matt Morehouse c1f9c1c13c [DFSan] Fix parameters to strtoull wrapper.
base and nptr_label were swapped, which meant we were passing nptr's
shadow as the base to the operation.  Usually, the shadow is 0, which
causes strtoull to guess the correct base from the string prefix (e.g.,
0x means base-16 and 0 means base-8), hiding this bug.  Adjust the test
case to expose the bug.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D85935
2020-08-14 08:02:30 -07:00
Chris Kennelly 800f0eda5b [NFC] Remove unused function.
Differential Revision: https://reviews.llvm.org/D85944
2020-08-13 21:20:15 -04:00
Chris Kennelly 097c8fb2cb [NFC] Remove unused variables.
These were no longer needed following
96855125e7.

Reviewed By: rupprecht

Differential Revision: https://reviews.llvm.org/D85939
2020-08-13 17:56:51 -04:00
Peter Collingbourne 9f8c4039f2 sanitizer_common: Introduce internal_madvise and start using it.
A recent change to sanitizer_common caused us to issue the syscall
madvise(MADV_HUGEPAGE) during HWASAN initialization. This may lead to a
problem if madvise is instrumented (e.g. because libc is instrumented
or the user intercepted it). For example, on Android the syscall may
fail if the kernel does not support transparent hugepages, which leads
to an attempt to set errno in a HWASAN instrumented function. Avoid
this problem by introducing a syscall wrapper and using it to issue
this syscall.

Tested only on Linux; includes untested updates for the other
platforms.

Differential Revision: https://reviews.llvm.org/D85870
2020-08-13 13:09:02 -07:00
Chris Kennelly 96855125e7 [tsan] Respect no_huge_pages_for_shadow.
Disable huge pages in the TSan shadow regions when no_huge_pages_for_shadow == true (default).

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D85841
2020-08-13 15:55:23 -04:00
Dávid Bolvanský 3944d3df4f [Tests] Removed debug copy command 2020-08-13 20:21:19 +02:00
Dávid Bolvanský 50c743fa71 [BPI] Improve static heuristics for integer comparisons
Similarly as for pointers, even for integers a == b is usually false.

GCC also uses this heuristic.

Reviewed By: ebrevnov

Differential Revision: https://reviews.llvm.org/D85781
2020-08-13 19:54:27 +02:00
Rainer Orth 55e472e9da [compiler-rt][asan][test] Skipt sanitizer_common tests on Sparc
When building on `sparc64-unknown-linux-gnu`, I found that a large number
of `SanitizerCommon-asan-sparc*-Linux` tests were `FAIL`ing, like

   SanitizerCommon-asan-sparc-Linux :: Linux/aligned_alloc-alignment.cpp
  [...]
   SanitizerCommon-asan-sparcv9-Linux :: Linux/aligned_alloc-alignment.cpp
  [...]

many of them due to

  fatal error: error in backend: Function "_Z14User_OnSIGSEGViP9siginfo_tPv": over-aligned dynamic alloca not supported.

which breaks ASan on Sparc.  Currently ASan is only built for the benefit
of `gcc` where it does work.  However, when enabling the compilation in
`compiler-rt` to make certain it continues to build, I missed
`compiler-rt/test/sanitizer_common` when disabling ASan testing on Sparc
(it's not yet enabled on Solaris).

This patch fixes the issue.

Tested on `sparcv9-sun-solaris2.11` with the `sanitizer_comon` testsuite enabled.

Differential Revision: https://reviews.llvm.org/D85732
2020-08-13 10:20:52 +02:00
Ilya Leoshkevich eca4b4007d [Sanitizer] Fix segfaults during unwinding on SystemZ
Every now and then SystemZ programs built with ASan crash with

    ERROR: AddressSanitizer: stack-overflow on address 0x040000000000

for no apparent reason. The problem is that
BufferedStackTrace::UnwindFast() is specialized for SystemZ: it takes
register 14 from the frame, however, IsValidFrame() is not
specialized, and does not guarantee that frame[14] is going to be a
valid memory access.

Fix by introducing per-arch kFrameSize and using it in IsValidFrame().

Reviewed By: uweigand

Differential Revision: https://reviews.llvm.org/D85822
2020-08-12 22:24:32 +02:00
Matt Morehouse fd893bda55 Fix sigaction interceptor to always correctly populate oldact
This fixes https://bugs.llvm.org/show_bug.cgi?id=47118. Before this change, when the sigaction interceptor prevented a signal from being changed, it also prevented the oldact output parameter from being written to. This resulted in a use-of-uninitialized-variable by any program that used sigaction for the purpose of reading signals.

This change fixes this: the regular sigaction implementation is still called, but with the act parameter nullified, preventing any changes.

Patch By: IanPudney

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D85797
2020-08-12 10:11:56 -07:00
Ilya Leoshkevich f5a252ed68 [SanitizerCoverage] Use zeroext for cmp parameters on all targets
Commit 9385aaa848 ("[sancov] Fix PR33732") added zeroext to
__sanitizer_cov_trace(_const)?_cmp[1248] parameters for x86_64 only,
however, it is useful on other targets, in particular, on SystemZ: it
fixes swap-cmp.test.

Therefore, use it on all targets. This is safe: if target ABI does not
require zero extension for a particular parameter, zeroext is simply
ignored. A similar change has been implemeted as part of commit
3bc439bdff ("[MSan] Add instrumentation for SystemZ"), and there were
no problems with it.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D85689
2020-08-12 18:38:12 +02:00
Petr Hosek 31e5f7120b [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-08-11 20:22:11 -07:00
Ilya Leoshkevich 9df7ee34e1 [libFuzzer] Fix minimizing timeouts
When one tries to minimize timeouts using -minimize_crash=1,
minimization immediately fails. The following sequence of events is
responsible for this:

[parent] SIGALRM occurs
[parent] read() returns -EINTR (or -ERESTARTSYS according to strace)
[parent] fgets() returns NULL
[parent] ExecuteCommand() closes child's stdout and returns
[child ] SIGALRM occurs
[child ] AlarmCallback() attempts to write "ALARM: ..." to stdout
[child ] Dies with SIGPIPE without calling DumpCurrentUnit()
[parent] Does not see -exact_artifact_path and exits

When minimizing, the timer in parent is not necessary, so fix by not
setting it in this case.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D85359
2020-08-11 22:16:12 +02:00
Julian Lettner bfb0b853b2 [compiler-rt] XFAIL test on iOS
On iOS, when we `longjmp()` out of the signal handler, a subsequent call
to `sigaltstack()` still reports that we are executing on the signal
handler stack.
Tracking rdar://66789814

Differential Revision: https://reviews.llvm.org/D85677
2020-08-11 11:58:40 -07:00
Rainer Orth dc7630dd44 [compiler-rt][builtins] Un-xfail two tests on sparcv9
Two tests currently `XPASS` on sparcv9:

  Unexpectedly Passed Tests (2):
    Builtins-sparcv9-sunos :: compiler_rt_logbl_test.c
    Builtins-sparcv9-sunos :: divtc3_test.c

The following patch fixes this.

Tested on `sparcv9-sun-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D85119
2020-08-11 15:47:17 +02:00
Rainer Orth 8144a7d8fc [compiler-rt][ubsan][test] Fix TypeCheck/misaligned.cpp on Sparc
Two ubsan tests FAIL on Sparc:

  UBSan-Standalone-sparc :: TestCases/TypeCheck/misaligned.cpp
  UBSan-Standalone-sparcv9 :: TestCases/TypeCheck/misaligned.cpp

I've reported the details in Bug 47015, but it boils down to the fact that
the `s1` subtest actually incurs a fault on strict-alignment targets like
Sparc which UBSan doesn't expect.

This can be fixed like the `w1` subtest by compiling with
`-fno-sanitize-recover=alignment`.

Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D85433
2020-08-11 12:46:34 +02:00
Gui Andrade c0b5000bd8 [MSAN RT] Use __sanitizer::mem_is_zero in __msan_test_shadow
The former function is particularly optimized for exactly the
use case we're interested in: an all-zero buffer.

This reduces the overhead of calling this function some 80% or
more. This is particularly for instrumenting code heavy with
string processing functions, like grep. An invocation of grep
with the pattern '[aeiou]k[aeiou]' has its runtime reduced by
~75% with this patch

Differential Revision: https://reviews.llvm.org/D84961
2020-08-10 19:22:27 +00:00
Petr Hosek a4d78d23c5 Revert "[CMake] Simplify CMake handling for zlib"
This reverts commit ccbc1485b5 which
is still failing on the Windows MLIR bots.
2020-08-08 17:08:23 -07:00
Petr Hosek ccbc1485b5 [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-08-08 16:44:08 -07:00
Teresa Johnson bb1456decf [sanitizer] Fix comment (NFC)
As pointed out in D85387, part of the comment for MapDynamicShadow
refactored to sanitizer_common in D83247 was incorrect for non-Linux
versions. Update the comment to reflect that.
2020-08-07 16:18:18 -07:00
Gui Andrade 17ff170e3a Revert "[MSAN] Instrument libatomic load/store calls"
Problems with instrumenting atomic_load when the call has no successor,
blocking compiler roll

This reverts commit 33d239513c.
2020-08-07 19:45:51 +00:00
Oliver Stannard be8b3f0c22 [AArch64] Disable waitid.cpp test for AArch64
This test is failing intermittently on the AArch64 build bots, disable
it for now to keep the bots green while we investigate it.
2020-08-07 10:42:15 +01:00
Christian Kühnel f3cc4df51d Revert "[CMake] Simplify CMake handling for zlib"
This reverts commit 1adc494bce.
This patch broke the Windows compilation on buildbot and pre-merge testing:
http://lab.llvm.org:8011/builders/mlir-windows/builds/5945
https://buildkite.com/llvm-project/llvm-master-build/builds/780
2020-08-07 09:36:49 +02:00
Sameer Sahasrabuddhe c530539bad [AArch64][NFC] require aarch64 support for hwasan test
This was breaking builds where the target is not enabled.

Reviewed By: danielkiss, eugenis

Differential Revision: https://reviews.llvm.org/D85412
2020-08-07 09:24:52 +05:30
Evgenii Stepanov aa57cabae2 [msan] Support %ms in scanf.
Differential Revision: https://reviews.llvm.org/D85350
2020-08-06 13:54:43 -07:00
Alex Richardson 6148cca708 [compiler-rt] Fix build of Sanitizer-Test_Nolibc after D80648
Running ninja check-sanitizer fails for after that patch (commit
058f5f6fd8) with the following error:

libRTSanitizerCommon.test.nolibc.x86_64.a(sanitizer_posix.cpp.o): In
function `__sanitizer::GetNamedMappingFd(char const*, unsigned long,
int*)':
..../llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix.cpp:358:
undefined reference to `fcntl'
clang-12: error: linker command failed with exit code 1 (use -v to see
invocation)

This patch works around the problem by only calling fcntl if O_CLOEXEC
is not defined.

Reviewed By: plopresti

Differential Revision: https://reviews.llvm.org/D85114
2020-08-06 09:16:52 +01:00
Alex Richardson 8803ebcf3b Fix qsort() interceptor for FreeBSD
When the FreeBSD qsort() implementation recurses, it does so using an
interposable function call, so we end up calling the interceptor again
and set the saved comparator to wrapped_qsort_compar. This results in an
infinite loop and a eventually a stack overflow since wrapped_qsort_compar
ends up calling itself. This means that ASAN is completely broken on
FreeBSD for programs that call qsort(). I found this while running
check-all on a FreeBSD system a ASAN-instrumented LLVM.

Fix this by checking whether we are recursing inside qsort before writing
to qsort_compar. The same bug exists in the qsort_r interceptor, so use the
same approach there. I did not test the latter since the qsort_r function
signature does not match and therefore it's not intercepted on FreeBSD/macOS.

Fixes https://llvm.org/PR46832

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D84509
2020-08-06 09:15:56 +01:00
Petr Hosek 1adc494bce [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-08-05 16:07:11 -07:00
Matt Morehouse b0c50ef759 Revert "Add libFuzzer shared object build output"
This reverts commit 98d91aecb2 since it
breaks on platforms without libstdc++.
2020-08-05 12:11:24 -07:00
Matt Morehouse 2918727588 [libFuzzer] Do not link pthreads on Android.
Android has pthreads included in bionic and doesn't recognize -lpthread.
2020-08-05 11:17:10 -07:00
Julian Lettner 1e90bd7f84 [ASan][Darwin] Adapt test for macOS 11+ version scheme
This test depends on the versioning scheme of OSX.
2020-08-05 10:14:57 -07:00
Matt Morehouse 98d91aecb2 Add libFuzzer shared object build output
This change adds a CMake rule to produce shared object versions of
libFuzzer (no-main). Like the static library versions, these shared
libraries have a copy of libc++ statically linked in. For i386 we don't
link with libc++ since i386 does not support mixing position-
independent and non-position-independent code in the same library.

Patch By: IanPudney

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D84947
2020-08-05 09:03:22 -07:00
Hans Wennborg 3ab01550b6 Revert "[CMake] Simplify CMake handling for zlib"
This quietly disabled use of zlib on Windows even when building with
-DLLVM_ENABLE_ZLIB=FORCE_ON.

> Rather than handling zlib handling manually, use find_package from CMake
> to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
> HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
> set to YES, which requires the distributor to explicitly select whether
> zlib is enabled or not. This simplifies the CMake handling and usage in
> the rest of the tooling.
>
> This is a reland of abb0075 with all followup changes and fixes that
> should address issues that were reported in PR44780.
>
> Differential Revision: https://reviews.llvm.org/D79219

This reverts commit 10b1b4a231 and follow-ups
64d99cc6ab and
f9fec0447e.
2020-08-05 12:31:44 +02:00
Pierre Gousseau 14948a08f3 [compiler-rt] Normalize some in/out doxygen parameter in interface
headers. NFC.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D84945
2020-08-05 10:17:25 +01:00
Ilya Leoshkevich ea9b82da41 [libFuzzer] Enable for SystemZ
* Add SystemZ to the list of supported architectures.

* XFAIL a few tests.

Coverage reporting is broken, and is not easy to fix (see comment in
coverage.test). Interaction with sanitizers needs to be investigated
more thoroughly, since they appear to reduce coverage in certain cases.
2020-08-04 21:53:27 +02:00
Ilya Leoshkevich a4e537d9c4 [libFuzzer] Fix endianness issue in ForEachNonZeroByte()
The usage pattern of Bundle variable assumes the machine is little
endian, which is not the case on SystemZ. Fix by converting Bundle to
little-endian when necessary.
2020-08-04 21:53:27 +02:00
Fangrui Song e4441fc653 sanitizer_symbolizer_libcdep.cpp: Change --inlining=true to --inlines and --inlining=false to --no-inlines 2020-08-04 10:24:59 -07:00
Julian Lettner 1d7790604c [UBSan] Increase robustness of tests
These UBSan tests assert the absence of runtime errors via `count 0`,
which means "expect no output".  This fails the test unnecessarily in
some environments (e.g., iOS simulator in our case).  Alter the test to
be a bit more specific and "expect no error" instead of "expect no
output".

rdar://65503408

Differential Revision: https://reviews.llvm.org/D85155
2020-08-03 15:26:58 -07:00
Mitch Phillips 9a05fa10bd [HWASan] [GlobalISel] Add +tagged-globals backend feature for GlobalISel
GlobalISel is the default ISel for aarch64 at -O0. Prior to D78465, GlobalISel
didn't have support for dealing with address-of-global lowerings, so it fell
back to SelectionDAGISel.

HWASan Globals require special handling, as they contain the pointer tag in the
top 16-bits, and are thus outside the code model. We need to generate a `movk`
in the instruction sequence with a G3 relocation to ensure the bits are
relocated properly. This is implemented in SelectionDAGISel, this patch does
the same for GlobalISel.

GlobalISel and SelectionDAGISel differ in their lowering sequence, so there are
differences in the final instruction sequence, explained in
`tagged-globals.ll`. Both of these implementations are correct, but GlobalISel
is slightly larger code size / slightly slower (by a couple of arithmetic
instructions). I don't see this as a problem for now as GlobalISel is only on
by default at `-O0`.

Reviewed By: aemerson, arsenm

Differential Revision: https://reviews.llvm.org/D82615
2020-08-03 14:28:44 -07:00
Hiroshi Yamauchi f78f509c75 [PGO] Extend the value profile buckets for mem op sizes.
Extend the memop value profile buckets to be more flexible (could accommodate a
mix of individual values and ranges) and to cover more value ranges (from 11 to
22 buckets).

Disabled behind a flag (to be enabled separately) and the existing code to be
removed later.

Differential Revision: https://reviews.llvm.org/D81682
2020-08-03 11:04:32 -07:00
Rainer Orth 39494d9c21 [compiler-rt][profile] Fix various InstrProf tests on Solaris
Currently, several InstrProf tests `FAIL` on Solaris (both sparc and x86):

  Profile-i386 :: Posix/instrprof-visibility.cpp
  Profile-i386 :: instrprof-merging.cpp
  Profile-i386 :: instrprof-set-file-object-merging.c
  Profile-i386 :: instrprof-set-file-object.c

On sparc there's also

  Profile-sparc :: coverage_comments.cpp

The failure mode is always the same:

  error: /var/llvm/local-amd64/projects/compiler-rt/test/profile/Profile-i386/Posix/Output/instrprof-visibility.cpp.tmp: Failed to load coverage: Malformed coverage data

The error is from `llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp`
(`loadBinaryFormat`), l.926:

  InstrProfSymtab ProfileNames;
  std::vector<SectionRef> NamesSectionRefs = *NamesSection;
  if (NamesSectionRefs.size() != 1)
    return make_error<CoverageMapError>(coveragemap_error::malformed);

where .size() is 2 instead.

Looking at the executable, I find (with `elfdump -c -N __llvm_prf_names`):

  Section Header[15]:  sh_name: __llvm_prf_names
      sh_addr:      0x8053ca5       sh_flags:   [ SHF_ALLOC ]
      sh_size:      0x86            sh_type:    [ SHT_PROGBITS ]
      sh_offset:    0x3ca5          sh_entsize: 0
      sh_link:      0               sh_info:    0
      sh_addralign: 0x1

  Section Header[31]:  sh_name: __llvm_prf_names
      sh_addr:      0x8069998       sh_flags:   [ SHF_WRITE SHF_ALLOC ]
      sh_size:      0               sh_type:    [ SHT_PROGBITS ]
      sh_offset:    0x9998          sh_entsize: 0
      sh_link:      0               sh_info:    0
      sh_addralign: 0x1

Unlike GNU `ld` (which primarily operates on section names) the Solaris
linker, following the ELF spirit, only merges input sections into an output
section if both section name and section flags match, so two separate
sections are maintained.

The read-write one comes from `lib/clang/12.0.0/lib/sunos/libclang_rt.profile-i386.a(InstrProfilingPlatformLinux.c.o)`
while the read-only one is generated by
`llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp` (`InstrProfiling::emitNameData`)
at l.1004 where `isConstant = true`.

The easiest way to avoid the mismatch is to change the definition in
`compiler-rt/lib/profile/InstrProfilingPlatformLinux.c` to `const`.

This fixes all failures observed.

Tested on `amd64-pc-solaris2.11`, `sparcv9-sun-solaris2.11`, and
`x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D85116
2020-08-03 19:56:05 +02:00
Fangrui Song 317e00dc54 [PGO] Change a `NumVSites == 0` workaround to assert
The root cause was fixed by 3d6f53018f.
The workaround added in 99ad956fda can be changed
to an assert now. (In case the fix regresses, there will be a heap-use-after-free.)
2020-08-03 10:14:03 -07:00
Alex Richardson 03affa8099 [msan] Compile the libatomic.c test with a C compiler
Otherwise we end up compiling in C++ mode and on FreeBSD
/usr/include/stdatomic.h is not compatible with C++ since it uses _Bool.

Reviewed By: guiand, eugenis, vitalybuka, emaste

Differential Revision: https://reviews.llvm.org/D84510
2020-08-03 10:51:35 +01:00
Alex Richardson 895878f456 [asan][tsan] Mark tests failing with debug checks as XFAIL
See https://llvm.org/PR46862. This does not fix the underlying issue but at
least it allows me to run check-all again without having to disable
building compiler-rt.

Reviewed By: #sanitizers, vitalybuka

Differential Revision: https://reviews.llvm.org/D84650
2020-08-03 10:51:35 +01:00
Evgenii Stepanov dc3388b020 [msan] Respect no_huge_pages_for_shadow.
Disable huge pages in the MSan shadow region when
no_huge_pages_for_shadow == true (default).

Differential Revision: https://reviews.llvm.org/D85061
2020-08-01 16:59:51 -07:00
Julian Lettner ff756f5231 [compiler-rt][Darwin] Fix linker errors for check-asan
A recent change broke `ninja check-asan` on Darwin by causing an error
during linking of ASan unit tests [1].

Move the addition of `-ObjC` compiler flag outside of the new
`if(COMPILER_RT_STANDALONE_BUILD)` block.  It doesn't add any global
flags (e.g, `${CMAKE_CXX_FLAGS}`) and the decision to add is based
solely on source paths (`${source_rpath}`).

[1] 8b2fcc42b8, https://reviews.llvm.org/D84466

Differential Revision: https://reviews.llvm.org/D85057
2020-07-31 16:43:21 -07:00
Julian Lettner 43bf902c2e [compiler-rt][Darwin] Fix GetOSMajorKernelOffset() on watchOS
`TARGET_OS_IOS` and `TARGET_OS_WATCH` are not mutually exclusive.
`SANITIZER_IOS` is defined for all embedded platforms.  So the branch
for watchOS is never taken.  We could fix this by switching the order
of the branches (but the reason for doing so is non-obvious).  Instead,
lets use the Darwin-specific `TARGET_OS_*` macros which are mutually
exclusive.
2020-07-31 11:47:09 -07:00
Keith Randall 6c75db8b4b Disable getauxval for Go
We want the Go build to not use getauxval, as we must support glibc < 2.16 platforms.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D84859
2020-07-31 10:13:03 -07:00
Nemanja Ivanovic acb3b8dce1 [Sanitizers] Fix lint failure with Python 3.6
There are some files in compiler-rt that use UTF-8 characters in some of the
comments. This causes lint failures with some versions of Python. This patch
just makes the encoding explicit in the call to open.
2020-07-31 06:20:49 -05:00
Vedant Kumar 896f797b8b [profile] Remove dependence on getpagesize from InstrProfilingBuffer.c.o
InstrProfilingBuffer.c.o is generic code that must support compilation
into freestanding projects. This gets rid of its dependence on the
_getpagesize symbol from libc, shifting it to InstrProfilingFile.c.o.

This fixes a build failure seen in a firmware project.

rdar://66249701
2020-07-30 16:22:40 -07:00
Peiyuan Song 14c1b40174 [compiler-rt] [profile] fix profile generate for mingw x86_64
Differential Revision: https://reviews.llvm.org/D84757
2020-07-30 23:37:33 +03:00
Kuba Mracek 1e8a9c3e02 [tsan] Fixup for 1260a155: Move variadic-open.cpp test into Darwin/ directory 2020-07-30 09:32:51 -07:00
Hiroshi Yamauchi 3d6f53018f [PGO] Include the mem ops into the function hash.
To avoid hash collisions when the only difference is in mem ops.
2020-07-30 09:26:20 -07:00
Kuba Mracek 1260a155c3 [tsan] Fix the open and open64 interceptors to have correct declarations (variadic functions)
Not matching the (real) variadic declaration makes the interceptor take garbage inputs on Darwin/AArch64.

Differential Revision: https://reviews.llvm.org/D84570
2020-07-30 09:00:47 -07:00
Rainer Orth bbe5099207 [compiler-rt][cmake] Simplify COMPILER_RT_HAS_VERSION_SCRIPT tests
As requested in the review, this patch removes the additional conditions in
the `COMPILER_RT_HAS_VERSION_SCRIPT` tests.

Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D84559
2020-07-30 10:27:36 +02:00
Vedant Kumar 618a0c0d3b [profile] Add InstrProfilingInternal.c.o to Darwin kext builtins
Fixes a build failure in the Darwin kernel. Tested with:

% nm -m ./lib/libclang_rt.cc_kext_x86_64_osx.a | grep lprofSetProfileDumped

rdar://66249602
2020-07-29 17:23:28 -07:00
Julian Lettner 44e7d44997 [Darwin][compiler-rt] kern.osproductversion unavailable pre macOS 10.13
Add a fallback for `sysctl kern.osproductversion` for XNU 17 (macOS
10.13) and below, which do not provide this property.

Unfortunately, this means we have to take the detour via Darwin kernel
version again (at least for the fallback).

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D84892
2020-07-29 17:18:10 -07:00
Matt Morehouse 005991a3fe [DFSan] Remove dfsan_use_fast16labels from abilist.
Its implementation was scrapped in the final fast16labels
instrumentation patch.
2020-07-29 23:18:07 +00:00
Francis Visoiu Mistrih 926c14798d Revert "build: always add support for assembly targets on Darwin"
This reverts commit d8e8e32d85.

This breaks the build on GreenDragon:
http://green.lab.llvm.org/green/job/clang-stage1-RA/13171/console
2020-07-29 15:41:04 -07:00
Alex Lorenz dc22182d1f [darwin] build and link with a separate compiler-rt builtins library
for device simulators

This change separates out the iOS/tvOS/watchOS simulator slices from the "libclang_rt.<os>.a"
fat archive, by moving them out to their own "libclang_rt.<os>sim.a" static archive.
This allows us to build and to link with an arm64 device simulator slice for the simulators running
on Apple Silicons, and to distribute it in one archive alongside the Intel simulator slices.

Differential Revision: https://reviews.llvm.org/D84564
2020-07-29 15:32:30 -07:00
Hiroshi Yamauchi ae7589e1f1 Revert "[PGO] Include the mem ops into the function hash."
This reverts commit 120e66b341.

Due to a buildbot failure.
2020-07-29 15:04:57 -07:00
Saleem Abdulrasool d8e8e32d85 build: always add support for assembly targets on Darwin
When attempting to build compiler-rt on a developer transition kit, the
build would fail due to `.S` files not being handled properly by the
Ninja generator.  Rather than conditionalising on Xcode, conditionalise
to Darwin.  Because we know that the system compiler is clang based, it
will always properly handle the pre-processing based on the extension.

Differential Revision: https://reviews.llvm.org/D84333
2020-07-29 14:01:02 -07:00
Hiroshi Yamauchi 120e66b341 [PGO] Include the mem ops into the function hash.
To avoid hash collisions when the only difference is in mem ops.

Differential Revision: https://reviews.llvm.org/D84782
2020-07-29 13:59:40 -07:00
Matt Morehouse e2d0b44a7c [DFSan] Add efficient fast16labels instrumentation mode.
Adds the -fast-16-labels flag, which enables efficient instrumentation
for DFSan when the user needs <=16 labels.  The instrumentation
eliminates most branches and most calls to __dfsan_union or
__dfsan_union_load.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D84371
2020-07-29 18:58:47 +00:00
Julian Lettner 7e946d0c82 [compiler-rt][Darwin] Disable EXC_GUARD exceptions
ASan/TSan use mmap in a way that creates “deallocation gaps” which
triggers EXC_GUARD exceptions on macOS 10.15+ (XNU 19.0+).  Let's
suppress those.
2020-07-29 11:21:25 -07:00
Kostya Serebryany 58743040ff Add more debug code for https://github.com/google/sanitizers/issues/1193 (getting desperate, not being able to reproduce it for a few months, but the users are seeing it)
mode debug code

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D84819
2020-07-29 11:13:11 -07:00
Arthur Eubanks e7196bdf81 [Scudo][CMake] Add -fno-lto to Scudo libraries
-fno-lto is in SANITIZER_COMMON_CFLAGS but not here.
Don't use SANITIZER_COMMON_CFLAGS because of performance issues.
See https://bugs.llvm.org/show_bug.cgi?id=46838.

Fixes
$ ninja TScudoCUnitTest-i386-Test
on an LLVM build with -DLLVM_ENABLE_LTO=Thin.
check-scudo now passes.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D84805
2020-07-29 10:24:06 -07:00
Stephan Bergmann 2ead4fca79 Test including rpc/xdr.h requires sunrpc
...which is set based on HAVE_RPC_XDR_H.  At least Fedora 32 does not have a
/usr/include/rpc/xdr.h, so failed this test introduced with
<https://reviews.llvm.org/D83358> "[Sanitizers] Add interceptor for
xdrrec_create".

Differential Revision: https://reviews.llvm.org/D84740
2020-07-29 08:20:20 +02:00
Christopher Ferris f14472a2e3 [scudo][standalone] Add new mallopt options.
This adds the code to support calling mallopt and converting the
options to the internal Option enum.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D84806
2020-07-28 16:28:16 -07:00
Zequan Wu b46176bbb0 Reland [Coverage] Add comment to skipped regions
Bug filled here: https://bugs.llvm.org/show_bug.cgi?id=45757.
Add comment to skipped regions so we don't track execution count for lines containing only comments.

Differential Revision: https://reviews.llvm.org/D83592
2020-07-28 13:20:57 -07:00
Ahsan Saghir 811828a01d [PowerPC] Mark allocator_oom_test.cpp unsupported on PowerPC
This patch marks compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cpp
unsupported on PowerPC 64bit-LE architecture since this test fails when run
on a machine with larger system memory.

Reviewed By: #powerpc, nemanjai

Differential Revision: https://reviews.llvm.org/D84786
2020-07-28 14:22:33 -05:00
Kostya Kortchinsky 6f00f3b56e [scudo][standalone] mallopt runtime configuration options
Summary:
Partners have requested the ability to configure more parts of Scudo
at runtime, notably the Secondary cache options (maximum number of
blocks cached, maximum size) as well as the TSD registry options
(the maximum number of TSDs in use).

This CL adds a few more Scudo specific `mallopt` parameters that are
passed down to the various subcomponents of the Combined allocator.

- `M_CACHE_COUNT_MAX`: sets the maximum number of Secondary cached items
- `M_CACHE_SIZE_MAX`: sets the maximum size of a cacheable item in the Secondary
- `M_TSDS_COUNT_MAX`: sets the maximum number of TSDs that can be used (Shared Registry only)

Regarding the TSDs maximum count, this is a one way option, only
allowing to increase the count.

In order to allow for this, I rearranged the code to have some `setOption`
member function to the relevant classes, using the `scudo::Option` class
enum to determine what is to be set.

This also fixes an issue where a static variable (`Ready`) was used in
templated functions without being set back to `false` every time.

Reviewers: pcc, eugenis, hctim, cferris

Subscribers: jfb, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D84667
2020-07-28 11:57:54 -07:00
Julian Lettner 12f27fc4b5 [Darwin] Cleanup code via improved GetMacosAlignedVersion()
Checking the OS version via `GetMacosAlignedVersion()` now works in
simulators [1].  Let's use it to simplify `DyldNeedsEnvVariable()`.

[1] 3fb0de8207

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D81197
2020-07-28 09:48:34 -07:00
Julian Lettner 3fb0de8207 [Darwin] Fix OS version checks inside simulators
compiler-rt checks OS versions by querying the Darwin kernel version.
This is not necessarily correct inside the simulators if the simulator
runtime is not aligned with the host macOS.  Let's instead check the
`SIMULATOR_RUNTIME_VERSION` env var.

rdar://63031937

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D83977
2020-07-28 09:27:48 -07:00
Arthur Eubanks 8b2fcc42b8 [CompilerRT] Don't pass global compile test flags in non-standalone build
In a build with -DLLVM_ENABLE_LTO=Thin:

$ ninja TSanitizer-x86_64-Test-Nolibc
[1/1] Generating Sanitizer-x86_64-Test-Nolibc
FAILED: projects/compiler-rt/lib/sanitizer_common/tests/Sanitizer-x86_64-Test-Nolibc
sanitizer_nolibc_test_main.x86_64.o: file not recognized: file format not recognized

because -flto=thin is getting passed to the clang_compile step.

For non-standalone builds, global compilation flags shouldn't be passed to compiler-rt tests, only the flags the test specifies.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D84466
2020-07-28 09:26:14 -07:00
Rainer Orth 443e734fb9 [compiler-rt][cmake] Don't pass --version-script to Illumos ld
Neither the Illumos `ld` nor the Solaris 11.3 one support the `--version-script` and
`z gnu-linker-script-compat` options, which breaks the `compiler-rt` build.

This patch checks for both options instead of hardcoding their use.

Tested on `amd-pc-solaris2.11` (all of Solaris 11.4, 11.3, and Illumos).

Differential Revision: https://reviews.llvm.org/D84559
2020-07-28 13:21:36 +02:00
Hans Wennborg 7e8d5a90f2 Avoid use of std::make_unique in compiler-rt/lib/scudo/standalone/tests/combined_test.cpp
make_unique is a C++14 feature, and this prevents us from building on
Ubuntu Trusty. While we do use a C++14 compatible toolchain for building
in general, we fall back to the system toolchain for building the
compiler-rt tests.

The reason is that those tests get cross-compiled for e.g. 32-bit and
64-bit x86, and while the toolchain provides libstdc++ in those
flavours, the resulting compiler-rt test binaries don't get RPATH set
and so won't start if they're linked with that toolchain.

We've tried linking the test binaries against libstdc++ statically, by
passing COMPILER_RT_TEST_COMPILER_CFLAGS=-static-libstdc++. That mostly
works, but some test targets append -lstdc++ to the compiler invocation.

So, after spending way too much time on this, let's just avoid C++14
here for now.
2020-07-28 12:33:51 +02:00
Sergej Jaskiewicz a566e61620 Reland "[compiler-rt] [test] Allow expanding lit substitutions recursively"
The commit 8372d50508 has been reverted
(eafeb8af34) because it broke asan
tests on green dragon buildbots.

The underlying issue has been fixed in 4dd5c2bee3.
2020-07-27 23:43:50 +03:00
Matt Morehouse 34ddf0b2b0 Replace fuzzer::FuzzerDriver's INTERFACE marking with new LLVMRunFuzzerDriver.
This adds a new extern "C" function that serves the same purpose. This removes the need for external users to depend on internal headers in order to use this feature. It also standardizes the interface in a way that other fuzzing engines will be able to match.

Patch By: IanPudney

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D84561
2020-07-27 18:38:04 +00:00
Dokyung Song b52b2e1c18 Recommit "[libFuzzer] Disable implicit builtin knowledge about memcmp-like functions when -fsanitize=fuzzer-no-link is given."
Summary: This patch disables implicit builtin knowledge about memcmp-like functions when compiling the program for fuzzing, i.e., when -fsanitize=fuzzer(-no-link) is given. This allows libFuzzer to always intercept memcmp-like functions as it effectively disables optimizing calls to such functions into different forms. This is done by adding a set of flags (-fno-builtin-memcmp and others) in the clang driver. Individual -fno-builtin-* flags previously used in several libFuzzer tests are now removed, as it is now done automatically in the clang driver.

The patch was once reverted in 8ef9e2bf35, as this patch was dependent on a reverted commit f78d9fceea. This reverted commit was recommitted in 831ae45e3d, so relanding this dependent patch too.

Reviewers: morehouse, hctim

Subscribers: cfe-commits, #sanitizers

Tags: #clang, #sanitizers

Differential Revision: https://reviews.llvm.org/D83987
2020-07-27 18:27:49 +00:00