Commit Graph

13511 Commits

Author SHA1 Message Date
Julian Lettner 8b0bd54d0e [sanitizer][Darwin] Suppress -Wno-non-virtual-dtor warning
Suppress the warning:
```
'fake_shared_weak_count' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
```

The warning has been recently enabled [1], but the associated cleanup
missed this instance in Darwin code [2].

[1] 9c31e12609
[2] d48f2d7c02

Differential Revision: https://reviews.llvm.org/D94139
2021-01-05 17:09:18 -08:00
Fangrui Song b7718b6175 Reland D93848 "[sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl""
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).

Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).

This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:

* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)

Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).

Many sanitizer features are available now.

```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
  Unsupported      : 185
  Passed           : 512
  Expectedly Failed:   1
  Failed           :  12

% ninja check-ubsan check-ubsan-minimal check-memprof # all passed

% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
  Unsupported      : 264
  Passed           :  80
  Expectedly Failed:   8
  Failed           :  32

% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
  Unsupported:  7
  Passed     : 65
  Failed     : 22

% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
  Unsupported      :   6
  Passed           : 764
  Expectedly Failed:   2
  Failed           :  58

% ninja check-tsan
Testing Time: 23.21s
  Unsupported      :  86
  Passed           : 295
  Expectedly Failed:   1
  Failed           :  25
```

Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.

Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93848
2021-01-05 16:18:43 -08:00
Roland McGrath 90b8fd6136 scudo: Fix compilation for non-Linux aarch64
Linux-specific aarch64 code was deconditionalized in commit
dfa40840e0.  This broke builds for
non-Linux aarch64 platforms.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D94108
2021-01-05 13:21:52 -08:00
Mitch Phillips 1f8031cd74 [android] Fix some tests for AOSP-master devices.
Some tests are broken at API level 30 on AOSP-master devices. When we
change the buildbuit to API level 30, the following tests get enabled.
They're currently broken due to various issues, and so fix up those
issues.

Reviewed By: oontvoo, eugenis

Differential Revision: https://reviews.llvm.org/D94100
2021-01-05 12:54:09 -08:00
Alan Phipps 9f2967bcfe [Coverage] Add support for Branch Coverage in LLVM Source-Based Code Coverage
This is an enhancement to LLVM Source-Based Code Coverage in clang to track how
many times individual branch-generating conditions are taken (evaluate to TRUE)
and not taken (evaluate to FALSE).  Individual conditions may comprise larger
boolean expressions using boolean logical operators.  This functionality is
very similar to what is supported by GCOV except that it is very closely
anchored to the ASTs.

Differential Revision: https://reviews.llvm.org/D84467
2021-01-05 09:51:51 -06:00
Martin Storsjö 979c38cc74 [compiler-rt] [windows] Add UNUSED attributes on variables/functions only used for 64 bit targets
This fixes warnings when building for 32 bit targets.

Differential Revision: https://reviews.llvm.org/D91852
2021-01-05 08:59:08 +02:00
Jan Kratochvil 6988f7a6f4 [compiler-rt] [Sanitizers] Extend ThreadDescriptorSize() for glibc-2.32-2.fc33.x86_64+i686
before:
  $ echo 'int main(){}'|clang -g -fsanitize=leak -x c++ -;./a.out
  Tracer caught signal 11: addr=0x7f4f73da5f40 pc=0x4222c8 sp=0x7f4f72cffd40
  ==1164171==LeakSanitizer has encountered a fatal error.
  ==1164171==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
  ==1164171==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
  $ _

after:
  $ echo 'int main(){}'|clang -g -fsanitize=leak -x c++ -;./a.out)
  $ _

I haven't verified the size cannot be affected by Fedora patches of
upstream glibc-2.32 - but I do not expect upstream glibc-2.32 would have
the last sizes `(1216, 2304)` from 2013 around glibc-2.12.

Differential Revision: https://reviews.llvm.org/D93386
2021-01-04 03:21:04 +01:00
Nico Weber fe9976c02c Revert "[sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl"
...and follow-ups. It still doesn't build on Android, see https://reviews.llvm.org/D93848#2476310

This reverts commit a92d01534f.
This reverts commit 52d7e183bf.
This reverts commit 34489da81b.
2021-01-02 18:59:26 -05:00
Fangrui Song 34489da81b [sanitizer] Enable mallopt and mallinfo interceptors on Android after D93848
Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D93970
2021-01-01 14:43:08 -08:00
Fangrui Song 20670ba440 [compiler-rt][test] Suppress stderr of ldd output 2021-01-01 14:09:13 -08:00
Fangrui Song 52d7e183bf [sanitizer] Include fstab.h on glibc/FreeBSD/NetBSD/macOS 2020-12-31 00:55:58 -08:00
Fangrui Song a92d01534f [sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).

Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).

This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:

* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)

Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).

Many sanitizer features are available now.

```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
  Unsupported      : 185
  Passed           : 512
  Expectedly Failed:   1
  Failed           :  12

% ninja check-ubsan check-ubsan-minimal check-memprof # all passed

% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
  Unsupported      : 264
  Passed           :  80
  Expectedly Failed:   8
  Failed           :  32

% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
  Unsupported      :   6
  Passed           : 764
  Expectedly Failed:   2
  Failed           :  58

% ninja check-tsan
Testing Time: 23.21s
  Unsupported      :  86
  Passed           : 295
  Expectedly Failed:   1
  Failed           :  25
```

Used `ASAN_OPTIONS=verbosity=2` to verify no unneeded interceptors.

Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93848
2020-12-31 00:44:25 -08:00
Vitaly Buka 9a0237011b [lsan] Ignore inderect leaks referenced by suppressed blocks
This makes suppression list to work similar to __lsan_ignore_object.

Existing behavior was inconsistent and very inconvenient for complex
data structures.

Example:

struct B;
struct A { B* ptr; };
A* t = makeA();
t->ptr = makeB();

Before the patch: if makeA suppressed by suppression file, lsan will
still report the makeB() leak, so we need two suppressions.

After the patch: a single makeA suppression is enough (the same as a
single __lsan_ignore_object(t)).

Differential Revision: https://reviews.llvm.org/D93884
2020-12-30 19:11:39 -08:00
Max Moroz 70de7e0d9a [compiler-rt] FuzzedDataProvider: Add PickValueInArray for std::array
This makes `PickValueInArray` work for `std::array<T, s>` (C++11). I've also tested the C++17 `std::array` (with compiler-deduced template parameters)

```
Author:
MarcoFalke <falke.marco@gmail.com>
```

Reviewed By: Dor1s

Differential Revision: https://reviews.llvm.org/D93412
2020-12-30 10:25:26 -08:00
Fangrui Song 9c0c123b0b [CMake][tsan] Remove --sysroot=.
rL254966 added `--sysroot=.` to prevent accidental including system headers.
It caused hassle to FreeBSD (D17383)/NetBSD. The next problem is that
we want to include `features.h` (usually `/usr/include/features.h`) to detect `__GLIBC__`.

At this point it seems that `--sysroot=.` adds lots of inconvenience so we disable it for now.
If there is a better way preventing accidental system header inclusion we can consider it again.

Reviewed By: #sanitizers, vitalybuka

Differential Revision: https://reviews.llvm.org/D93921
2020-12-30 09:30:58 -08:00
Vitaly Buka 9b25b8068d [NFC][lsan] Extract PrintResults function 2020-12-29 17:28:31 -08:00
Vitaly Buka 9a5261efd7 [lsan] Parse suppressions just before leak reporting
Without leaks suppressions are not needed.
2020-12-29 17:28:31 -08:00
Vitaly Buka 3c0d36f977 [NFC][lsan] Add nested leak in test 2020-12-29 14:01:43 -08:00
Vitaly Buka ababeca34b [NFC][sanitizer] Add SortAndDedup function 2020-12-29 14:01:43 -08:00
Vitaly Buka 4e74480e02 [NFC][sanitizer] Simplify InternalLowerBound 2020-12-29 14:01:43 -08:00
Vitaly Buka 673b12e76f [tsan] Remove stdlib.h from dd_interceptors.cpp
This fixes "realpath already defined" error.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D93877
2020-12-29 14:00:54 -08:00
Michael Stapelberg c1e85b6c1b sanitizer: fix typo/spelling: Dissassemble → Disassemble
Differential Revision: https://reviews.llvm.org/D93902
2020-12-29 12:26:57 -05:00
Fangrui Song a8970dff1a [ubsan][test] FLush stdout before checking interleaved stdout/stderr
Detected by musl.
2020-12-28 20:30:32 -08:00
Fangrui Song 55d13e6a86 [asan][test] Annotate glibc specific tests with REQUIRES: glibc-2.27 2020-12-28 19:56:08 -08:00
Vitaly Buka b4655a0815 [NFC][sanitizer] Remove unused typedef 2020-12-28 13:17:07 -08:00
Fangrui Song 60afb58bfe [msan] Delete unused glibc header <execinfo.h>
The file does not call backtrace/backtrace_symbols.
2020-12-27 21:59:23 -08:00
Fangrui Song 99d650b369 [compiler-rt][test] Make glibc-* feature detection work on a musl distribution
... where `ldd --version` has empty stdout and non-empty stderr.
2020-12-27 21:50:47 -08:00
Fangrui Song 0b56e3cdda [sanitizer] Defined SANITIZER_TEST_HAS_PVALLOC only on glibc
This simplifies the condition and makes it work on musl.
2020-12-27 21:33:41 -08:00
Fangrui Song fde3ae88ee [asan][memprof] Declare _DYNAMIC and fix -Wparentheses
Declare `extern ElfW(Dyn) _DYNAMIC[];` so that it will trivially work on musl.
2020-12-27 20:28:59 -08:00
Peter Collingbourne faac1c02c8 scudo: Move the management of the UseMemoryTagging bit out of the Primary. NFCI.
The primary and secondary allocators will need to share this bit,
so move the management of the bit to the combined allocator and
make useMemoryTagging() a free function.

Differential Revision: https://reviews.llvm.org/D93730
2020-12-22 16:52:54 -08:00
Peter Collingbourne e6b3db6309 scudo: Replace the Cache argument on MapAllocator with a Config argument. NFCI.
This will allow the secondary allocator to access the
MaySupportMemoryTagging bool.

Differential Revision: https://reviews.llvm.org/D93729
2020-12-22 16:52:48 -08:00
Peter Collingbourne 6dfe5801e0 scudo: Move the configuration for the primary allocator to Config. NFCI.
This will allow the primary and secondary allocators to share
the MaySupportMemoryTagging bool.

Differential Revision: https://reviews.llvm.org/D93728
2020-12-22 14:54:40 -08:00
Alex Richardson 6277bd75dc [compiler-rt] Fix atomic_test.c on macOS
The macOS name mangling adds another underscore. Therefore, on macOS
the __atomic_* functions are actually ___atomic_* in libcompiler_rt.dylib.
To handle this case, prepend the asm() argument with __USER_LABEL_PREFIX__
in the same way that atomic.c does.

Reviewed By: ldionne

Differential Revision: https://reviews.llvm.org/D92833
2020-12-22 11:56:20 +00:00
Nemanja Ivanovic ba1202a1e4 [PowerPC] Restore stack ptr from base ptr when available
On subtargets that have a red zone, we will copy the stack pointer to the base
pointer in the prologue prior to updating the stack pointer. There are no other
updates to the base pointer after that. This suggests that we should be able to
restore the stack pointer from the base pointer rather than loading it from the
back chain or adding the frame size back to either the stack pointer or the
frame pointer.
This came about because functions that call setjmp need to restore the SP from
the FP because the back chain might have been clobbered
(see https://reviews.llvm.org/D92906). However, if the stack is realigned, the
restored SP might be incorrect (which is what caused the failures in the two
ASan test cases).

This patch was tested quite extensivelly both with sanitizer runtimes and
general code.

Differential revision: https://reviews.llvm.org/D93327
2020-12-22 05:44:03 -06:00
Peter Collingbourne dfa40840e0 scudo: Remove ANDROID_EXPERIMENTAL_MTE macro.
Kernel support for MTE has been released in Linux 5.10. This means
that it is a stable API and we no longer need to make the support
conditional on a macro. We do need to provide conditional definitions
of the new macros though in order to avoid a dependency on new
kernel headers.

Differential Revision: https://reviews.llvm.org/D93513
2020-12-21 10:53:24 -08:00
Mitch Phillips 13261f4c03 Revert "[sanitizer-common] Force pickup of llvm-symbolizer from new binaries."
This reverts commit 66ee0d3d84.

Broke the bots, reverting for full fix.
2020-12-17 16:17:56 -08:00
Mitch Phillips ab1a05d57f Revert "[sanitizer-common] Pickup llvm-symbolizer from $OUT/bin IFF exists."
This reverts commit 30d292ddbb.

Broke the bots, reverting for full fix.
2020-12-17 16:17:56 -08:00
Peter Collingbourne e22d802e58 scudo: Adjust test to use correct check for primary allocations.
canAllocate() does not take into account the header size so it does
not return the right answer in borderline cases. There was already
code handling this correctly in isTaggedAllocation() so split it out
into a separate function and call it from the test.

Furthermore the test was incorrect when MTE is enabled because MTE
does not pattern fill primary allocations. Fix it.

Differential Revision: https://reviews.llvm.org/D93437
2020-12-17 10:42:17 -08:00
Kostya Kortchinsky 1dbf2c96bc [scudo][standalone] Allow the release of smaller sizes
Initially we were avoiding the release of smaller size classes due to
the fact that it was an expensive operation, particularly on 32-bit
platforms. With a lot of batches, and given that there are a lot of
blocks per page, this was a lengthy operation with little results.

There has been some improvements since then to the 32-bit release,
and we still have some criterias preventing us from wasting time
(eg, 9x% free blocks in the class size, etc).

Allowing to release blocks < 128 bytes helps in situations where a lot
of small chunks would not have been reclaimed if not for a forced
reclaiming.

Additionally change some `CHECK` to `DCHECK` and rearrange a bit the
code.

I didn't experience any regressions in my benchmarks.

Differential Revision: https://reviews.llvm.org/D93141
2020-12-17 10:01:57 -08:00
Mitch Phillips 30d292ddbb [sanitizer-common] Pickup llvm-symbolizer from $OUT/bin IFF exists.
Fix-forward for D93352.

Slight rework of the same idea, pickup the external symbolizer from the
binary directory iff it exists.
2020-12-16 11:59:10 -08:00
Mitch Phillips 66ee0d3d84 [sanitizer-common] Force pickup of llvm-symbolizer from new binaries.
It's possible currently that the sanitizer runtimes when testing grab
the path to the symbolizer through *SAN_SYMBOLIZER_PATH=...

This can be polluted by things like Android's setup script. This patch
forces external_symbolizer_path=$new_build_out_dir/llvm-symbolizer when
%env_tool_options is used.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93352
2020-12-16 06:36:26 -08:00
Stephan Bergmann 75f98f0f8c [PATCH] [compiler-rt] [builtins] Fix name of __aarch64_have_lse_atomics on Darwin
...where the name of that variable defined in
compiler-rt/lib/builtins/cpu_model.c is decorated with a leading underscore

Differential Revision: https://reviews.llvm.org/D93390
2020-12-16 14:42:03 +01:00
Raul Tambre b2851aea80 Revert "[Compiler-rt][AArch64] Workaround for .cfi_startproc assembler parser bug."
039cb03dd0 (D93378) fixed the assembly separator, so the workaround is no longer necessary.

This reverts commit 3000c19df6.

Differential Revision: https://reviews.llvm.org/D93379
2020-12-16 10:22:04 +02:00
Fangrui Song 039cb03dd0 [builtins] Use %% as the separator for aarch64-*-darwin and ; for powerpc{32,64}
`;` is the default comment marker, which is also used by powerpc*-*-elf target triples.
`@` is the comment marker of powerpc*-*-darwin but the Darwin support has been deleted for PowerPC (D72063).
`%%` is the statement separator used by aarch64-*-darwin (see AArch64MCAsmInfoDarwin, it uses `;` as the comment marker, which is different from most other targets)

Reviewed By: tambre

Differential Revision: https://reviews.llvm.org/D93378
2020-12-16 00:13:22 -08:00
Raul Tambre a72d462eb9 Re-apply "[CMake][compiler-rt][AArch64] Avoid preprocessing LSE builtins separately"
aa772fc85e (D92530) has landed fixing relocations on Darwin.
3000c19df6 (D93236) has landed working around an assembly parser bug on Darwin.
Previous quick-fix d9697c2e6b (D93198) included in this commit.

Invoking the preprocessor ourselves is fragile and would require us to replicate CMake's handling of definitions, compiler flags, etc for proper compatibility.
In my toolchain builds this notably resulted in a bunch of warnings from unused flags as my CMAKE_C_FLAGS includes CPU-specific optimization options.
Notably this part was already duplicating the logic for VISIBILITY_HIDDEN define.

Instead, symlink the files and set the proper set of defines on each.
This should also be faster as we avoid invoking the compiler multiple times.

Fixes https://llvm.org/PR48494

Differential Revision: https://reviews.llvm.org/D93278
2020-12-15 08:18:07 +02:00
Raul Tambre 3000c19df6 [Compiler-rt][AArch64] Workaround for .cfi_startproc assembler parser bug.
Put .cfi_startproc on a new line to avoid hitting the assembly parser bug in MasmParser::parseDirectiveCFIStartProc().

Reviewed By: tambre

Differential Revision: https://reviews.llvm.org/D93236
2020-12-15 08:14:42 +02:00
Nemanja Ivanovic eed0b9acdf [PowerPC] Temporarily disable asan longjmp tests
Commit bfdc19e778 seems to have broken
some PPC bots with a couple of asan test cases. Disable those test
cases for now until I can resolve the issue.
2020-12-14 18:22:08 -06:00
Peter Collingbourne f21f3339ba scudo: Remove positional template arguments for secondary cache. NFCI.
Make these arguments named constants in the Config class instead
of being positional arguments to MapAllocatorCache. This makes the
configuration easier to follow.

Eventually we should follow suit with the other classes but this is
a start.

Differential Revision: https://reviews.llvm.org/D93251
2020-12-14 15:40:07 -08:00
Kuba Mracek f276c00898 [sanitizer] Restrict querying VM size on Darwin only to iOS devices
We currently do this for SANITIZER_IOS, which includes devices *and* simulators. This change opts out the check for simulators to unify the behavior with macOS, because VM size is really a property of the host OS, and not the simulator.

<rdar://problem/72129387>

Differential Revision: https://reviews.llvm.org/D93140
2020-12-14 10:48:48 -08:00
Raul Tambre c21df2a79c Revert "Re-apply "[CMake][compiler-rt][AArch64] Avoid preprocessing LSE builtins separately""
This reverts commit 03ebe1937192c247c4a7b8ec19dde2cf9845c914.

It's still breaking bots, e.g. http://green.lab.llvm.org/green/job/clang-stage1-RA/17027/console although it doesn't change any actual code.
The compile errors don't make much sense either. Revert for now.

Differential Revision: https://reviews.llvm.org/D93228
2020-12-14 18:43:55 +02:00