Commit Graph

9768 Commits

Author SHA1 Message Date
Rainer Orth 55c0f12a86 [compiler-rt][sanitizers] Fix Solaris madvise declaration
A last-minute silent change in  D84046 <https://reviews.llvm.org/D84046> broke the Solaris buildbots (Solaris/sparcv9 <http://lab.llvm.org:8014/builders/clang-solaris11-sparcv9/builds/6772>, Solaris/amd64 <http://lab.llvm.org:8014/builders/clang-solaris11-amd64/builds/5434>):

  [2/3679] Building CXX object projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.sparc.dir/sanitizer_posix_libcdep.cpp.o
  FAILED: projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.sparc.dir/sanitizer_posix_libcdep.cpp.o
  /opt/llvm-buildbot/bin/c++  -DHAVE_RPC_XDR_H=1 -D_DEBUG -D_FILE_OFFSET_BITS=64 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iprojects/compiler-rt/lib/sanitizer_common -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/sanitizer_common -Iinclude -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/llvm/include -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/llvm/include/llvm/Support/Solaris -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/sanitizer_common/.. -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -Wall -std=c++14 -Wno-unused-parameter -O3     -m32 -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fvisibility=hidden -fno-lto -O3 -g -Wno-variadic-macros -Wno-non-virtual-dtor -fno-rtti -Wframe-larger-than=570 -UNDEBUG -std=c++14 -MD -MT projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.sparc.dir/sanitizer_posix_libcdep.cpp.o -MF projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.sparc.dir/sanitizer_posix_libcdep.cpp.o.d -o projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonLibc.sparc.dir/sanitizer_posix_libcdep.cpp.o -c /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp
  /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:50:16: error: conflicting declaration of C function ‘int madvise(caddr_t, std::size_t, int)’
   extern "C" int madvise(caddr_t, size_t, int);
                  ^~~~~~~
  In file included from /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:32:0:
  /usr/include/sys/mman.h:232:12: note: previous declaration ‘int madvise(void*, std::size_t, int)’
   extern int madvise(void *, size_t, int);
              ^~~~~~~

This patch undoes that change.

Tested on `amd64-pc-solaris2.11` (Solaris 11.4 and OpenIndiana).

Differential Revision: https://reviews.llvm.org/D84388
2020-07-23 09:37:14 +02:00
Fangrui Song 99ad956fda [PGO] Don't call calloc(0, sizeof(ValueProfNode *))
A malloc implementation may return a pointer to some allocated space.  It is
undefined for libclang_rt.profile- to access the object - which actually happens
in instrumentTargetValueImpl, where ValueCounters[CounterIndex] may access a
ValueProfNode (from another allocated object) and crashes when the code accesses
the object referenced by CurVNode->Next.
2020-07-22 18:49:25 -07:00
Matt Morehouse 19d9c0397e [DFSan] Handle fast16labels for all API functions.
Summary:
Support fast16labels in `dfsan_has_label`, and print an error for all
other API functions.

Reviewers: kcc, vitalybuka, pcc

Reviewed By: kcc

Subscribers: jfb, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D84215
2020-07-22 23:54:26 +00:00
Fangrui Song 27650ec554 Revert D81682 "[PGO] Extend the value profile buckets for mem op sizes."
This reverts commit 4a539faf74.

There is a __llvm_profile_instrument_range related crash in PGO-instrumented clang:

```
(gdb) bt
llvm::ConstantRange const&, llvm::APInt const&, unsigned int, bool) ()
llvm::ScalarEvolution::getRangeForAffineAR(llvm::SCEV const*, llvm::SCEV
const*, llvm::SCEV const*, unsigned int) ()
```

(The body of __llvm_profile_instrument_range is inlined, so we can only find__llvm_profile_instrument_target in the trace)

```
 23│    0x000055555dba0961 <+65>:    nopw   %cs:0x0(%rax,%rax,1)
 24│    0x000055555dba096b <+75>:    nopl   0x0(%rax,%rax,1)
 25│    0x000055555dba0970 <+80>:    mov    %rsi,%rbx
 26│    0x000055555dba0973 <+83>:    mov    0x8(%rsi),%rsi  # %rsi=-1 -> SIGSEGV
 27│    0x000055555dba0977 <+87>:    cmp    %r15,(%rbx)
 28│    0x000055555dba097a <+90>:    je     0x55555dba0a76 <__llvm_profile_instrument_target+342>
```
2020-07-22 16:08:25 -07:00
Andrew Grieve a305d25029 asan_device_setup's wrapper scripts not handling args with spaces correctly
Summary: Came up in Chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=1103108#c21

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D84237
2020-07-22 12:55:28 -07:00
Louis Dionne afa1afd410 [CMake] Bump CMake minimum version to 3.13.4
This upgrade should be friction-less because we've already been ensuring
that CMake >= 3.13.4 is used.

This is part of the effort discussed on llvm-dev here:

  http://lists.llvm.org/pipermail/llvm-dev/2020-April/140578.html

Differential Revision: https://reviews.llvm.org/D78648
2020-07-22 14:25:07 -04:00
Gui Andrade f93b55a5ab [Sanitizers] Add interceptor for xdrrec_create
For now, xdrrec_create is only intercepted Linux as its signature
is different on Solaris.

The method of intercepting xdrrec_create isn't super ideal but I
couldn't think of a way around it: Using an AddrHashMap combined
with wrapping the userdata field.

We can't just allocate a handle on the heap in xdrrec_create and leave
it at that, since there'd be no way to free it later. This is because it
doesn't seem to be possible to access handle from the XDR struct, which
is the only argument to xdr_destroy.
On the other hand, the callbacks don't have a way to get at the
x_private field of XDR, which is what I chose for the HashMap key. So we
need to wrap the handle parameter of the callbacks. But we can't just
pass x_private as handle (as it hasn't been set yet). We can't put the
wrapper struct into the HashMap and pass its pointer as handle, as the
key we need (x_private again) hasn't been set yet.

So I allocate the wrapper struct on the heap, pass its pointer as
handle, and put it into the HashMap so xdr_destroy can find it later and
destroy it.

Differential Revision: https://reviews.llvm.org/D83358
2020-07-22 17:29:41 +00:00
David Carlier c61dcb8f62 [compiler-rt] fix build on Illumos
- there are additional fields for glob_t struct, thus size check is failing.
- to access old mman.h api based on caddr_t, _XOPEN_SOURCE needs to be not defined
 thus we provide the prototype.
- prxmap_t constified.

Reviewers: ro, eugenis

Reviewed-By: ro

Differential Revision: https://reviews.llvm.org/D84046
2020-07-22 15:19:56 +01:00
Vitaly Buka 952fdb6b43 [sanitizer,NFC] InternalAlloc cleanup 2020-07-22 03:08:24 -07:00
Dan Liew 923cf890d1 Avoid failing a CHECK in `DlAddrSymbolizer::SymbolizePC`.
Summary:
It turns out the `CHECK(addr >= reinterpret_cast<upt>(info.dli_saddr)`
can fail because on armv7s on iOS 9.3 `dladdr()` returns
`info.dli_saddr` with an address larger than the address we provided.

We should avoid crashing here because crashing in the middle of reporting
an issue is very unhelpful. Instead we now try to compute a function offset
if the value we get back from `dladdr()` looks sane, otherwise we don't
set the function offset.

A test case is included. It's basically a slightly modified version of
the existing `test/sanitizer_common/TestCases/Darwin/symbolizer-function-offset-dladdr.cpp`
test case that doesn't run on iOS devices right now.

More details:

In the concrete scenario on armv7s `addr` is `0x2195c870` and the returned
`info.dli_saddr` is `0x2195c871`.

This what LLDB says when disassembling the code.

```
(lldb) dis -a 0x2195c870
libdyld.dylib`<redacted>:
    0x2195c870 <+0>: nop
    0x2195c872 <+2>: blx    0x2195c91c                ; symbol stub for: exit
    0x2195c876 <+6>: trap
```

The value returned by `dladdr()` doesn't make sense because it points
into the middle of a instruction.

There might also be other bugs lurking here because I noticed that the PCs we
gather during stackunwinding (before changing them with
`StackTrace::GetPreviousInstructionPc()`) look a little suspicious (e.g.  the
PC stored for the frame with fail to symbolicate is 0x2195c873) as they don't
look properly aligned. This probably warrants further investigation in the future.

rdar://problem/65621511

Reviewers: kubamracek, yln

Subscribers: kristof.beyls, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D84262
2020-07-21 12:49:50 -07:00
Marco Vanotti db00fac2a2 [compiler-rt][asan] decommit shadow memory for unmaps in fuchsia.
This CL allows asan allocator in fuchsia to decommit shadow memory
for memory allocated using mmap.

Big allocations in asan end up being allocated via `mmap` and freed with
`munmap`. However, when that memory is freed, asan returns the
corresponding shadow memory back to the OS via a call to
`ReleaseMemoryPagesToOs`.

In fuchsia, `ReleaseMemoryPagesToOs` is a no-op: to be able to free
memory back to the OS, you have to hold a handle to the vmo you want to
modify, which is tricky at the ReleaseMemoryPagesToOs level as that
function is not exclusively used for shadow memory.

The function `__sanitizer_fill_shadow` fills a given shadow memory range
with a specific value, and if that value is 0 (unpoison) and the memory
range is bigger than a threshold parameter, it will decommit that memory
if it is all zeroes.

This CL modifies the `FlushUnneededASanShadowMemory` function in
`asan_poisoning.cpp` to add a call to `__sanitizer_fill_shadow` with
value and threshold = 0. This way, all the unneeded shadow memory gets
returned back to the OS.

A test for this behavior can be found in fxrev.dev/391974

Differential Revision: https://reviews.llvm.org/D80355

Change-Id: Id6dd85693e78a222f0329d5b2201e0da753e01c0
2020-07-21 11:07:47 -07:00
Mitch Phillips 4f029d1be4 [GWP-ASan] Split the unwinder into segv/non-segv.
Note: Resubmission with frame pointers force-enabled to fix builds with
-DCOMPILER_RT_BUILD_BUILTINS=False

Summary:
Splits the unwinder into a non-segv (for allocation/deallocation traces) and a
segv unwinder. This ensures that implementations can select an accurate, slower
unwinder in the segv handler (if they choose to use the GWP-ASan provided one).
This is important as fast frame-pointer unwinders (like the sanitizer unwinder)
don't like unwinding through signal handlers.

Reviewers: morehouse, cryptoad

Reviewed By: morehouse, cryptoad

Subscribers: cryptoad, mgorny, eugenis, pcc, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D83994
2020-07-21 08:25:37 -07:00
Louis Dionne 14c4de13e9 [compiler-rt] Use -nostdinc++ in clang_rt.profile to avoid including C++ headers
Most of the code in compiler_rt is C code. However, clang_rt.profile
contains the InstrProfilingRuntime.cpp file, which builds as C++. This
means that including e.g. <stdint.h> will actually include libc++'s
<stdint.h> and then #include_next the system's <stdint.h>. However, if
the target we're building compiler-rt for isn't supported by libc++,
this will lead to a failure since libc++'s <stdint.h> includes <__config>,
which performs various checks.

Since the goal seems to *not* be including any header from the C++ Standard
Library in clang_rt.profile, using -nostdinc++ to ensure that doesn't
happen unknowingly seems to make sense.

rdar://65852694

Differential Revision: https://reviews.llvm.org/D84205
2020-07-21 09:14:33 -04:00
Hans Wennborg ab6263c925 Revert 502f0cc0e3 "[GWP-ASan] Split the unwinder into segv/non-segv."
It was causing tests to fail in -DCOMPILER_RT_BUILD_BUILTINS=OFF builds:

   GwpAsan-Unittest :: ./GwpAsan-x86_64-Test/BacktraceGuardedPoolAllocator.DoubleFree
   GwpAsan-Unittest :: ./GwpAsan-x86_64-Test/BacktraceGuardedPoolAllocator.UseAfterFree

see comment on the code review.

> Summary:
> Splits the unwinder into a non-segv (for allocation/deallocation traces) and a
> segv unwinder. This ensures that implementations can select an accurate, slower
> unwinder in the segv handler (if they choose to use the GWP-ASan provided one).
> This is important as fast frame-pointer unwinders (like the sanitizer unwinder)
> don't like unwinding through signal handlers.
>
> Reviewers: morehouse, cryptoad
>
> Reviewed By: morehouse, cryptoad
>
> Subscribers: cryptoad, mgorny, eugenis, pcc, #sanitizers
>
> Tags: #sanitizers
>
> Differential Revision: https://reviews.llvm.org/D83994

This reverts commit 502f0cc0e3.
2020-07-21 11:18:07 +02:00
Logan Smith 8ed021382e Fix typo causing build failure
(facepalm)
2020-07-20 15:14:08 -07:00
Logan Smith 8b6179f48c [NFC] Add missing 'override's 2020-07-20 14:39:36 -07:00
Fangrui Song 5809a32e7c [gcov] Add __gcov_dump/__gcov_reset and delete __gcov_flush
GCC r187297 (2012-05) introduced `__gcov_dump` and `__gcov_reset`.
  `__gcov_flush = __gcov_dump + __gcov_reset`

The resolution to https://gcc.gnu.org/PR93623 ("No need to dump gcdas when forking" target GCC 11.0) removed the unuseful and undocumented __gcov_flush.

Close PR38064.

Reviewed By: calixte, serge-sans-paille

Differential Revision: https://reviews.llvm.org/D83149
2020-07-18 15:07:46 -07:00
Gui Andrade 780528d9da sanitizers: Implement sig{and,or}set interceptors
Also adds a sanitizers-wide test, and a msan test, for these functions.
2020-07-18 03:09:39 +00:00
Vitaly Buka c12f111846 Corrected __libfuzzer_is_present symbol inclusion for MSVC x86 32-bit
The incorrect symbol will cause linking failures for 32-bit targets:

clang_rt.fuzzer-i386.lib(FuzzerDriver.obj) : error LNK2001: unresolved external symbol __libfuzzer_is_present

Verified no longer fails to link with this change for 32-bit and still succeeds for 64-bit MSVC.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D83594
2020-07-17 18:29:47 -07:00
Kuba Mracek 176a6e7abe [asan] Use dynamic shadow memory position on Apple Silicon macOS
This is needed because macOS on Apple Silicon has some reserved pages inside the "regular" shadow memory location, and mapping over that location fails.

Differential Revision: https://reviews.llvm.org/D82912
2020-07-17 17:40:21 -07:00
Teresa Johnson db15b8ab90 [compiler-rt][asan][hwasan] Fix Fuchsia build
Fix build failure in Fuchsia build from refactoring in
5d2be1a188

Guard the moved versions of ReserveShadowMemoryRange and ProtectGap
the same way they were in the asan code originally (not for Fuchsia or
RTEMS). Otherwise we end up with unsats as they invoke functions not
defined there.
2020-07-17 14:50:01 -07:00
Mitch Phillips 502f0cc0e3 [GWP-ASan] Split the unwinder into segv/non-segv.
Summary:
Splits the unwinder into a non-segv (for allocation/deallocation traces) and a
segv unwinder. This ensures that implementations can select an accurate, slower
unwinder in the segv handler (if they choose to use the GWP-ASan provided one).
This is important as fast frame-pointer unwinders (like the sanitizer unwinder)
don't like unwinding through signal handlers.

Reviewers: morehouse, cryptoad

Reviewed By: morehouse, cryptoad

Subscribers: cryptoad, mgorny, eugenis, pcc, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D83994
2020-07-17 12:59:47 -07:00
Vitaly Buka e412cc402e [asan] Dedup MemToShadowSize
Added D83247 and D84004.
2020-07-17 02:40:08 -07:00
Rainer Orth 0db3ac3354 [compiler-rt][asan] Define MemToShadowSize for sparc64
[compiler-rt][asan][hwasan] Refactor shadow setup into sanitizer_common (NFCI) <https://reviews.llvm.org/D83247>
broke the Solaris/sparcv9 <http://lab.llvm.org:8014/builders/clang-solaris11-sparcv9/builds/6623> build:

  FAILED: projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.sparcv9.dir/asan_linux.cpp.o
  /opt/llvm-buildbot/bin/c++  -D_DEBUG -D_FILE_OFFSET_BITS=64 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iprojects/compiler-rt/lib/asan -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/asan -Iinclude -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/llvm/include -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/llvm/include/llvm/Support/Solaris -I/opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/asan/.. -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -Wall -std=c++14 -Wno-unused-parameter -O3     -m64 -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fvisibility=hidden -fno-lto -O3 -g -Wno-variadic-macros -Wno-non-virtual-dtor -fno-rtti -UNDEBUG -std=c++14 -MD -MT projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.sparcv9.dir/asan_linux.cpp.o -MF projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.sparcv9.dir/asan_linux.cpp.o.d -o projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.sparcv9.dir/asan_linux.cpp.o -c /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/asan/asan_linux.cpp
  /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/asan/asan_linux.cpp: In function ‘__sanitizer::uptr __asan::FindDynamicShadowStart()’:
  /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/asan/asan_linux.cpp:103:28: error: ‘MemToShadowSize’ was not declared in this scope
     uptr shadow_size_bytes = MemToShadowSize(kHighMemEnd);
                              ^~~~~~~~~~~~~~~
  /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/compiler-rt/lib/asan/asan_linux.cpp:103:28: note: suggested alternative: ‘MemToShadow’
     uptr shadow_size_bytes = MemToShadowSize(kHighMemEnd);
                              ^~~~~~~~~~~~~~~
                              MemToShadow

Fixed by also definining `MemToShadowSize` in `compiler-rt/lib/asan/asan_mapping_sparc64.h`.

Tested on `sparcv9-sun-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D84004
2020-07-17 10:04:19 +02:00
Vitaly Buka dbf39e54f6 [sanitizer] Revert accidentally committed file 2020-07-17 00:08:45 -07:00
Vitaly Buka 1cb8cc1fd1 [profile] Enabled test on windows
Works with D34797
2020-07-17 00:05:46 -07:00
Logan Smith 947bf0fdf6 [compiler-rt][NFC] Add missing 'override's 2020-07-16 20:07:57 -07:00
Vitaly Buka 147a608b94 [sanitizer] Fix symbolizer build broken by D82702 2020-07-16 19:34:45 -07:00
Richard Smith 8ef9e2bf35 Revert "[libFuzzer] Link libFuzzer's own interceptors when other compiler runtimes are not linked."
This causes binaries linked with this runtime to crash on startup if
dlsym uses any of the intercepted functions. (For example, that happens
when using tcmalloc as the allocator: dlsym attempts to allocate memory
with malloc, and tcmalloc uses strncmp within its implementation.)

Also revert dependent commit "[libFuzzer] Disable implicit builtin knowledge about memcmp-like functions when -fsanitize=fuzzer-no-link is given."

This reverts commit f78d9fceea and 12d1124c49.
2020-07-16 18:06:37 -07:00
Min-Yih Hsu 0f6220ddd6 [profile] Move __llvm_profile_raw_version into a separate file
Similar to the reason behind moving __llvm_profile_filename into a
separate file[1]. When users try to use Full LTO with BFD linker to
generate IR level PGO profile, the __llvm_profile_raw_version variable,
which is used for marking instrumentation level, generated by frontend
would somehow conflict with the weak symbol provided by profiling
runtime.

In most of the cases, BFD linkers will pick profiling runtime's weak symbol
as the real definition and thus generate the incorrect instrumentation
level metadata in the final executables.

Moving __llvm_profile_raw_version into a separate file would make
linkers not seeing the weak symbol in the archive unless the frontend
doesn't generate one.

[1] https://reviews.llvm.org/D34797

Differential Revision: https://reviews.llvm.org/D83967
2020-07-16 16:02:04 -07:00
Jon Roelofs 2cf3458c3b [tsan][go] Fix for missing symbols needed by GotsanRuntimeCheck 2020-07-16 17:01:15 -06:00
Nico Weber 669b070936 cmake list formatting fix 2020-07-16 18:29:48 -04:00
Ryan Prichard 15b37e1cfa [builtins] Omit 80-bit builtins on Android and MSVC
long double is a 64-bit double-precision type on:
 - MSVC (32- and 64-bit x86)
 - Android (32-bit x86)

long double is a 128-bit quad-precision type on x86_64 Android.

The assembly variants of the 80-bit builtins are correct, but some of
the builtins are implemented in C and require that long double be the
80-bit type passed via an x87 register.

Reviewed By: compnerd

Differential Revision: https://reviews.llvm.org/D82153
2020-07-16 15:11:26 -07:00
Teresa Johnson c489ae2a86 [compiler-rt][asan][hwasan] Fix Android bots
Fix failure in Android bots from refactoring in
5d2be1a188 (https://crbug.com/1106482).

We need to make the UnmapFromTo available outside sanitizer_common for
calls from hwasan and asan linux handling. While here, remove
declaration of GetHighMemEnd which is no longer in sanitizer_common.
2020-07-16 15:11:02 -07:00
Dokyung Song f78d9fceea [libFuzzer] Link libFuzzer's own interceptors when other compiler runtimes are not linked.
Summary: libFuzzer intercepts certain library functions such as memcmp/strcmp by defining weak hooks. Weak hooks, however, are called only when other runtimes such as ASan is linked. This patch defines libFuzzer's own interceptors, which is linked into the libFuzzer executable when other runtimes are not linked, i.e., when -fsanitize=fuzzer is given, but not others.

Reviewers: kcc, morehouse, hctim

Reviewed By: morehouse, hctim

Subscribers: krytarowski, mgorny, cfe-commits, #sanitizers

Tags: #clang, #sanitizers

Differential Revision: https://reviews.llvm.org/D83494
2020-07-16 20:26:35 +00:00
serge-sans-paille 515bc8c155 Harmonize Python shebang
Differential Revision: https://reviews.llvm.org/D83857
2020-07-16 21:53:45 +02:00
Julian Lettner bd88991a01 Revert "[Darwin] Fix OS version checks inside simulators"
This reverts commit b16dfbead2.

Accidental push, reverting and creating a new revision.
2020-07-16 11:47:41 -07:00
Teresa Johnson 5d2be1a188 [compiler-rt][asan][hwasan] Refactor shadow setup into sanitizer_common (NFCI)
Summary:
This refactors some common support related to shadow memory setup from
asan and hwasan into sanitizer_common. This should not only reduce code
duplication but also make these facilities available for new compiler-rt
uses (e.g. heap profiling).

In most cases the separate copies of the code were either identical, or
at least functionally identical. A few notes:

In ProtectGap, the asan version checked the address against an upper
bound (kZeroBaseMaxShadowStart, which is (2^18). I have created a copy
of kZeroBaseMaxShadowStart in hwasan_mapping.h, with the same value, as
it isn't clear why that code should not do the same check. If it
shouldn't, I can remove this and guard this check so that it only
happens for asan.

In asan's InitializeShadowMemory, in the dynamic shadow case it was
setting __asan_shadow_memory_dynamic_address to 0 (which then sets both
macro SHADOW_OFFSET as well as macro kLowShadowBeg to 0) before calling
FindDynamicShadowStart(). AFAICT this is only needed because
FindDynamicShadowStart utilizes kHighShadowEnd to
get the shadow size, and kHighShadowEnd is a macro invoking
MEM_TO_SHADOW(kHighMemEnd) which in turn invokes:
(((kHighMemEnd) >> SHADOW_SCALE) + (SHADOW_OFFSET))
I.e. it computes the shadow space needed by kHighMemEnd (the shift), and
adds the offset. Since we only want the shadow space here, the earlier
setting of SHADOW_OFFSET to 0 via __asan_shadow_memory_dynamic_address
accomplishes this. In the hwasan version, it simply gets the shadow
space via "MemToShadowSize(kHighMemEnd)", where MemToShadowSize just
does the shift. I've simplified the asan handling to do the same
thing, and therefore was able to remove the setting of the SHADOW_OFFSET
via __asan_shadow_memory_dynamic_address to 0.

Reviewers: vitalybuka, kcc, eugenis

Subscribers: dberris, #sanitizers, llvm-commits, davidxl

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D83247
2020-07-16 11:47:05 -07:00
Julian Lettner b16dfbead2 [Darwin] Fix OS version checks inside simulators
compiler-rt checks OS versions by querying the Darwin kernel version.
This is not necessarily correct inside the simulators if the simulator
runtime is not aligned with the host macOS.  Let's instead check the
`SIMULATOR_RUNTIME_VERSION` env var.

Note that we still use the old code path as a fallback in case the
`SIMULATOR_RUNTIME_VERSION` environment variable isn't set.

rdar://63031937

Reviewers: delcypher

Differential Revision: https://reviews.llvm.org/D79979
2020-07-16 10:48:04 -07:00
Kostya Kortchinsky 79de8f8441 [scudo][standalone] Release smaller blocks less often
Summary:
Releasing smaller blocks is costly and only yields significant
results when there is a large percentage of free bytes for a given
size class (see numbers below).

This CL introduces a couple of additional checks for sizes lower
than 256. First we want to make sure that there is enough free bytes,
relatively to the amount of allocated bytes. We are looking at 8X% to
9X% (smaller blocks require higher percentage). We also want to make
sure there has been enough activity with the freelist to make it
worth the time, so we now check that the bytes pushed to the freelist
is at least 1/16th of the allocated bytes for those classes.

Additionally, we clear batches before destroying them now - this
could have prevented some releases to occur (class id 0 rarely
releases anyway).

Here are the numbers, for about 1M allocations in multiple threads:

Size: 16
85% freed -> 0% released
86% freed -> 0% released
87% freed -> 0% released
88% freed -> 0% released
89% freed -> 0% released
90% freed -> 0% released
91% freed -> 0% released
92% freed -> 0% released
93% freed -> 0% released
94% freed -> 0% released
95% freed -> 0% released
96% freed -> 0% released
97% freed -> 2% released
98% freed -> 7% released
99% freed -> 27% released
Size: 32
85% freed -> 0% released
86% freed -> 0% released
87% freed -> 0% released
88% freed -> 0% released
89% freed -> 0% released
90% freed -> 0% released
91% freed -> 0% released
92% freed -> 0% released
93% freed -> 0% released
94% freed -> 0% released
95% freed -> 1% released
96% freed -> 3% released
97% freed -> 7% released
98% freed -> 17% released
99% freed -> 41% released
Size: 48
85% freed -> 0% released
86% freed -> 0% released
87% freed -> 0% released
88% freed -> 0% released
89% freed -> 0% released
90% freed -> 0% released
91% freed -> 0% released
92% freed -> 0% released
93% freed -> 0% released
94% freed -> 1% released
95% freed -> 3% released
96% freed -> 7% released
97% freed -> 13% released
98% freed -> 27% released
99% freed -> 52% released
Size: 64
85% freed -> 0% released
86% freed -> 0% released
87% freed -> 0% released
88% freed -> 0% released
89% freed -> 0% released
90% freed -> 0% released
91% freed -> 0% released
92% freed -> 1% released
93% freed -> 2% released
94% freed -> 3% released
95% freed -> 6% released
96% freed -> 11% released
97% freed -> 20% released
98% freed -> 35% released
99% freed -> 59% released
Size: 80
85% freed -> 0% released
86% freed -> 0% released
87% freed -> 0% released
88% freed -> 0% released
89% freed -> 0% released
90% freed -> 1% released
91% freed -> 1% released
92% freed -> 2% released
93% freed -> 4% released
94% freed -> 6% released
95% freed -> 10% released
96% freed -> 17% released
97% freed -> 26% released
98% freed -> 41% released
99% freed -> 64% released
Size: 96
85% freed -> 0% released
86% freed -> 0% released
87% freed -> 0% released
88% freed -> 0% released
89% freed -> 1% released
90% freed -> 1% released
91% freed -> 3% released
92% freed -> 4% released
93% freed -> 6% released
94% freed -> 10% released
95% freed -> 14% released
96% freed -> 21% released
97% freed -> 31% released
98% freed -> 47% released
99% freed -> 68% released
Size: 112
85% freed -> 0% released
86% freed -> 1% released
87% freed -> 1% released
88% freed -> 2% released
89% freed -> 3% released
90% freed -> 4% released
91% freed -> 6% released
92% freed -> 8% released
93% freed -> 11% released
94% freed -> 16% released
95% freed -> 22% released
96% freed -> 30% released
97% freed -> 40% released
98% freed -> 55% released
99% freed -> 74% released
Size: 128
85% freed -> 0% released
86% freed -> 1% released
87% freed -> 1% released
88% freed -> 2% released
89% freed -> 3% released
90% freed -> 4% released
91% freed -> 6% released
92% freed -> 8% released
93% freed -> 11% released
94% freed -> 16% released
95% freed -> 22% released
96% freed -> 30% released
97% freed -> 40% released
98% freed -> 55% released
99% freed -> 74% released
Size: 144
85% freed -> 1% released
86% freed -> 2% released
87% freed -> 3% released
88% freed -> 4% released
89% freed -> 6% released
90% freed -> 7% released
91% freed -> 10% released
92% freed -> 13% released
93% freed -> 17% released
94% freed -> 22% released
95% freed -> 28% released
96% freed -> 37% released
97% freed -> 47% released
98% freed -> 61% released
99% freed -> 78% released
Size: 160
85% freed -> 1% released
86% freed -> 2% released
87% freed -> 3% released
88% freed -> 4% released
89% freed -> 5% released
90% freed -> 7% released
91% freed -> 10% released
92% freed -> 13% released
93% freed -> 17% released
94% freed -> 22% released
95% freed -> 28% released
96% freed -> 37% released
97% freed -> 47% released
98% freed -> 61% released
99% freed -> 78% released
Size: 176
85% freed -> 2% released
86% freed -> 3% released
87% freed -> 4% released
88% freed -> 6% released
89% freed -> 7% released
90% freed -> 9% released
91% freed -> 12% released
92% freed -> 15% released
93% freed -> 20% released
94% freed -> 25% released
95% freed -> 32% released
96% freed -> 40% released
97% freed -> 51% released
98% freed -> 64% released
99% freed -> 80% released
Size: 192
85% freed -> 4% released
86% freed -> 5% released
87% freed -> 6% released
88% freed -> 8% released
89% freed -> 10% released
90% freed -> 13% released
91% freed -> 16% released
92% freed -> 20% released
93% freed -> 24% released
94% freed -> 30% released
95% freed -> 37% released
96% freed -> 45% released
97% freed -> 55% released
98% freed -> 68% released
99% freed -> 82% released
Size: 224
85% freed -> 8% released
86% freed -> 10% released
87% freed -> 12% released
88% freed -> 14% released
89% freed -> 17% released
90% freed -> 20% released
91% freed -> 23% released
92% freed -> 28% released
93% freed -> 33% released
94% freed -> 39% released
95% freed -> 46% released
96% freed -> 53% released
97% freed -> 63% released
98% freed -> 73% released
99% freed -> 85% released
Size: 240
85% freed -> 8% released
86% freed -> 10% released
87% freed -> 12% released
88% freed -> 14% released
89% freed -> 17% released
90% freed -> 20% released
91% freed -> 23% released
92% freed -> 28% released
93% freed -> 33% released
94% freed -> 39% released
95% freed -> 46% released
96% freed -> 54% released
97% freed -> 63% released
98% freed -> 73% released
99% freed -> 85% released

Reviewers: cferris, pcc, hctim, eugenis

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D82031
2020-07-16 09:44:25 -07:00
Joachim Protze 7358a1104a [TSan] Optimize handling of racy address
This patch splits the handling of racy address and racy stack into separate
functions. If a race was already reported for the address, we can avoid the
cost for collecting the involved stacks.

This patch also removes the race condition in storing the racy address / racy
stack. This race condition allowed all threads to report the race.

This patch changes the transitive suppression of reports. Previously
suppression could transitively chain memory location and racy stacks.
Now racy memory and racy stack are separate suppressions.

Commit again, now with fixed tests.

Reviewed by: dvyukov

Differential Revision: https://reviews.llvm.org/D83625
2020-07-16 16:22:57 +02:00
Hiroshi Yamauchi 4a539faf74 [PGO] Extend the value profile buckets for mem op sizes.
Extend the memop value profile buckets to be more flexible (could accommodate a
mix of individual values and ranges) and to cover more value ranges (from 11 to
22 buckets).

Disabled behind a flag (to be enabled separately) and the existing code to be
removed later.
2020-07-15 10:26:15 -07:00
Joachim Protze d3849dddd2 Revert "[TSan] Optimize handling of racy address"
This reverts commit 00e3a1ddec.
The commit broke most build bots, investigating.
2020-07-15 17:40:28 +02:00
Joachim Protze 00e3a1ddec [TSan] Optimize handling of racy address
This patch splits the handling of racy address and racy stack into separate
functions. If a race was already reported for the address, we can avoid the
cost for collecting the involved stacks.

This patch also removes the race condition in storing the racy address / racy
stack. This race condition allowed all threads to report the race.

This patch changes the transitive suppression of reports. Previously
suppression could transitively chain memory location and racy stacks.
Now racy memory and racy stack are separate suppressions.

Reviewed by: dvyukov

Differential Revision: https://reviews.llvm.org/D83625
2020-07-15 16:50:08 +02:00
Dokyung Song 750369e2e8 [libFuzzer] Fix compilation error by including missing platform macro definitions.
Summary: This patch fixes sanitizer-windows build errors.

Reviewers: morehouse, hctim

Reviewed By: morehouse, hctim

Subscribers: #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D83823
2020-07-14 22:26:50 +00:00
Dokyung Song 226866e115 [libFuzzer] Separate platform related macros out from FuzzerDefs.h into FuzzerPlatform.h, and adjust includes in other files.
Summary: This patch separates platform related macros in lib/fuzzer/FuzzerDefs.h into lib/fuzzer/FuzzerPlatform.h, and use FuzzerPlatform.h where necessary. This separation helps when compiling libFuzzer's interceptor module (under review); an unnecessary include of standard headers (such as string.h) may produce conflicts/ambiguation with the interceptor's declarations/definitions of library functions, which complicates interceptor implementation.

Reviewers: morehouse, hctim

Reviewed By: morehouse

Subscribers: krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D83805
2020-07-14 21:31:56 +00:00
Vedant Kumar a8694eb562 Update ubsan_interface.inc for D71491 (second try)
I mistyped the ubsan objc_cast handler names on the first try.

Testing:

./bin/llvm-lit projects/compiler-rt/test/asan/X86_64HDarwinConfig/TestCases/Darwin/interface_symbols_darwin.cpp
2020-07-14 11:16:11 -07:00
Rainer Orth f0e9b76c35 [compiler-rt][sanitizers] Fix GetPcSpBp determination of SP on 32-bit Solaris/x86
A dozen 32-bit `AddressSanitizer` testcases FAIL on the latest beta of Solaris 11.4/x86, e.g.
`AddressSanitizer-i386-sunos :: TestCases/null_deref.cpp` produces

  AddressSanitizer:DEADLYSIGNAL
  =================================================================
  ==29274==ERROR: AddressSanitizer: stack-overflow on address 0x00000028 (pc 0x08135efd bp 0xfeffdfd8 sp 0x00000000 T0)
      #0 0x8135efd in NullDeref(int*) /vol/llvm/src/llvm-project/dist/compiler-rt/test/asan/TestCases/null_deref.cpp:15:10
      #1 0x8135ea6 in main /vol/llvm/src/llvm-project/dist/compiler-rt/test/asan/TestCases/null_deref.cpp:21:3
      #2 0x8084b85 in _start (null_deref.cpp.tmp+0x8084b85)

   SUMMARY: AddressSanitizer: stack-overflow /vol/llvm/src/llvm-project/dist/compiler-rt/test/asan/TestCases/null_deref.cpp:15:10 in NullDeref(int*)
  ==29274==ABORTING

instead of the expected

  AddressSanitizer:DEADLYSIGNAL
  =================================================================
  ==29276==ERROR: AddressSanitizer: SEGV on unknown address 0x00000028 (pc 0x08135f1f bp 0xfeffdf48 sp 0xfeffdf40 T0)
  ==29276==The signal is caused by a WRITE memory access.
  ==29276==Hint: address points to the zero page.
      #0 0x8135f1f in NullDeref(int*) /vol/llvm/src/llvm-project/local/compiler-rt/test/asan/TestCases/null_deref.cpp:15:10
      #1 0x8135efa in main /vol/llvm/src/llvm-project/local/compiler-rt/test/asan/TestCases/null_deref.cpp:21:3
      #2 0x8084be5 in _start (null_deref.cpp.tmp+0x8084be5)

  AddressSanitizer can not provide additional info.
   SUMMARY: AddressSanitizer: SEGV /vol/llvm/src/llvm-project/local/compiler-rt/test/asan/TestCases/null_deref.cpp:15:10 in NullDeref(int*)
  ==29276==ABORTING

I managed to trace this to a change in `<sys/regset.h>`: previously the header would
primarily define the short register indices (like `UESP`). While they are required by the
i386 psABI, they are only required in `<ucontext.h>` and could previously leak into
unsuspecting user code, polluting the namespace and requiring elaborate workarounds
like that in `llvm/include/llvm/Support/Solaris/sys/regset.h`. The change fixed that by restricting
the definition of the short forms appropriately, at the same time defining all `REG_` prefixed
forms for compatiblity with other systems.  This exposed a bug in `compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp`, however:
Previously, the index for the user stack pointer would be hardcoded if `REG_ESP`
wasn't defined. Now with that definition present, it turned out that `REG_ESP` was the wrong index to use: the previous value 17 (and `REG_SP`) corresponds to `REG_UESP`
instead.

With that change, the failures are all gone.

Tested on `amd-pc-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D83664
2020-07-14 12:56:18 +02:00
Ryan Prichard 8cbb6ccc7f [builtins] Cleanup generic-file filtering
Split filter_builtin_sources into two functions:
 - filter_builtin_sources that removes generic files when an
   arch-specific file is selected.
 - darwin_filter_builtin_sources that implements the EXCLUDE/INCLUDE
   lists (using the files in lib/builtins/Darwin-excludes).

darwin_filter_builtin_sources delegates to filter_builtin_sources.

Previously, lib/builtins/CMakeLists.txt had a number of calls to
filter_builtin_sources (with a confusing/broken use of the
`excluded_list` parameter), as well as a redundant arch-vs-generic
filtering for the non-Apple code path at the end of the file. Replace
all of this with a single call to filter_builtin_sources.

Remove i686_SOURCES. Previously, this list contained only the
arch-specific files common to 32-bit and 64-bit x86, which is a strange
set. Normally the ${ARCH}_SOURCES list contains everything needed for
the arch. "i686" isn't in ALL_BUILTIN_SUPPORTED_ARCH.

NFCI, but i686_SOURCES won't be defined, and the order of files in
${arch}_SOURCES lists will change.

Differential Revision: https://reviews.llvm.org/D82151
2020-07-13 16:53:07 -07:00
Ryan Prichard f398e0f3d1 [builtins][Android] Define HAS_80_BIT_LONG_DOUBLE to 0
Android 32-bit x86 uses a 64-bit long double.

Android 64-bit x86 uses a 128-bit quad-precision long double.

Differential Revision: https://reviews.llvm.org/D82152
2020-07-13 16:53:07 -07:00