Commit Graph

9218 Commits

Author SHA1 Message Date
Rainer Orth 4a9a772f44 Enable compiler-rt on SPARC
This patch enables compiler-rt on SPARC targets. Most of the changes are straightforward:

- Add 32 and 64-bit sparc to compiler-rt

- lib/builtins/fp_lib.h needed to check if the int128_t and uint128_t types exist (which they don't on sparc)

There's one issue of note: many asan tests fail to compile on Solaris/SPARC:

fatal error: error in backend: Function "_ZN7testing8internal16BoolFromGTestEnvEPKcb": over-aligned dynamic alloca not supported.

Therefore, while asan is still built, both asan and ubsan-with-asan testing is disabled. The
goal is to check if asan keeps compiling on Solaris/SPARC. This serves asan in gcc,
which doesn't have the problem above and works just fine.

With this patch, sparcv9-sun-solaris2.11 test results are pretty good:

Failing Tests (9):
    Builtins-sparc-sunos :: divtc3_test.c
    Builtins-sparcv9-sunos :: compiler_rt_logbl_test.c
    Builtins-sparcv9-sunos :: divtc3_test.c
[...]
    UBSan-Standalone-sparc :: TestCases/TypeCheck/misaligned.cpp
    UBSan-Standalone-sparcv9 :: TestCases/TypeCheck/misaligned.cpp

The builtin failures are due to Bugs 42493 and 42496. The tree contained a few additonal
patches either currently in review or about to be submitted.

Tested on sparcv9-sun-solaris2.11.

Differential Revision: https://reviews.llvm.org/D40943

llvm-svn: 365880
2019-07-12 08:30:17 +00:00
Rainer Orth 56f2f1cdb3 [Sanitizers] Fix SanitizerCommon-Unit :: ./Sanitizer-*-Test/MemoryMappingLayout.DumpListOfModules on Solaris
The MemoryMappingLayout.DumpListOfModules currently FAILs on Solaris:

  [ RUN      ] MemoryMappingLayout.DumpListOfModules
  /vol/llvm/src/compiler-rt/local/lib/sanitizer_common/tests/sanitizer_procmaps_test.cc:52: Failure
  Value of: found
    Actual: false
  Expected: true
  [  FAILED  ] MemoryMappingLayout.DumpListOfModules (22 ms)

The problem is that the test expects the executable name from modules[i].full_name(),
however the pr_mapname field of struct prmap is just the entry in /proc/<pid>/object,
which is "a.out" instead of "Sanitizer-i386-Test".  Fortunately, the real name can
be determined by looking in proc/<pid>/path where "a.out" is a symlink to the
real path.

Tested on x86_64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D64559

llvm-svn: 365879
2019-07-12 08:27:50 +00:00
Kostya Kortchinsky 8f18a4c980 [scudo][standalone] NFC corrections
Summary:
A few corrections:
- rename `TransferBatch::MaxCached` to `getMaxCached` to conform with
  the style guide;
- move `getBlockBegin` from `Chunk::` to `Allocator::`: I believe it
  was a fallacy to have this be a `Chunk` method, as chunks'
  relationship to backend blocks are up to the frontend allocator. It
  makes more sense now, particularly with regard to the offset. Update
  the associated chunk test as the method isn't available there
  anymore;
- add a forgotten `\n` to a log string;
- for `releaseToOs`, instead of starting at `1`, start at `0` and
  `continue` on `BatchClassId`: in the end it's identical but doesn't
  assume a particular class id for batches;
- change a `CHECK` to a `reportOutOfMemory`: it's a clearer message

Reviewers: hctim, morehouse, eugenis, vitalybuka

Reviewed By: hctim

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64570

llvm-svn: 365816
2019-07-11 19:55:53 +00:00
Julian Lettner 9d41429ff7 [TSan] Move DECLARE_REAL macro outside of namespace
This should fix the "undefined reference to
tsan::interception::real_setjmp" linker errors.

llvm-svn: 365806
2019-07-11 18:26:33 +00:00
Mitch Phillips af3dc759e7 Explicitly define __STDC_FORMAT_MACROS for PRIu64
Summary:
Builds are failing on RHEL machines because of PRIu64.

lvm/projects/compiler-rt/lib/gwp_asan/guarded_pool_allocator.cpp:420:50: error: expected ')'

`snprintf(ThreadBuffer, kThreadBufferLen, "%" PRIu64, ThreadID);`
inttypes.h in RHEL uses PRIu64 macros only when __STDC_FORMAT_MACROS is defined.

Author: DTharun

Reviewers: hctim

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D64388

llvm-svn: 365801
2019-07-11 18:07:03 +00:00
Kostya Kortchinsky aeb3826228 [scudo][standalone] Merge Spin & Blocking mutex into a Hybrid one
Summary:
We ran into a problem on Fuchsia where yielding threads would never
be deboosted, ultimately resulting in several threads spinning on the
same TSD, and no possibility for another thread to be scheduled,
dead-locking the process.

While this was fixed in Zircon, this lead to discussions about if
spinning without a break condition was a good decision, and settled on
a new hybrid model that would spin for a while then block.

Currently we are using a number of iterations for spinning that is
mostly arbitrary (based on sanitizer_common values), but this can
be tuned in the future.

Since we are touching `common.h`, we also use this change as a vehicle
for an Android optimization (the page size is fixed in Bionic, so use
a fixed value too).

Reviewers: morehouse, hctim, eugenis, dvyukov, vitalybuka

Reviewed By: hctim

Subscribers: srhines, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64358

llvm-svn: 365790
2019-07-11 15:32:26 +00:00
Petr Hosek 274ad9c371 [ASan] Use __sanitizer_fill_shadow for FastPoisonShadow on Fuchsia
This is the optimized implementation for Fuchsia provided by the libc.

Differential Revision: https://reviews.llvm.org/D64166

llvm-svn: 365739
2019-07-11 07:30:11 +00:00
Kamil Rytarowski 983d7ddd0b Add NetBSD LSan support
Summary:
Combine few relatively small changes into one:

 - implement internal_ptrace() and internal_clone() for NetBSD
 - add support for stoptheworld based on the ptrace(2) API
 - define COMPILER_RT_HAS_LSAN for NetBSD
 - enable tests for NetBSD/amd64

Inspired by the original implementation by Christos Zoulas in netbsd/src for GCC.

The implementation is in theory CPU independent through well defined macros
across all NetBSD ports, however only the x86_64 version was tested.

Reviewers: mgorny, dvyukov, vitalybuka, joerg, jfb

Reviewed By: vitalybuka

Subscribers: dexonsmith, jfb, srhines, kubamracek, llvm-commits, christos

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64057

llvm-svn: 365735
2019-07-11 06:22:35 +00:00
Julian Lettner 6a155f52bc [TSan] Fix linker error on Linux/AArch64
llvm-svn: 365707
2019-07-10 22:50:07 +00:00
Julian Lettner 098e0a79fa [TSan] Fix compiler error on Linux/AArch64
llvm-svn: 365693
2019-07-10 19:46:08 +00:00
Julian Lettner 1a2c880924 [TSan] Attempt to fix linker error on Linux/AArch64
llvm-svn: 365681
2019-07-10 18:37:57 +00:00
Julian Lettner ba9eb48318 Reland "[TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.8"
Fix compilation errors related to `SANITIZER_GO` `#ifdef`s.

Refine longjmp key management.  For Linux, re-implement key retrieval in
C (instead of assembly).  Removal of `InitializeGuardPtr` and a final
round of cleanups will be done in the next commit.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D64092

> llvm-svn: 365513

llvm-svn: 365560
2019-07-09 21:27:37 +00:00
Julian Lettner c11c43b64f [TSan] Refine longjmp key management on Darwin
NFC.

llvm-svn: 365554
2019-07-09 20:47:37 +00:00
Peter Collingbourne 1366262b74 hwasan: Improve precision of checks using short granule tags.
A short granule is a granule of size between 1 and `TG-1` bytes. The size
of a short granule is stored at the location in shadow memory where the
granule's tag is normally stored, while the granule's actual tag is stored
in the last byte of the granule. This means that in order to verify that a
pointer tag matches a memory tag, HWASAN must check for two possibilities:

* the pointer tag is equal to the memory tag in shadow memory, or
* the shadow memory tag is actually a short granule size, the value being loaded
  is in bounds of the granule and the pointer tag is equal to the last byte of
  the granule.

Pointer tags between 1 to `TG-1` are possible and are as likely as any other
tag. This means that these tags in memory have two interpretations: the full
tag interpretation (where the pointer tag is between 1 and `TG-1` and the
last byte of the granule is ordinary data) and the short tag interpretation
(where the pointer tag is stored in the granule).

When HWASAN detects an error near a memory tag between 1 and `TG-1`, it
will show both the memory tag and the last byte of the granule. Currently,
it is up to the user to disambiguate the two possibilities.

Because this functionality obsoletes the right aligned heap feature of
the HWASAN memory allocator (and because we can no longer easily test
it), the feature is removed.

Also update the documentation to cover both short granule tags and
outlined checks.

Differential Revision: https://reviews.llvm.org/D63908

llvm-svn: 365551
2019-07-09 20:22:36 +00:00
Max Moroz 2d73994046 [libFuzzer] Include FuzzedDataProvider.h in the test without "utils" subdir.
Summary:
This way the test would better match the intended usage of the header,
plus it makes some additional testing (e.g. in CI) a bit easier to set up.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64440

llvm-svn: 365544
2019-07-09 19:28:14 +00:00
Julian Lettner 95176d72c7 Revert "[TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.8"
This reverts commit 521f77e635.

llvm-svn: 365534
2019-07-09 18:42:01 +00:00
Julian Lettner 521f77e635 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.8
Refine longjmp key management.  For Linux, re-implement key retrieval in
C (instead of assembly).  Removal of `InitializeGuardPtr` and a final
round of cleanups will be done in the next commit.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D64092

llvm-svn: 365513
2019-07-09 16:49:43 +00:00
Matthew G McGovern 4e636156ef [sanitizers][windows] FIX: Rtl-Heap Interception and tests
- Adds interceptors for Rtl[Allocate|Free|Size|ReAllocate]Heap
   - Adds unit tests for the new interceptors and expands HeapAlloc
     tests to demonstrate new functionality.
   Reviewed as D62927
   - adds fixes for ~win and x64 tests

> llvm-svn: 365381

llvm-svn: 365424
2019-07-09 01:55:11 +00:00
Matthew G McGovern 848a19e4eb [sanitizers][windows] Rtl-Heap Interception and tests
- Adds interceptors for Rtl[Allocate|Free|Size|ReAllocate]Heap
   - Adds unit tests for the new interceptors and expands HeapAlloc
     tests to demonstrate new functionality.
   Reviewed as D62927

llvm-svn: 365422
2019-07-09 01:47:08 +00:00
Julian Lettner c6fe02b60b [TSan] Fix linker error for Linux/AArch64
llvm-svn: 365415
2019-07-09 00:48:38 +00:00
Rainer Orth 40f159cc40 [Sanitizers] Remove clang_rt.sancov_{begin,end} on Solaris
There's no point to manually create the __start___sancov_guards and __stop___sancov_guards
sections and labels on Solaris any longer.  They were originally introduced in
https://reviews.llvm.org/D40899 and https://reviews.llvm.org/D40903.

- The Solaris 11.4 ld supports creating them out of the box.
- We already unconditionally use Solaris 11.4 features like the ld -z gnu-version-script-compat option and fully working .preinit_array support in compiler-rt.
- The current files don't assemble on SPARC because the assembler syntax may be different between /bin/as and GNU as.

Tested on amd64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D63601

llvm-svn: 365399
2019-07-08 22:08:33 +00:00
JF Bastien c5630ac641 Revert "[sanitizers][windows] Rtl-Heap Interception and tests"
Causes build failure on clang-ppc64be-linux-lnt:

compiler-rt/lib/asan/asan_malloc_win.cc:23:2: error: #error "Missing arch or unsupported platform for Windows."
 #error "Missing arch or unsupported platform for Windows."
  ^~~~~
compiler-rt/lib/asan/asan_malloc_win.cc:25:10: fatal error: heapapi.h: No such file or directory
 #include <heapapi.h>
          ^~~~~~~~~~~
compilation terminated.
[39/1151] Building CXX object projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_debugging.cc.o
[40/1151] Building CXX object projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o
FAILED: projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o
/usr/bin/c++  -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iprojects/compiler-rt/lib/asan -Icompiler-rt/lib/asan -Iinclude -I/home/buildbots/ppc64be-clang-lnt-test/clang-ppc64be-lnt/llvm/include -Icompiler-rt/lib/asan/.. -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -Wall -std=c++11 -Wno-unused-parameter -O2    -UNDEBUG  -m64 -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fvisibility=hidden -fno-lto -O3 -g -Wno-variadic-macros -Wno-non-virtual-dtor -fno-rtti -MD -MT projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o -MF projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o.d -o projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o -c compiler-rt/lib/asan/asan_malloc_win.cc
compiler-rt/lib/asan/asan_malloc_win.cc:23:2: error: #error "Missing arch or unsupported platform for Windows."
 #error "Missing arch or unsupported platform for Windows."
  ^~~~~
compiler-rt/lib/asan/asan_malloc_win.cc:25:10: fatal error: heapapi.h: No such file or directory
 #include <heapapi.h>
          ^~~~~~~~~~~

llvm-svn: 365384
2019-07-08 20:21:09 +00:00
Matthew G McGovern c9fa99d066 [sanitizers][windows] Rtl-Heap Interception and tests
- Adds interceptors for Rtl[Allocate|Free|Size|ReAllocate]Heap
   - Adds unit tests for the new interceptors and expands HeapAlloc
     tests to demonstrate new functionality.
   Reviewed as D62927

llvm-svn: 365381
2019-07-08 19:58:50 +00:00
Julian Lettner fa0d1d8e45 Revert "[TSan] Attempt to fix linker error for Linux on AArch64"
This reverts commit be4148062b.

llvm-svn: 365367
2019-07-08 18:37:36 +00:00
Julian Lettner 2f533bdd2a Revert "[TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.4"
This reverts commit 6bb13da7c1.

llvm-svn: 365244
2019-07-05 21:03:46 +00:00
Julian Lettner 515fdfd512 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.5
Remove unnecessary computation of mangled SP for ARM64 architecture.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D64022

llvm-svn: 365230
2019-07-05 18:00:16 +00:00
Julian Lettner 6bb13da7c1 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.4
For x86_64, since we don't clobber %rsi (2nd argument) anymore, we don't
have to save/restore it.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D63946

llvm-svn: 365229
2019-07-05 18:00:12 +00:00
Kostya Kortchinsky f50a953ea7 [scudo][standalone] Link tests against libatomic
Summary:
Some clang versions (< 6.0) do not inline the atomic builtin functions
leaving unresolved references to `__atomic_load_8` and so on (seems to
be mostly 64-bit atomics on 32-bit platforms).
I tried without success to use some cmake magic to detect when that
would be the case, and decided to fall back to unconditionally
linking libatomic.

Reviewers: morehouse, eugenis, vitalybuka, hctim, tejohnson

Reviewed By: tejohnson

Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64134

llvm-svn: 365052
2019-07-03 16:32:01 +00:00
Kostya Kortchinsky d63d14f76b [scudo][standalone] Potential fix for missing sized delete
Summary:
In some setups, using `-fsized-deallocation` would end up not finding
a sized delete operator at link time. For now, avoid using the flag
and declare the sized delete operator in the cpp test only.

This is a tentative fix as I do not have the failing setup.

Reviewers: rnk, morehouse, hctim, eugenis, vitalybuka

Reviewed By: rnk, hctim

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64086

llvm-svn: 365045
2019-07-03 14:38:47 +00:00
Mitch Phillips aa141b6477 Attempt #2 to fix gcc builds. This time check
against CXX compiler ID instead of CRT test ID.

llvm-svn: 364975
2019-07-02 22:36:45 +00:00
Mitch Phillips 3f9628b3fd Only use -mno-omit-leaf-frame-pointer with clang builds.
Fix build breakage caused by D64085

llvm-svn: 364972
2019-07-02 21:58:19 +00:00
Mitch Phillips d4acc4720e [GWP-ASan] [Scudo] Add GWP-ASan backtrace for alloc/free to Scudo.
Summary:
Adds allocation and deallocation stack trace support to Scudo. The
default provided backtrace library for GWP-ASan is supplied by the libc
unwinder, and is suitable for production variants of Scudo. If Scudo in future
has its own unwinder, it may choose to use its own over the generic unwinder
instead.

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: kubamracek, mgorny, #sanitizers, llvm-commits, morehouse, vlad.tsyrklevich, eugenis

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D64085

llvm-svn: 364966
2019-07-02 20:33:19 +00:00
Julian Lettner 36face4c1d [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.7
Factor out `ExtractLongJmpSp` helper function and move platform-specific
code to tsan_platform_{linux,mac}.cc.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D64050

llvm-svn: 364947
2019-07-02 17:32:04 +00:00
Mitch Phillips 7339ca278c [GWP-ASan] Add generic unwinders and structure backtrace output.
Summary:
Adds two flavours of generic unwinder and all the supporting cruft. If the
supporting allocator is okay with bringing in sanitizer_common, they can use
the fast frame-pointer based unwinder from sanitizer_common. Otherwise, we also
provide the backtrace() libc-based unwinder as well. Of course, the allocator
can always specify its own unwinder and unwinder-symbolizer.

The slightly changed output format is exemplified in the first comment on this
patch. It now better incorporates backtrace information, and displays
allocation details on the second line.

Reviewers: eugenis, vlad.tsyrklevich

Reviewed By: eugenis, vlad.tsyrklevich

Subscribers: srhines, kubamracek, mgorny, cryptoad, #sanitizers, llvm-commits, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63841

llvm-svn: 364941
2019-07-02 16:04:52 +00:00
Rainer Orth 8d568d044c [Sanitizers] Remove obsolete OpenFile from sanitizer_solaris.cc
I noticed that the instance of OpenFile in sanitizer_solaris.cc is no
longer needed.

Removed as follows, tested on x86_64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D63764

llvm-svn: 364892
2019-07-02 08:40:29 +00:00
Reid Kleckner adeab8d754 Revert Remove scudo standalone tests from check-all
This reverts r364877 (git commit dfae3705b7)

This didn't solve my problem so I've reverted it.

llvm-svn: 364878
2019-07-01 23:53:16 +00:00
Reid Kleckner dfae3705b7 Remove scudo standalone tests from check-all
They appear to fail to link in various 32-bit configurations for unknown
reasons. This change was already reverted, and it seems preferable to me
to make forward progress and remove this once the problems are fully
understood.

llvm-svn: 364877
2019-07-01 23:29:59 +00:00
Julian Lettner d66c606a34 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.3
Remove unnecessary computation of mangled SP for x86_64 architecture.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D63944

llvm-svn: 364874
2019-07-01 23:23:18 +00:00
Julian Lettner b7fb723ea3 [TSan] Fix initialized and declared ‘extern’ warning
Avoid the following warning which fails a bot (via -Werror).

```
/tmp/gotsan.JfrpVPu7pG/gotsan.cc:10456:25: error: ‘_tsan_pointer_chk_guard’ initialized and declared ‘extern’ [-Werror]
extern "C" __tsan::uptr _tsan_pointer_chk_guard = 0;
                        ^~~~~~~~~~~~~~~~~~~~~~~
```

llvm-svn: 364823
2019-07-01 18:06:31 +00:00
Julian Lettner e934586680 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.2
Switch `LongJmp` over to lookup JmpBuf via plain old (unmangled) SP.
This makes the computation of mangled SPs in the TSan assembly files
unnecessary, which will be cleaned up in follow-up revisions.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D63942

llvm-svn: 364818
2019-07-01 17:39:39 +00:00
Julian Lettner be4148062b [TSan] Attempt to fix linker error for Linux on AArch64
Introduced in 5be69ebe12.

llvm-svn: 364700
2019-06-29 00:40:45 +00:00
Julian Lettner 1c6337ca5a [TSan] Fix build build breakage on Android
Introduced in 5be69ebe12.

llvm-svn: 364676
2019-06-28 19:49:14 +00:00
Julian Lettner 5be69ebe12 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.1
TSan needs to infer which calls to setjmp/longjmp are corresponding
pairs. My understanding is, that we can't simply use the jmp_buf
address, since this buffer is just a plain data structure storing the
environment (registers) with no additional semantics, i.e., it can be
copied around and is still expected to work. So we use the stack pointer
(SP) instead.

The setjmp interceptor stores some metadata, which is then consumed in
the corresponding call to longjmp. We use the SP as an "index" (stable
identifier) into the metadata table. So far so good.

However, when mangling is used, the setjmp interceptor observes the
UNmangled SP, but the longjmp interceptor only knows the mangled value
for SP. To still correlate corresponding pairs of calls, TSan currently
derives the mangled representation in setjmp and uses it as the stable
identifer, so that longjmp can do it's lookup.

Currently, this works since "mangling" simply means XOR with a secret
value. However, in the future we want to use operations that do not
allow us to easily go from unmangled -> mangled (pointer
authentication). Going from mangled -> unmangled should still be
possible (for pointer authentication it means zeroing a few bits).

This patch is part 1 of changing set/longjmp interceptors to use the
unmangled SP for metadata lookup. Instead of deriving the mangled SP in
setjmp, we will derive the unmangled SP in longjmp. Since this change
involves difficult-to-test code, it will be done in (at least) 2 parts:
This patch only replicates the existing behavior and checks that the
newly computed value for SP matches with what we have been doing so far.
This should help me to fix issues on architectures I cannot test
directly. I tested this patch on x86-64 (Linux/Darwin) and arm64
(Darwin).

This patch will also address an orthogonal issue: there is a lot of code
duplication in the assembly files, because the
`void __tsan_setjmp(uptr sp, uptr mangled_sp)` already demands the
mangled SP. This means that the code for computing the mangled SP is
duplicated at every call site (in assembly).

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D60981

llvm-svn: 364662
2019-06-28 17:27:30 +00:00
Peter Collingbourne 559b2e0187 hwasan: Fix an off-by-one error in PrintTagsAroundAddr.
Previously we were printing 16 rows of tags, not 17.

Differential Revision: https://reviews.llvm.org/D63906

llvm-svn: 364609
2019-06-27 23:24:36 +00:00
Peter Collingbourne d11ea6518c hwasan: Teach the runtime to identify the local variable being accessed in UAR reports.
Each function's PC is recorded in the ring buffer. From there we can access
the function's local variables and reconstruct the tag of each one with the
help of the information printed by llvm-symbolizer's new FRAME command. We
can then find the variable that was likely being accessed by matching the
pointer's tag against the reconstructed tag.

Differential Revision: https://reviews.llvm.org/D63469

llvm-svn: 364607
2019-06-27 23:16:13 +00:00
Petr Hosek d9a59aeb04 [libFuzzer] Migrate to the new exception syscalls on Fuchsia
This is part of the transition to the new Fuchsia exception syscalls
signature.

Differential Revision: https://reviews.llvm.org/D63897

llvm-svn: 364594
2019-06-27 21:13:06 +00:00
Petr Hosek 379a9f5e24 [sanitizer_common] Switch from zx_clock_get_new to zx_clock_get
This is part of the soft-transition to the new system call name.
These two system calls are the same so this change is no-op.

Differential Revision: https://reviews.llvm.org/D63895

llvm-svn: 364593
2019-06-27 21:13:05 +00:00
Kostya Kortchinsky d44cb7a656 [scudo][standalone] Introduce the C & C++ wrappers [fixed]
Summary:
This is a redo of D63612.

Two problems came up on some bots:
- `__builtin_umull_overflow` was not declared. This is likely due to an
  older clang or gcc, so add a guard with `__has_builtin` and fallback
  to a division in the event the builtin doesn't exist;
- contradicting definition for `malloc`, etc. This is AFAIU due to the
  fact that we ended up transitively including `stdlib.h` in the `.inc`
  due to it being the flags parser header: so move the include to the
  cc instead.

This should fix the issues, but since those didn't come up in my local
tests it's mostly guesswork.

Rest is the same!

Reviewers: morehouse, hctim, eugenis, vitalybuka, dyung, hans

Reviewed By: morehouse, dyung, hans

Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63831

llvm-svn: 364547
2019-06-27 14:23:26 +00:00
Hans Wennborg 1a0810407e Revert r364332 "[scudo][standalone] Introduce the C & C++ wrappers"
Makes the build fail with e.g.

  llvm/projects/compiler-rt/lib/scudo/standalone/wrappers_c.inc:20:68: error:
  declaration of 'void* calloc(size_t, size_t)' has a different exception
  specifier
   INTERFACE WEAK void *SCUDO_PREFIX(calloc)(size_t nmemb, size_t size) {
                                                                      ^

See llvm-commits thread.

> Summary:
> This CL adds C & C++ wrappers and associated tests. Those use default
> configurations for a Scudo combined allocator that will likely be
> tweaked in the future.
>
> This is the final CL required to have a functional C & C++ allocator
> based on Scudo.
>
> The structure I have chosen is to define the core C allocation
> primitives in an `.inc` file that can be customized through defines.
> This allows to easily have 2 (or more) sets of wrappers backed by
> different combined allocators, as demonstrated by the `Bionic`
> wrappers: one set for the "default" allocator, one set for the "svelte"
> allocator.
>
> Currently all the tests added have been gtests, but I am planning to
> add some more lit tests as well.
>
> Reviewers: morehouse, eugenis, vitalybuka, hctim, rengolin
>
> Reviewed By: morehouse
>
> Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits
>
> Tags: #llvm, #sanitizers
>
> Differential Revision: https://reviews.llvm.org/D63612

llvm-svn: 364400
2019-06-26 09:46:26 +00:00
Mitch Phillips 05d1a2bd1b [GWP-ASan] Guard against recursive allocs. Pack TLS for perf.
Summary:
Add a recursivity guard for GPA::allocate(). This means that any
recursive allocations will fall back to the supporting allocator. In future
patches, we will introduce stack trace collection support. The unwinder will be
provided by the supporting allocator, and we can't guarantee they don't call
malloc() (e.g. backtrace() on posix may call dlopen(), which may call malloc().

Furthermore, this patch packs the new TLS recursivity guard into a thread local
struct, so that TLS variables should be hopefully not fall across cache lines.

Reviewers: vlad.tsyrklevich, morehouse, eugenis

Reviewed By: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits, eugenis

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63736

llvm-svn: 364356
2019-06-25 22:29:05 +00:00
Kostya Kortchinsky 3ad32a037e [scudo] Correct a behavior on the shared TSD registry
Summary:
There is an error in the shared TSD registry logic when looking for a
TSD in the slow path. There is an unlikely event when a TSD's precedence
was 0 after attempting a `tryLock` which indicated that it was grabbed
by another thread in between. We dealt with that case by continuing to
the next iteration, but that meant that the `Index` was not increased
and we ended up trying to lock the same TSD.
This would manifest in heavy contention, and in the end we would still
lock a TSD, but that was a wasted iteration.
So, do not `continue`, just skip the TSD as a potential candidate.

This is in both the standalone & non-standalone versions.

Reviewers: morehouse, eugenis, vitalybuka, hctim

Reviewed By: morehouse

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63783

llvm-svn: 364345
2019-06-25 19:58:11 +00:00
Kostya Kortchinsky 37340e3cd6 [scudo][standalone] Introduce the C & C++ wrappers
Summary:
This CL adds C & C++ wrappers and associated tests. Those use default
configurations for a Scudo combined allocator that will likely be
tweaked in the future.

This is the final CL required to have a functional C & C++ allocator
based on Scudo.

The structure I have chosen is to define the core C allocation
primitives in an `.inc` file that can be customized through defines.
This allows to easily have 2 (or more) sets of wrappers backed by
different combined allocators, as demonstrated by the `Bionic`
wrappers: one set for the "default" allocator, one set for the "svelte"
allocator.

Currently all the tests added have been gtests, but I am planning to
add some more lit tests as well.

Reviewers: morehouse, eugenis, vitalybuka, hctim, rengolin

Reviewed By: morehouse

Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63612

llvm-svn: 364332
2019-06-25 16:51:27 +00:00
Sajjad Mirza 6694b2b36b (Reland with changes) Adding a function for setting coverage output file.
Summary:
User code can open a file on its own and pass it to the runtime, rather than
specifying a name and having the runtime open the file. This supports the use
case where a process cannot open a file on its own but can receive a file
descriptor from another process.

Relanding https://reviews.llvm.org/D62541. The original revision unlocked
the file before calling flush, this revision fixes that.

Reviewers: Dor1s, davidxl

Reviewed By: Dor1s

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63581

llvm-svn: 364231
2019-06-24 21:32:50 +00:00
Julian Lettner 19c4d660f4 [ASan] Use dynamic shadow on 32-bit iOS and simulators
The VM layout on iOS is not stable between releases. On 64-bit iOS and
its derivatives we use a dynamic shadow offset that enables ASan to
search for a valid location for the shadow heap on process launch rather
than hardcode it.

This commit extends that approach for 32-bit iOS plus derivatives and
their simulators.

rdar://50645192
rdar://51200372
rdar://51767702

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D63586

llvm-svn: 364105
2019-06-21 21:01:39 +00:00
Reid Kleckner c90de6375e [asan] Avoid two compiler-synthesized calls to memset & memcpy
Otherwise the tests hang on Windows attempting to report nested errors.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D63627

llvm-svn: 364070
2019-06-21 16:15:09 +00:00
Kostya Serebryany 679669a77e [libFuzzer] split DataFlow.cpp into two .cpp files, one of which can be compiled w/o dfsan to speed things up (~25% speedup)
llvm-svn: 364002
2019-06-21 01:39:35 +00:00
Kostya Serebryany 27cf743bff [libFuzzer] ensure that DFT and autofocus works for C++ (mangled) functions
llvm-svn: 363905
2019-06-20 01:48:45 +00:00
Max Moroz 46972b065c [libFuzzer] Remove too aggressive static_assert in FuzzedDataProvider.
Summary:
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/31

error: static_assert failed due to requirement
'std::numeric_limits<char>::is_signed' "Destination type must be
signed."
    static_assert(std::numeric_limits<TS>::is_signed,
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/buildslave/buildslave/clang-cmake-aarch64-full/llvm/projects/compiler-rt/lib/fuzzer/utils/FuzzedDataProvider.h:126:19:
note: in instantiation of function template specialization
'FuzzedDataProvider::ConvertUnsignedToSigned<char, unsigned char>'
requested here
      char next = ConvertUnsignedToSigned<char>(data_ptr_[0]);
                  ^
1 error generated.

Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: javed.absar, kristof.beyls, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63553

llvm-svn: 363805
2019-06-19 14:14:27 +00:00
Rainer Orth cd31e7868b [Sanitizers] Fix sanitizer_posix_libcdep.cc compilation on Solaris 11.5
A recent build of Solaris 11.5 Beta (st_047) gained madvise(MADV_DONTDUMP)
support for Linux compatibility.  This broke the compiler-rt build:

  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_comm/sanitizer_posix_libcdep.cc: In function ‘bool __sanitizer::DontDumpShadowMemory(__sanitizer::uptr, __sanitizer::uptr)’:
  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cc:81:18: error: invalid conversion from ‘void*’ to ‘caddr_t’ {aka ‘char*’} [-fpermissive]
     81 |   return madvise((void *)addr, length, MADV_DONTDUMP) == 0;
        |                  ^~~~~~~~~~~~
        |                  |
        |                  void*
  In file included from
/vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cc:32:
  /usr/include/sys/mman.h:231:20: note: initializing argument 1 of ‘int
madvise(caddr_t, std::size_t, int)’
    231 | extern int madvise(caddr_t, size_t, int);
        |                    ^~~~~~~

The obvious fix is to use the same solution that has already been used a
couple of lines earlier:

  // In the default Solaris compilation environment, madvise() is declared
  // to take a caddr_t arg; casting it to void * results in an invalid
  // conversion error, so use char * instead.

This allowed the compiler-rt build to finish and was tested successfully on 
i386-pc-solaris2.11 and x86_64-pc-linux-gnu.

Differential Revision: https://reviews.llvm.org/D62892

llvm-svn: 363778
2019-06-19 08:59:05 +00:00
Evgeniy Stepanov c6b5be6cf0 Don't crash if PR_SET_VMA_ANON_NAME fails.
This prctl is not implemented on very old devices.
It is not necessary for the core functionality of the tool. Simply
ignore the failure.

llvm-svn: 363755
2019-06-18 23:50:43 +00:00
Max Moroz a0eb49c26e [libFuzzer] Improve FuzzedDataProvider helper.
Summary:
The following changes are made based on the feedback from Tim King:
- Removed default template parameters, to have less assumptions.
- Implemented `ConsumeBytesWithTerminator` method.
- Made `PickValueInArray` method work with `initializer_list` argument.
- Got rid of `data_type` type alias, that was redundant.
- Refactored `ConsumeBytes` logic into a private method for better code reuse.
- Replaced implementation defined unsigned to signed conversion.
- Fixed `ConsumeRandomLengthString` to always call `shrink_to_fit`.
- Clarified and fixed some commments.
- Applied clang-format to both the library and the unittest source.

Tested on Linux, Mac, Windows.

Reviewers: morehouse, metzman

Reviewed By: morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63348

llvm-svn: 363735
2019-06-18 20:29:11 +00:00
Kostya Kortchinsky 5cf216c9a7 [scudo][standalone] Fuchsia related changes
Summary:
Fuchsia wants to use mutexes with PI in the Scudo code, as opposed to
our own implementation. This required making `lock` & `unlock` platform
specific (as opposed to `wait` & `wake`) [code courtesy of John
Grossman].
There is an additional flag required now for mappings as well:
`ZX_VM_ALLOW_FAULTS`.

Reviewers: morehouse, mcgrathr, eugenis, vitalybuka, hctim

Reviewed By: morehouse

Subscribers: delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63435

llvm-svn: 363705
2019-06-18 17:41:17 +00:00
Dan Liew 3c9f66dccf [asan_symbolize] Teach `asan_symbolize.py` to symbolicate partially symbolicated ASan reports.
Summary:
The use case here is to be able symbolicate ASan reports that might be
partially symbolicated, in particular where the function name is known but no source
location is available. This can be caused by missing debug info. Previously we
would only try to symbolicate completely unsymbolicated reports.

The code currently contains an unfortunate quirk to handle a darwin
specific bug (rdar://problem/49784442) in the way partially symbolicated
reports are emitted when the source location is missing.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: aprantl, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60533

llvm-svn: 363639
2019-06-18 01:21:16 +00:00
Peter Collingbourne d57f7cc15e hwasan: Use bits [3..11) of the ring buffer entry address as the base stack tag.
This saves roughly 32 bytes of instructions per function with stack objects
and causes us to preserve enough information that we can recover the original
tags of all stack variables.

Now that stack tags are deterministic, we no longer need to pass
-hwasan-generate-tags-with-calls during check-hwasan. This also means that
the new stack tag generation mechanism is exercised by check-hwasan.

Differential Revision: https://reviews.llvm.org/D63360

llvm-svn: 363636
2019-06-17 23:39:51 +00:00
Mitch Phillips 8fbb88fbff [GWP-ASan] Disable GWP-ASan on Android for now.
Summary:
Temporarily disable GWP-ASan for android until the bugs at:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-android/builds/87
... can be fixed. See comments for the full bug trace.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: srhines, kubamracek, mgorny, cryptoad, jfb, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63460

llvm-svn: 363624
2019-06-17 22:35:32 +00:00
Evgeniy Stepanov 17bd226b6a Stop counting pops in tsan/check_analyze.sh.
Summary:
It looks like LLVM has started doing less tail duplication in this code,
or something like that, resulting in a significantly smaller number of
pop instructions (16 -> 12). Removing the check.

Reviewers: vitalybuka, dvyukov

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63450

llvm-svn: 363615
2019-06-17 20:47:16 +00:00
Mitch Phillips 79bc188dc2 Attempt to fix GWP-ASan build failure on sanitizer-android. Add -fPIC.
llvm-svn: 363604
2019-06-17 19:39:29 +00:00
Mitch Phillips 21184ec5c4 [GWP-ASan] Integration with Scudo [5].
Summary:
See D60593 for further information.

This patch adds GWP-ASan support to the Scudo hardened allocator. It also
implements end-to-end integration tests using Scudo as the backing allocator.
The tests include crash handling for buffer over/underflow as well as
use-after-free detection.

Reviewers: vlad.tsyrklevich, cryptoad

Reviewed By: vlad.tsyrklevich, cryptoad

Subscribers: kubamracek, mgorny, #sanitizers, llvm-commits, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62929

llvm-svn: 363584
2019-06-17 17:45:34 +00:00
Kostya Kortchinsky e4eadf174c [scudo][standalone] Introduce the combined allocator
Summary:
The Combined allocator hold together all the other components, and
provides a memory allocator interface based on various template
parameters. This will be in turn used by "wrappers" that will provide
the standard C and C++ memory allocation functions, but can be
used as is as well.

This doesn't depart significantly from the current Scudo implementation
except for a few details:
- Quarantine batches are now protected by a header a well;
- an Allocator instance has its own TSD registry, as opposed to a
  static one for everybody;
- a function to iterate over busy chunks has been added, for Android
  purposes;

This also adds the associated tests, and a few default configurations
for several platforms, that will likely be further tuned later on.

Reviewers: morehouse, hctim, eugenis, vitalybuka

Reviewed By: morehouse

Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63231

llvm-svn: 363569
2019-06-17 15:23:11 +00:00
Shoaib Meenai 744870f469 [compiler-rt] Respect CMAKE_NM
The default nm executable may not be able to handle the architecture
we're building the sanitizers for. Respect CMAKE_NM if it's set to
ensure we're using the correct nm tool. Preserve the existing NM
environment variable override to not break its users.

Differential Revision: https://reviews.llvm.org/D63368

llvm-svn: 363483
2019-06-15 04:40:16 +00:00
Vitaly Buka 0b1ea8cb28 Improve error message when '=' is missing in {ASAN,...}_OPTIONS.
Summary:
It's handling isses as described here:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89832

Patch by Martin Liška.

Reviewers: kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: cryptoad, kubamracek

Differential Revision: https://reviews.llvm.org/D59876

llvm-svn: 363480
2019-06-15 01:37:14 +00:00
Kostya Serebryany 0feed5d585 [libFuzzer] in autofocus mode, give more weight to functions with DFT
llvm-svn: 363473
2019-06-14 23:29:56 +00:00
Kostya Serebryany db88fc56b9 [libFuzzer] implement a better queue for the fork mode. Add an internal flag -stop_file to allow graceful shutdown of fuzzing. Enhance the logging in the fork mode
llvm-svn: 363470
2019-06-14 22:56:50 +00:00
Kostya Serebryany 2fa6838e5f [libFuzzer] fix -Werror build
llvm-svn: 363469
2019-06-14 22:34:30 +00:00
Kostya Serebryany 3f39123d15 [libFuzzer] simplify the DFT trace collection using the new faster DFSan mode that traces up to 16 labels at a time and never runs out of labels. Second attempt. This time with a fix for windows (putenv instead of setenv))
llvm-svn: 363445
2019-06-14 19:54:32 +00:00
Max Moroz 0784e01a98 [libFuzzer] Disable len_control by default if LLVMFuzzerCustomMutator is used.
Summary:
Some custom mutators may not peform well when size restriction is
enforced by len_control. Because of that, it's safer to disable len_control
by default in such cases, but still allow users to enable it manually.
Bug example: https://bugs.chromium.org/p/chromium/issues/detail?id=919530.

Tested manually with LPM-based and regular fuzz targets.

Reviewers: kcc, vitalybuka, metzman

Reviewed By: kcc, metzman

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63334

llvm-svn: 363443
2019-06-14 19:34:11 +00:00
Hans Wennborg 9bc707c0e5 Revert r363326 "[libFuzzer] simplify the DFT trace collection using the new faster DFSan mode that traces up to 16 labels at a time and never runs out of labels."
It broke the Windows build:

C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\lib\fuzzer\FuzzerDataFlowTrace.cpp(243): error C3861: 'setenv': identifier not found

This also reverts the follow-up r363327.

llvm-svn: 363358
2019-06-14 07:32:22 +00:00
Kostya Serebryany 2fa83cb7ee [libFuzzer] simplify the DFT trace collection using the new faster DFSan mode that traces up to 16 labels at a time and never runs out of labels.
llvm-svn: 363326
2019-06-13 21:17:49 +00:00
Kostya Serebryany 6b936d88a4 [dfsan] Introduce dfsan_flush().
Summary:
dfsan_flush() allows to restart tain tracking from scratch in the same process.
The primary purpose right now is to allow more efficient data flow tracing
for DFT fuzzing: https://github.com/google/oss-fuzz/issues/1632

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63037

llvm-svn: 363321
2019-06-13 20:11:06 +00:00
Hans Wennborg 05d44139ee Revert r362676 "[Profile]: Add runtime interface to specify file handle for profile data."
This caused instrumented Clang to become crashy. See llvm-commits thread
for repro steps.

This also reverts follow-up r362716 which added test cases.

> Author: Sajjad Mirza
>
> Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 363134
2019-06-12 08:44:32 +00:00
Julian Lettner 6d9ce4e5a2 [Sanitizers] Add case MACOS_VERSION_CATALINA
Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D63096

llvm-svn: 363104
2019-06-11 21:54:15 +00:00
Kostya Kortchinsky 624a24e156 [scudo][standalone] Unmap memory in tests
Summary:
The more tests are added, the more we are limited by the size of the
address space on 32-bit. Implement `unmapTestOnly` all around (like it
is in sanitzer_common) to be able to free up some memory.
This is not intended to be a proper "destructor" for an allocator, but
allows us to not fail due to having no memory left.

Reviewers: morehouse, vitalybuka, eugenis, hctim

Reviewed By: morehouse

Subscribers: delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63146

llvm-svn: 363095
2019-06-11 19:50:12 +00:00
Max Moroz 10ed68189a Add FuzzedDataProvider helper class / single header library.
Summary:
This class is useful for writing fuzz target that have multiple inputs.

Current CL imports the existing `FuzzedDataProvider` from Chromium
without any modifications. Feel free to review it thoroughly, if you're
interested, but I'd prefer changing the class in a follow up CL.

The CL also introduces an exhaustive test for the library, as the behavior
of `FuzzedDataProvider` must not change over time.

In follow up CLs I'm planning on changing some implementation details
(I can share a doc with some comments to be addressed). After that, we
will document how `FuzzedDataProvider` should be used.

I have tested this on Linux, Windows and Mac platforms.

Reviewers: morehouse, metzman, kcc

Reviewed By: morehouse

Subscribers: metzman, thakis, rnk, mgorny, ormris, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D62733

llvm-svn: 363071
2019-06-11 14:30:18 +00:00
Reid Kleckner a5f2c20815 Add unused symbol to thunk files to force wholearchive inclusion
These "dynamic_runtime_thunk" object files exist to create a weak alias
from 'foo' to 'foo_dll' for all weak sanitizer runtime symbols. The weak
aliases are implemented as /alternatename linker options in the
.drective section, so they are not actually in the symbol table. In
order to force the Visual C++ linker to load the object, even with
-wholearchive:, we have to provide at least one external symbol. Once we
do that, it will read the .drective sections and see the weak aliases.

Fixes PR42074

llvm-svn: 362970
2019-06-10 17:50:28 +00:00
Kostya Kortchinsky 52bfd673d1 [scudo][standalone] Introduce the thread specific data structures
Summary:
This CL adds the structures dealing with thread specific data for the
allocator. This includes the thread specific data structure itself and
two registries for said structures: an exclusive one, where each thread
will have its own TSD struct, and a shared one, where a pool of TSD
structs will be shared by all threads, with dynamic reassignment at
runtime based on contention.

This departs from the current Scudo implementation: we intend to make
the Registry a template parameter of the allocator (as opposed to a
single global entity), allowing various allocators to coexist with
different TSD registry models. As a result, TSD registry and Allocator
are tightly coupled.

This also corrects a couple of things in other files that I noticed
while adding this.

Reviewers: eugenis, vitalybuka, morehouse, hctim

Reviewed By: morehouse

Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D62258

llvm-svn: 362962
2019-06-10 16:50:52 +00:00
Kostya Serebryany 300c0c79de Experimantal dfsan mode "fast16labels=1"
Summary:
dfsan mode "fast16labels=1".
In this mode the labels are treated as 16-bit bit masks.

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D62870

llvm-svn: 362859
2019-06-08 00:22:23 +00:00
Mitch Phillips e41e366ae7 Change GWP-ASan build to use '-pthread' instead of '-lpthread' in order
to try and fix android buildbot. Also make sure that the empty dummy
test contains an output file name so the android_build.py wrapper script
doesn't check fail.

llvm-svn: 362758
2019-06-06 23:43:25 +00:00
Douglas Yung dbceb9b220 Fixup files added in r362636 to build with gcc 5.4. NFCI
llvm-svn: 362682
2019-06-06 08:04:33 +00:00
Xinliang David Li c1867557d9 [Profile]: Add runtime interface to specify file handle for profile data.
Author: Sajjad Mirza

Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 362676
2019-06-06 06:35:18 +00:00
Mitch Phillips a95edb9dc1 [GWP-ASan] Core Guarded Pool Allocator [4].
Summary:
See D60593 for further information.

This patch introduces the core of GWP-ASan, being the guarded pool allocator. This class contains the logic for creating and maintaining allocations in the guarded pool. Its public interface is to be utilised by supporting allocators in order to provide sampled guarded allocation behaviour.

This patch also contains basic functionality tests of the allocator as unittests. The error-catching behaviour will be tested in upcoming patches that use Scudo as an implementing allocator.

Reviewers: vlad.tsyrklevich, eugenis, jfb

Reviewed By: vlad.tsyrklevich

Subscribers: dexonsmith, kubamracek, mgorny, cryptoad, jfb, #sanitizers, llvm-commits, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62872

llvm-svn: 362636
2019-06-05 19:42:48 +00:00
Mitch Phillips c012188ada Changed force build of GWP-ASan options parser to be statically
linked instead of dynamic. This should help resolve a downstream
build order issue against libc++.

llvm-svn: 362566
2019-06-05 01:27:39 +00:00
Mitch Phillips 2133daf232 [GWP-ASan] Configuration options [3].
Summary:
See D60593 for further information.

This patch introduces the configuration options for GWP-ASan. In general, we expect the supporting allocator to populate the options struct, and give that to GWP-ASan during initialisation. For allocators that are okay with pulling in sanitizer_common, we also provide an optional parser that populates the gwp_asan::Options struct with values provided in the GWP_ASAN_OPTIONS environment variable.

This patch contains very little logic, and all of the testable components (i.e. the optional parser's internal logic) is tested as part of the sanitizer_common testbed.

Reviewers: vlad.tsyrklevich, morehouse, jfb

Reviewed By: morehouse

Subscribers: dexonsmith, kubamracek, mgorny, #sanitizers, llvm-commits, vitalybuka

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62698

llvm-svn: 362527
2019-06-04 17:01:11 +00:00
Eugene Leviant c3c686f5f8 [HWASAN] Make new/delete weak
This allows instrumenting programs which have their own
versions of new and delete operators.

Differential revision: https://reviews.llvm.org/D62794

llvm-svn: 362478
2019-06-04 09:20:02 +00:00
Petr Hosek d2d6c17760 [builtins] Use libtool for builtins when building for Apple platform
compiler-rt already uses libtool instead of ar when building for
Apple platform, but that's not being used when builtins are being
built separately e.g. as part of the runtimes build. This change
extracts the logic setting up libtool into a separate file and uses
it from both the compiler-rt and standalone builtins build.

Differential Revision: https://reviews.llvm.org/D62820

llvm-svn: 362466
2019-06-04 02:38:15 +00:00
Saleem Abdulrasool aad5d51882 builtins: correct function name for AEABI
If `COMPILER_RT_ARMHF_TARGET` is set , the definition of the AEABI runtime
function `__aeabi_fcmpun` is misspelt: `__aeabi_fcmpum` instead of
`__aeabi_fcmpun`.

Patch by Konstantin Schwarz!

llvm-svn: 362424
2019-06-03 17:08:13 +00:00
Mitch Phillips 365e592480 Attempt to fix test failure for armv8.
Looks like armv8 can't handle a thousand threads, which GWP-ASan
requests when running a synchronised mutex test. Limiting this to 100 to
attempt to fix the build issue.

llvm-svn: 362163
2019-05-30 21:58:47 +00:00
Mitch Phillips 9bd9a03ad0 Attempt to fix 'mutex.h' not found when building mutex_posix.
llvm-svn: 362149
2019-05-30 20:48:05 +00:00
Mitch Phillips 5f0f4e3ae0 [GWP-ASan] Mutex implementation [2].
Summary:
See D60593 for further information.
This patch pulls out the mutex implementation and the required definitions file.

We implement our own mutex for GWP-ASan currently, because:

1. We must be compatible with the sum of the most restrictive elements of the supporting allocator's build system. Current targets for GWP-ASan include Scudo (on Linux and Fuchsia), and bionic (on Android).
2. Scudo specifies `-nostdlib++ -nonodefaultlibs`, meaning we can't use `std::mutex` or `mtx_t`.
3. We can't use `sanitizer_common`'s mutex, as the supporting allocators cannot afford the extra maintenance (Android, Fuchsia) and code size (Fuchsia) overheads that this would incur.

In future, we would like to implement a shared base mutex for GWP-ASan, Scudo and sanitizer_common. This will likely happen when both GWP-ASan and Scudo standalone are not in the development phase, at which point they will have stable requirements.

Reviewers: vlad.tsyrklevich, morehouse, jfb

Reviewed By: morehouse

Subscribers: dexonsmith, srhines, cfe-commits, kubamracek, mgorny, cryptoad, jfb, #sanitizers, llvm-commits, vitalybuka, eugenis

Tags: #sanitizers, #llvm, #clang

Differential Revision: https://reviews.llvm.org/D61923

llvm-svn: 362138
2019-05-30 19:45:32 +00:00
Eugene Leviant db15e68e4d [HWASAN] Remove unused code
Differential revision: https://reviews.llvm.org/D62489

llvm-svn: 361942
2019-05-29 10:13:41 +00:00
Petr Hosek 529118fc87 [builtins] Move the compare2f definition outside of the macro
This should hopefully address the error we're seeing in older versions
of Clang.

Differential Revision: https://reviews.llvm.org/D62554

llvm-svn: 361909
2019-05-29 01:51:56 +00:00
Kostya Serebryany 060f4b48d5 [libFuzzer] when using data-flow-trace (DFT) only load the DFT for the files present in the corpus
llvm-svn: 361579
2019-05-24 00:43:52 +00:00
Kostya Serebryany eac9a7830b [libFuzzer] remove the data-flow-trace (DFT) python scripts; their functionality is now part of libFuzzer proper; also write functions.txt to the disk only if this file doesn't exist yet
llvm-svn: 361452
2019-05-23 01:03:42 +00:00
Kostya Serebryany b7cc3d9953 [libFuzzer] automatically collect the data flow trace (DFT) in the fork mode if -collect_data_flow= is given
llvm-svn: 361448
2019-05-23 00:22:46 +00:00
Reid Kleckner 9e0edce564 [asan] Fix debug asan build link error
Reviewers: dmajor, vitalybuka, waltl

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61871

llvm-svn: 361427
2019-05-22 20:29:12 +00:00
Craig Topper 6dbf4a86a7 [X86] Add more icelake model numbers to compiler-rt implementation of __builtin_cpu_is.
Using model numbers found in Table 2-1 of the May 2019 version
of the Intel Software Developer's Manual Volume 4.

llvm-svn: 361423
2019-05-22 19:51:48 +00:00
Petr Hosek 9bd4dc929c [libFuzzer] Ignore synthetic exceptions on Fuchsia
Fuchsia has several exceptions which are merely informational and should
not be treated as crashes. This patch puts logic that read from the
exception port and handled the exception in a loop, and ignores
non-exceptions and informational exceptions.

Patch By: aarongreen

Differential Revision: https://reviews.llvm.org/D62226

llvm-svn: 361407
2019-05-22 16:36:35 +00:00
Pavel Labath 269340f1cf [Sanitizer] Add interceptor for wcsdup
Summary: The wide-string equivalent of strdup. Implementation trivial.

Reviewers: vitalybuka, eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D62189

llvm-svn: 361357
2019-05-22 08:34:56 +00:00
Vitaly Buka 9bd4fe80f0 [asan] clang-format parent patch
llvm-svn: 361305
2019-05-21 20:24:42 +00:00
Vitaly Buka e756730c23 [asan] Replace assignments with internal_memcpy ErrorDescription
For consistency with nearby code and to avoid interceptors during reports.

llvm-svn: 361304
2019-05-21 20:23:10 +00:00
Petr Hosek 48140db797 [builtins] Deduplicate __eqsf2 and __gtsf2 via macro
The only difference between __eqsf2 and __gtsf2 is whether they return
1 or -1 on NaN. Rather than duplicating all the code, use a macro to
define the function twice and use an argument to decide whether to
negate the return value.

Differential Revision: https://reviews.llvm.org/D61919

llvm-svn: 361207
2019-05-20 23:34:24 +00:00
Max Moroz 52fa90a348 This change adds an API to allow setting the flag to indicate that the profile data has been dumped to the file.
Summary:
The main use is for users to disable dumping profile data to the file
for certain processes in case the processes don't have permission to
write to the disks, and trying to do so would result in side effects
such as crashes.

Patch by Yuke Liao (@liaoyuke).

Additional context (Chromium use case):
- https://bugs.chromium.org/p/chromium/issues/detail?id=842424
- https://bugs.chromium.org/p/chromium/issues/detail?id=957655
- https://chromium-review.googlesource.com/c/chromium/src/+/1610093

Reviewers: Dor1s, vsk, davidxl

Reviewed By: Dor1s, davidxl

Subscribers: delcypher, davidxl, sajjadm, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62078

llvm-svn: 361194
2019-05-20 20:02:20 +00:00
Craig Topper b93f8ae7a7 [X86] Add icelake-client and tremont model numbers to compiler-rt's implementation of __builtin_cpu_is.
llvm-svn: 361175
2019-05-20 16:58:38 +00:00
Kostya Kortchinsky 52f0130216 [scudo][standalone] Introduce the Primary(s) and LocalCache
Summary:
This CL introduces the 32 & 64-bit primary allocators, and associated
Local Cache. While the general idea is mostly similar to what exists
in sanitizer_common, it departs from the original code somewhat
significantly:
- the 64-bit primary no longer uses a free array at the end of a region
  but uses batches of free blocks in region 0, allowing for a
  convergence with the 32-bit primary behavior;
- as a result, there is only one (templated) local cache type for both
  primary allocators, and memory reclaiming can be implemented similarly
  for the 32-bit & 64-bit platforms;
- 64-bit primary regions are handled a bit differently: we do not
  reserve 4TB of memory that we split, but reserve `NumClasses *
  2^RegionSizeLog`, each region being offseted by a random number of
  pages from its computed base. A side effect of this is that the 64-bit
  primary works on 32-bit platform (I don't think we want to encourage
  it but it's an interesting side effect);

Reviewers: vitalybuka, eugenis, morehouse, hctim

Reviewed By: morehouse

Subscribers: srhines, mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61745

llvm-svn: 361159
2019-05-20 14:40:04 +00:00
Petr Hosek cd78209a35 Use CMAKE_C_COMPILER_ARG1 in compiler invocation
This is needed when using compiler wrappers such as ccache or distcc
and should address the failure on clang-x86_64-debian-fast bot.

Differential Revision: https://reviews.llvm.org/D62104

llvm-svn: 361111
2019-05-19 03:29:15 +00:00
Vitaly Buka a9c7b2583f [sanitizer] Update symbolizer/scripts/global_symbols.txt
llvm-svn: 361077
2019-05-17 21:37:34 +00:00
Mitch Phillips 182c638fe0 Explicitly remove -stdlib=libc++, as we pass -nostdinc++ anyway. This should fix the android builtbots, and is a direct copy from what Scudo does.
llvm-svn: 361073
2019-05-17 20:34:37 +00:00
Mitch Phillips 364f662fa3 [GWP-ASan] Fixed issue with c++ standard library dependency.
Summary:
Removed dependency on c++ standard library. Some supporting allocators (namely Scudo on Fuchsia, and shortly, scudo standalone) has a hard requirement of no c++stdlib.

This patch updates the build system so that we don't have any c++ stdlib dependencies. It also will conveniently fix a racy build-order bug discrepency between GWP-ASan and libc++.

Reviewers: phosek, morehouse

Reviewed By: phosek, morehouse

Subscribers: kubamracek, mgorny, cryptoad, #sanitizers, llvm-commits, beanz, smeenai, vitalybuka

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62048

llvm-svn: 360982
2019-05-17 03:20:53 +00:00
Vitaly Buka 627e383ce5 [sanitizer] Don't generate "failed to intercept" constants for each function
llvm-svn: 360958
2019-05-16 23:46:15 +00:00
Kostya Serebryany 2e2dfe2186 [libFuzzer] replace string_view with string to fix the bots. This is NFC, just slower.
llvm-svn: 360717
2019-05-14 22:16:04 +00:00
Kostya Serebryany 1e8104daf2 [libFuzzer] #include <string_view>, hopefully should fix the windows build
llvm-svn: 360715
2019-05-14 22:05:41 +00:00
Kostya Serebryany 27d22b6b7a [libFuzzer] reimplement DFT's collect_data_flow inside libFuzzer so that we don't need external python scripts
llvm-svn: 360712
2019-05-14 21:47:35 +00:00
Mitch Phillips c9dd299736 [GWP-ASan] Initial build files, implementation of PRNG [1].
Summary:
See D60593 for further information.
This patch slices off the PRNG implementation and the initial build files for GWP-ASan.

Reviewers: vlad.tsyrklevich, morehouse, vitalybuka

Reviewed By: morehouse

Subscribers: srhines, kubamracek, mgorny, #sanitizers, llvm-commits, cryptoad, eugenis

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61867

llvm-svn: 360710
2019-05-14 21:43:11 +00:00
Shoaib Meenai bbdc8593e1 [compiler-rt] Fix crtbegin.c compilation
We're building with -std=c11 now (as opposed to -std=gnu11), so we can't
use GNU extensions and need to spell inline assembly as __asm__.

llvm-svn: 360503
2019-05-11 02:33:33 +00:00
Petr Hosek fc86c7fca3 [crt] Use -std=c11 for crtbegin.o/crtend.o
The source uses C11 syntax such as comments and some compilers print
warnings without specifying this flag.

Differential Revision: https://reviews.llvm.org/D61797

llvm-svn: 360459
2019-05-10 19:23:56 +00:00
Nico Weber 37dc0ced7a Fix some gcc warnings in compiler-rt
- Several "warning: extra ';' [-Wpedantic]"
- One "C++ style comments are not allowed in ISO C90 [enabled by default]"
  in a file that uses C style comments everywhere but in one place

llvm-svn: 360430
2019-05-10 14:15:13 +00:00
Kostya Serebryany 4c7353c53b [libFuzzer] code refactoring; NFC
llvm-svn: 360400
2019-05-10 01:34:26 +00:00
Kostya Serebryany da96d92175 [libFuzzer] small refactoring in the driver; dummy implementation of collect_data_flow; attempt to fix the windows bot
llvm-svn: 360399
2019-05-10 00:59:32 +00:00
Petr Hosek 4fa4df9cdc [compiler-rt] Migrate to _zx_clock_get_new/_zx_clock_get_monotonic
This is part of the soft-transition to the new _zx_clock_get signature.

Differential Revision: https://reviews.llvm.org/D61768

llvm-svn: 360394
2019-05-09 23:42:58 +00:00
Matt Morehouse 3478494c1f [libFuzzer] Unpoison parameters before calling user callback.
Summary:
Fixes an MSan false positive when compiling with
-fsanitize=memory,fuzzer.

See https://github.com/google/oss-fuzz/issues/2369 for more details.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, metzman, eugenis

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61753

llvm-svn: 360390
2019-05-09 22:48:46 +00:00
Leonard Chan 992021335c [NFC][compiler-rt][builtins] Tidy and match comments for floating point operations
Differential Revision: https://reviews.llvm.org/D61762

llvm-svn: 360389
2019-05-09 22:48:30 +00:00
Kostya Serebryany 4a5793f7d0 [libFuzzer] perform more agressive value profiling in memcmp
llvm-svn: 360385
2019-05-09 22:09:25 +00:00
Matt Morehouse a612b5adb7 [MSan] Introduce __msan_unpoison_param().
Summary:
This allows libFuzzer to unpoison parameter shadow before calling
LLVMFuzzerTestOneInput to eliminate the false positives described
in https://github.com/google/oss-fuzz/issues/2369.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, metzman, kcc

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61751

llvm-svn: 360379
2019-05-09 21:50:57 +00:00
Kostya Serebryany e9aaa5582f [libFuzzer] implement -focus_function=auto, to be used with Data Flow Traces
llvm-svn: 360378
2019-05-09 21:29:45 +00:00
Kostya Kortchinsky 6a85631d87 [scudo][standalone] Introduce the chunk header
Summary:
... and its related functions.

The structure and its functionalities are identical to existing ones.
The header stores information on a `scudo::Chunk` to be able to detect
inconsitencies or potential corruption attempts. It is checksummed for
that purpose.

Reviewers: morehouse, eugenis, vitalybuka, hctim

Reviewed By: vitalybuka

Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61654

llvm-svn: 360290
2019-05-08 21:54:02 +00:00
Kostya Serebryany e13eff293d [libFuzzer] DFT: when dumping coverage, also dump the total number of instrumented blocks in a function; update merge_data_flow.py to merge coverage
llvm-svn: 360272
2019-05-08 17:20:09 +00:00
Kostya Serebryany 219b2b3a4a [libFuzzer] extend the data flow tracer to also produce basic block coverage for every input. An extended test coming in a separte change.
llvm-svn: 360213
2019-05-08 00:51:15 +00:00
Kamil Rytarowski 5d3548b852 Fix build on NetBSD 8.99.38
With recent changes the dev/nvmm/nvmm_ioctl.h header is no longer
a standalone NVMM header. Disable it until the NVMM operations will
stabilize and be included in the ioctl(2) interceptors.

llvm-svn: 360212
2019-05-08 00:44:41 +00:00
Kostya Kortchinsky ab8c8da609 [scudo][standalone] Introduce the Quarantine
Summary:
The Quarantine is used to hold chunks for a little while prior to
actually releasing them for potential reuse. The code is pretty much
the same as the sanitizer_common one, with additional shuffling of
the quarantine batches to decrease predictability of allocation
patterns when it is enabled.

Reviewers: vitalybuka, eugenis, hctim, morehouse

Reviewed By: morehouse

Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61385

llvm-svn: 360163
2019-05-07 15:40:09 +00:00
Evgeniy Stepanov c4bfa0d662 [hwasan] Fix HWASAN_WITH_INTERCEPTORS=OFF build on not-android.
Summary:
I'm not aware of any platforms where this will work, but the code should at least compile.
HWASAN_WITH_INTERCEPTORS=OFF means there is magic in libc that would call __hwasan_thread_enter /
__hwasan_thread_exit as appropriate.

Reviewers: pcc, winksaville

Subscribers: srhines, kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61337

llvm-svn: 359914
2019-05-03 18:20:16 +00:00
Martin Storsjo b1f3910283 Avoid duplicate function aliases on MinGW after SVN r359835
On MinGW, the same alias mechanism as for ELF, using
__attribute__((__alias__())), is used.

llvm-svn: 359865
2019-05-03 07:43:23 +00:00
Fangrui Song 3947968e7f [crtbegin] Fix an off-by-1 bug in __do_fini
Differential Revision: https://reviews.llvm.org/D61367

llvm-svn: 359850
2019-05-03 00:11:53 +00:00
Reid Kleckner 3961507ba1 Fix check-builtins on Windows after alias changes
llvm-svn: 359835
2019-05-02 22:11:55 +00:00
Petr Hosek 156226bb70 [compiler-rt] Set the ZX_VMO_RESIZABLE option for zx_vmo_create
Currently VMO in Zircon create using the zx_vmo_create is resizable
by default, but we'll be changing this in the future, requiring an
explicit flag to make the VMO resizable.

Prepare for this change by passing ZX_VMO_RESIZABLE option to all
zx_vmo_create calls that need resizable VMO.

Differential Revision: https://reviews.llvm.org/D61450

llvm-svn: 359803
2019-05-02 17:24:53 +00:00
Stephan Bergmann 2f0ef58e53 Fix for Windows
...after 5745eccef5 "Adapt -fsanitize=function to
SANITIZER_NON_UNIQUE_TYPEINFO"

llvm-svn: 359760
2019-05-02 07:05:29 +00:00
Stephan Bergmann 5745eccef5 Adapt -fsanitize=function to SANITIZER_NON_UNIQUE_TYPEINFO
This follows up after b7692bc3e9 "[UBSan] Fix
isDerivedFromAtOffset on iOS ARM64" fixed the RTTI comparison in
isDerivedFromAtOffset on just one platform and then
a25a2c7c9a "Always compare C++ typeinfo (based on
libstdc++ implementation)" extended that fix to more platforms.

But there is another RTTI comparison for -fsanitize=function generated in
clang's CodeGenFunction::EmitCall as just a pointer comparison.  For
SANITIZER_NON_UNIQUE_TYPEINFO platforms this needs to be extended to also do
string comparison.  For that, __ubsan_handle_function_type_mismatch[_abort]
takes the two std::type_info pointers as additional parameters now, checks them
internally for potential equivalence, and returns without reporting failure if
they turn out to be equivalent after all.  (NORETURN needed to be dropped from
the _abort variant for that.)  Also these functions depend on ABI-specific RTTI
now, so needed to be moved from plain UBSAN_SOURCES (ubsan_handlers.h/cc) to
UBSAN_CXXABI_SOURCES (ubsan_handlers_cxx.h/cc), but as -fsanitize=function is
only supported in C++ mode that's not a problem.

Differential Revision: https://reviews.llvm.org/D60760

llvm-svn: 359759
2019-05-02 06:40:33 +00:00
Hubert Tong 50721b27c4 [compiler-rt][tests] Propagate COMPILER_RT_UNITTEST_LINK_FLAGS
`COMPILER_RT_UNITTEST_LINK_FLAGS` is dropped in many places, unlike
`COMPILER_RT_UNITTEST_CFLAGS`. This patch attempts to remove that
inconsistency.

Previously reviewed as part of D58951.

Reviewers: sfertile, peter.smith, pzheng, phosek, Hahnfeld, nemanjai, jasonliu

Reviewed By: sfertile

Subscribers: jsji, kubamracek, dberris, mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60143

llvm-svn: 359733
2019-05-01 22:25:16 +00:00
Vitaly Buka 9ef838761e [sanitizer][NFC] Remove unneeded SizeClassAllocatorLocalCache
llvm-svn: 359729
2019-05-01 21:23:07 +00:00
Vitaly Buka 59bcb788ec [sanitizer][NFC] Add const/static into a couple of methods
llvm-svn: 359728
2019-05-01 21:22:59 +00:00
Julian Lettner 91c166cbb0 [Sanitizer] Reland "Cleanup INTERCEPT_FUNCTION macro"
On Linux both version of the INTERCEPT_FUNCTION macro now return true
when interception was successful. Adapt and cleanup some usages.

Also note that `&(func) == &WRAP(func)` is a link-time property, but we
do a runtime check.

Tested on Linux and macOS.

Previous attempt reverted by: 5642c3feb0

This attempt to bring order to the interceptor macro goes the other
direction and aligns the Linux implementation with the way things are
done on Windows.

Reviewed By: vitalybuka, rnk

Differential Revision: https://reviews.llvm.org/D61358

llvm-svn: 359725
2019-05-01 20:57:59 +00:00
Vitaly Buka 20c5676416 [sanitizer][NFC] Set LargeMmapAllocator type from PrimaryAllocator
They need to have same AddressSpaceView and MapUnmapCallback.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61168

llvm-svn: 359719
2019-05-01 19:41:54 +00:00
Vitaly Buka 7780f51614 [scudo][NFC] Remove unneeded template from scudo::CombinedAllocator
Reviewers: cryptoad, eugenis

Reviewed By: cryptoad

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61162

llvm-svn: 359718
2019-05-01 19:36:26 +00:00
Vitaly Buka 76931df40f [sanitizer][NFC] Get type of AllocatorCache from CombinedAllocator
Reviewers: eugenis, cryptoad, kcc

Reviewed By: kcc

Subscribers: kcc, kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61155

llvm-svn: 359715
2019-05-01 19:30:49 +00:00
Evgeniy Stepanov d1a710047b [sanitizer] Implement reallocarray.
Summary:
It's a cross of calloc and realloc. Sanitizers implement calloc-like check for size
overflow.

Reviewers: vitalybuka, kcc

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61108

llvm-svn: 359708
2019-05-01 17:33:01 +00:00
Petr Hosek 3e28075ecc [compiler-rt] Use correct spelling of the -Wno-pedantic variable
This addresses the issue introduced in r359646 which is causing
sanitizer bots to fail.

llvm-svn: 359659
2019-05-01 07:00:27 +00:00
Fangrui Song 99055b9c38 Use __extension__ to suppress __EH_FRAME_LIST__ warnings
gcc -pedantic => warning: ISO C forbids empty initializer braces
clang -pedantic => -Wzero-length-array + -Wgnu-empty-initializer

llvm-svn: 359655
2019-05-01 06:25:21 +00:00
Petr Hosek 738697ea4e [compiler-rt] Pass sysroot and disable pedantic for crtbegin.o/crtend.o
These are needed to make bots happy.

Differential Revision: https://reviews.llvm.org/D61363

llvm-svn: 359646
2019-05-01 03:30:51 +00:00
Mitch Phillips 40f98d4b78 Added test.
llvm-svn: 359632
2019-04-30 23:07:12 +00:00
Mitch Phillips 84ea3f2a48 [sanitizer_common] Added 64-bit signed flag parser.
Summary: Adds a 64-bit signed flag parser for GWP-ASan's use.

Reviewers: vlad.tsyrklevich, eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits, vitalybuka, morehouse, pcc, kcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61342

llvm-svn: 359631
2019-04-30 23:07:10 +00:00
Reid Kleckner 5642c3feb0 Revert r359325 "[NFC][Sanitizer] Change "return type" of INTERCEPT_FUNCTION to void"
Changing INTERCEPT_FUNCTION to return void is not functionally correct.
IMO the best way to communicate failure or success of interception is
with a return value, not some external address comparison.

This change was also creating link errors for _except_handler4_common,
which is exported from ucrtbase.dll in 32-bit Windows.

Also revert dependent changes r359362 and r359466.

llvm-svn: 359611
2019-04-30 20:59:56 +00:00
Jonathan Metzman f3ee97731e [libFuzzer] Replace -seed_corpus to better support fork mode on Win
Summary:
Pass seed corpus list in a file to get around argument length limits on Windows.
This limit was preventing many uses of fork mode on Windows.

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60980

llvm-svn: 359610
2019-04-30 20:56:18 +00:00
Petr Hosek 999bb5ac27 Reland "[compiler-rt] Simple crtbegin.o and crtend.o implementation"
Clang relies on existence of certain symbols that are normally
provided by crtbegin.o/crtend.o. However, LLVM does not currently
provide implementation of these files, instead relying on either
libgcc or implementations provided as part of the system.

This change provides an initial implementation of crtbegin.o/crtend.o
that can be used on system that don't provide crtbegin.o/crtend.o as
part of their C library.

Differential Revision: https://reviews.llvm.org/D28791

llvm-svn: 359591
2019-04-30 18:13:22 +00:00
Jonathan Metzman d61084901c change order
llvm-svn: 359587
2019-04-30 17:58:57 +00:00
Jonathan Metzman 2697664582 [libFuzzer] Enable for i386
Summary: Get libFuzzer to build on i386 and fix tests.

Subscribers: mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61070

llvm-svn: 359585
2019-04-30 17:58:56 +00:00
Jonathan Metzman f721230831 Enable x86 builds
llvm-svn: 359583
2019-04-30 17:58:54 +00:00
Petr Hosek 9300f60c8c Revert "[compiler-rt] Simple crtbegin.o and crtend.o implementation"
This reverts commit r359576 since it's failing on Windows bots.

llvm-svn: 359579
2019-04-30 17:32:05 +00:00
Petr Hosek c8be6e670e [compiler-rt] Simple crtbegin.o and crtend.o implementation
Clang relies on existence of certain symbols that are normally
provided by crtbegin.o/crtend.o. However, LLVM does not currently
provide implementation of these files, instead relying on either
libgcc or implementations provided as part of the system.

This change provides an initial implementation of crtbegin.o/crtend.o
that can be used on system that don't provide crtbegin.o/crtend.o as
part of their C library.

Differential Revision: https://reviews.llvm.org/D28791

llvm-svn: 359576
2019-04-30 17:21:13 +00:00
Kostya Kortchinsky 21c31f5e7b [scudo][standalone] Add the memory reclaiming mechanism
Summary:
This CL implements the memory reclaiming function `releaseFreeMemoryToOS`
and its associated classes. Most of this code was originally written by
Aleksey for the Primary64 in sanitizer_common, and I made some changes to
be able to implement 32-bit reclaiming as well. The code has be restructured
a bit to accomodate for freelist of batches instead of the freearray used
in the current sanitizer_common code.

Reviewers: eugenis, vitalybuka, morehouse, hctim

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61214

llvm-svn: 359567
2019-04-30 14:56:18 +00:00
Reid Kleckner bb92282b29 Fix FIXME added in r359339
We have windows.h in asan_win.cc, so we can just use the correct
prototypes for these EH-related interceptors without worrying.

Also fix an unused variable warning while I'm here.

llvm-svn: 359500
2019-04-29 20:44:26 +00:00
Reid Kleckner c69cd30455 Pacify sanitizer lint script after r359498
llvm-svn: 359499
2019-04-29 20:40:08 +00:00
Matthew G McGovern d62416dfcd [AddressSanitizer] [Windows] Fix HeapReAlloc and _recalloc bugs in asan_malloc_win.cc
HeapReAlloc should allow for 0 sized reallocations without freeing the memory block provided by the user.

_recalloc previously did not zero new memory after reallocation.
https://reviews.llvm.org/D61268

llvm-svn: 359498
2019-04-29 20:26:19 +00:00
Julian Lettner 1f10f6ea51 [Sanitizer] Cleanup {ASAN, MSAN}_INTERCEPT_FUNC[_VER] macro
Note that this change is not strictly NFC since we add the
`(&(name) != &WRAP(name)` part to the conditional for the `_VER` variant
of the macro.

Reviewers: vitalybuka

Differential Revision: https://reviews.llvm.org/D61204

llvm-svn: 359466
2019-04-29 16:39:18 +00:00
Petr Hosek e62915bcc1 [builtins] Use __APPLE__ instead of __MACH__ in check
The latter doesn't seem to be working for all targets. This addresses
the issue introduced in r359413.

llvm-svn: 359423
2019-04-29 08:38:43 +00:00
Petr Hosek cb929dcebe [builtins] Fix the missing assembly on Darwin
This was introduced in r359413.

llvm-svn: 359421
2019-04-29 07:45:15 +00:00
Petr Hosek ba45daab14 [builtins] Fix the typo in the preprocessor check
This was introduced in r359413.

llvm-svn: 359419
2019-04-29 06:30:50 +00:00
Petr Hosek 84da0e1bb7 [builtins] Use aliases for function redirects
Symbol aliases are supported by all platforms that compiler-rt builtins
target, and we can use these instead of function redirects to avoid the
extra indirection.

This is part of the cleanup proposed in "[RFC] compiler-rt builtins
cleanup and refactoring".

Differential Revision: https://reviews.llvm.org/D60931

llvm-svn: 359413
2019-04-29 00:46:23 +00:00
Petr Hosek 0ba22f51d1 [builtins] Use single line C++/C99 comment style
Use the uniform single line C++/99 style for code comments.

This is part of the cleanup proposed in "[RFC] compiler-rt builtins
cleanup and refactoring".

Differential Revision: https://reviews.llvm.org/D60352

llvm-svn: 359411
2019-04-28 22:47:49 +00:00
Petr Hosek 082b89b25f [builtins] Reformat builtins with clang-format
Update formatting to use the LLVM style.

This is part of the cleanup proposed in "[RFC] compiler-rt builtins
cleanup and refactoring".

Differential Revision: https://reviews.llvm.org/D60351

llvm-svn: 359410
2019-04-28 21:53:32 +00:00
Vitaly Buka 0f21545a3c [sanitizer] Calculate SizeClassAllocator32::ByteMap type from Params::kSpaceSize and Params::kRegionSizeLog
Reviewers: eugenis

Subscribers: kubamracek, cryptoad, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61206

llvm-svn: 359374
2019-04-27 06:30:52 +00:00
Vitaly Buka 83c6d10b62 [sanitizer] NFC: add static_assert to confirm that we use reasonable ByteMap type
Summary: If bots work we can replace #ifs with template specialization by TwoLevelByteMapSize1.

There is known users of TwoLevelByteMap with TwoLevelByteMapSize1 equal 8,
and users of FlatByteMap with TwoLevelByteMapSize1 equal 2.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61200

llvm-svn: 359364
2019-04-27 02:13:26 +00:00
Julian Lettner c1008e4d3d [NFC][Sanitizer] Remove GetRealFunctionAddress and replace usages
Reviewers: vitalybuka

Differential Revision: https://reviews.llvm.org/D61205

llvm-svn: 359362
2019-04-27 00:49:14 +00:00
JF Bastien 8504b5f64f Revert "[sanitizer] NFC: add static_assert to confirm that we use optimal ByteMap type"
Fails on bots with:

/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-expensive/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary32.h:69:3: error: static_assert failed due to requirement 'TwoLevelByteMapSize1 > 128' "TwoLevelByteMap should be used"
  static_assert(TwoLevelByteMapSize1 > 128, "TwoLevelByteMap should be used");
  ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-expensive/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator_combined.h:29:34: note: in instantiation of template class '__sanitizer::SizeClassAllocator32<__sanitizer::AP32>' requested here
                        typename PrimaryAllocator::AddressSpaceView>::value,
                                 ^

http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-expensive/13960/console

llvm-svn: 359352
2019-04-26 22:29:49 +00:00
Julian Lettner 93c05f097a [Sanitizer][Windows] Speculative fix for broken build on Windows
llvm-svn: 359339
2019-04-26 19:06:55 +00:00
Julian Lettner 74967cb4e0 [Sanitizer] Fix test
I broke the build, panicked and applied the wrong fix in my previous
commit. The ASSERT was obsolete, but not the call INTERCEPT_FUNCTION.

llvm-svn: 359336
2019-04-26 18:39:02 +00:00
Julian Lettner d0e5830017 [Sanitizer] Fix compliation error in test
Remove obsolete assert. I missed this in my previous patch:
https://reviews.llvm.org/D61145

llvm-svn: 359335
2019-04-26 18:27:35 +00:00
Vitaly Buka 077c502ddc Implement __sanitizer::conditional<B, T, F>
llvm-svn: 359334
2019-04-26 18:22:55 +00:00
Vitaly Buka 85dcdae5e3 [sanitizer] NFC: add static_assert to confirm that we use optimal ByteMap type
Summary: If bots work we can replace #ifs with template specialization by TwoLevelByteMapSize1.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61200

llvm-svn: 359333
2019-04-26 18:22:47 +00:00
Julian Lettner aebd3014a5 [NFC][Sanitizer] Change "return type" of INTERCEPT_FUNCTION to void
This temporary change tells us about all the places where the return
value of the INTERCEPT_FUNCTION macro is actually used. In the next
patch I will cleanup the macro and remove GetRealFuncAddress.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D61145

llvm-svn: 359325
2019-04-26 17:29:22 +00:00
Vitaly Buka efc94feef9 [NFC] Remove ::kForTest from AP64, it does not use it.
llvm-svn: 359323
2019-04-26 17:04:05 +00:00
Dan Liew 60f5df948b [asan_symbolize] Teach `asan_symbolize.py` to symbolicate using a module map
Summary:
The use case here is to be able get the UUIDs of the modules that need
to be symbolicated so that external plugins can see them. This
information can be extracted from ASan reports if the `print_module_map`
ASan option is enabled. Currently printing of the module map is only
implemented on Darwin and so this is effectively a Darwin only feature
right now.

The module map hooks into symbolization using the new plugin
infrastructure. A new hook in `AsanSymbolizerPlugInProxy` (and in
`AsanSymbolizerPlugIn`) is also provided to allow external plugins to hook
into the module look up process. This will allow external plugins to
look up modules with knowledge of their UUID.

The new plug-in is currently stored in the `asan_symbolize.py` script.
We could potentially move this into a separate file in the future (to
reduce clutter) if we can come up with a policy for where to search for
plugins that should always get loaded.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60531

llvm-svn: 359322
2019-04-26 16:54:09 +00:00
Vitaly Buka 1a607ff043 [lsan] Use SANITIZER_WORDSIZE when selecting ByteMap
Originally this code was added for 64-bit platform and it was never update.
Add static_assert to validate type of ByteMap.

llvm-svn: 359286
2019-04-26 08:24:38 +00:00
Vitaly Buka d8e9c3a999 Revert "[lsan] Use SANITIZER_WORDSIZE when selecting ByteMap"
New static_assert fails on a bot.

This reverts commit r359269.

llvm-svn: 359276
2019-04-26 05:19:32 +00:00
Vitaly Buka a9e7fd2188 NFC: Clang-format msan_allocator.cc
llvm-svn: 359270
2019-04-26 04:22:28 +00:00
Vitaly Buka 3db2a7a04f [lsan] Use SANITIZER_WORDSIZE when selecting ByteMap
Originally this code as added for 64-bit platform and was never changed.
Add static_assert to make sure that we have correct map on all platforms.

llvm-svn: 359269
2019-04-26 04:20:27 +00:00
Matt Morehouse 1b76063a5e [libFuzzer] Disable MSan interceptors in SIGINT handler.
Summary:
Avoids an MSan false positive if the SIGINT comes while the user
callback is running.  The false positive happens when the interrupt
handler calls opendir() to remove some temporary files, which is
intercepted by MSan.

Fixes https://github.com/google/oss-fuzz/issues/2332.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, Dor1s, metzman

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61163

llvm-svn: 359254
2019-04-26 00:17:41 +00:00
Julian Lettner 8b36610bfa [NFC][Sanitizer] Extract GetFuncAddr from GetRealFunctionAddress
Summary:
Hopefully, this will enable cleanup/removal of GetRealFunctionAddress in
follow-up commits.

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61110

llvm-svn: 359213
2019-04-25 17:46:29 +00:00
Kostya Kortchinsky 3fad6a206f [scudo][standalone] Introduce the SizeClassMap
Summary:
As with the sanitizer_common allocator, the SCM allows for efficient
mapping between sizes and size-classes, table-free.

It doesn't depart significantly from the original, except that we
allow the use of size-class 0 for other purposes (as opposed to
chunks of size 0). The Primary will use it to hold TransferBatches.

Reviewers: vitalybuka, eugenis, hctim, morehouse

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61088

llvm-svn: 359199
2019-04-25 15:49:34 +00:00
Mitch Phillips 1af5a85e48 [sanitizer_common] Remove some old commented-out printf statements, and fixed up the order of includes.
llvm-svn: 359150
2019-04-24 23:03:32 +00:00
Vitaly Buka 0ddd12e156 [compiler-rt] NFC fixed [whitespace/braces] LINT warning
llvm-svn: 359133
2019-04-24 20:01:50 +00:00
Vitaly Buka 97bdb04a3b [compiler-rt] Fix warning about virtual destructor in sanitizer_flag_parser.h
llvm-svn: 359125
2019-04-24 19:01:04 +00:00
Mitch Phillips ca0a68705f [llvm-symbolizer] Quick fix for broken sanitizer bot
(sanitizer-x86_64-linux) until I can triage the issue properly. The
build has been broken due to the symbolizer build checks failing.

As the symbolizer build script relies on the old svn repo layout, it may
take a little while longer to find the responsible patch for the
breakage. This may be a completely valid fix, but I will need to confirm
it. For now, it unbreaks the build.

Tracking data:
Build where the break first occurred: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/21211
Changelist authors: @grimar, @maskray, @whitequark, @spatel, @dpreobra

For the authors, no action needed (yet). Will follow up when I can
identify the cause.

llvm-svn: 359123
2019-04-24 18:37:55 +00:00