Commit Graph

9102 Commits

Author SHA1 Message Date
Dan Liew 62a8728750 [asan_symbolize] More debugging output
When a line fails to match the stackframe regex we now report
it in the log.

rdar://problem/49476995

llvm-svn: 367380
2019-07-31 00:59:10 +00:00
Dan Liew 51b1d858d3 [asan_symbolize] Add `--skip-uuid-validation` option to `ModuleMapPlugIn`.
This option disables the validation of binary UUIDs. This is useful
in environments where the `otool` binary is not available.

rdar://problem/49476995

llvm-svn: 367379
2019-07-31 00:59:10 +00:00
Dan Liew 23fd828d4e [asan_symbolize] Provide better error message when extracting the UUID of a binary fails
rdar://problem/49476995

llvm-svn: 367378
2019-07-31 00:59:09 +00:00
Francis Visoiu Mistrih fb3c53af34 Revert "[compiler-rt] Appending COMPILER_RT_LIBCXX_PATH -isystem include for xray (2)"
This reverts commit r367250. It's failing on green dragon:
http://lab.llvm.org:8080/green/job/clang-stage1-RA/482/console.

llvm-svn: 367265
2019-07-29 22:08:12 +00:00
Puyan Lotfi a9c59b2873 [compiler-rt] Appending COMPILER_RT_LIBCXX_PATH -isystem include for xray (2)
Second attempt.

Haven't found a better way to pass the libcxx include path for building
compiler-rt with libcxx; this seems to be missing only for xray.

Differential Revision: https://reviews.llvm.org/D65307

llvm-svn: 367250
2019-07-29 19:21:58 +00:00
Puyan Lotfi 5bf16ec02b [compiler-rt] Reverting r367239 (xray include path) to fix bot build breakage.
llvm-svn: 367246
2019-07-29 17:50:39 +00:00
Puyan Lotfi ba7b7f1de4 Appending COMPILER_RT_LIBCXX_PATH include path to -isystem for compiler-rt xray
Haven't found a better way to pass the libcxx include path for building
compiler-rt with libcxx; this seems to be missing only for xray.

Differential Revision: https://reviews.llvm.org/D65307

llvm-svn: 367239
2019-07-29 16:46:35 +00:00
Eugene Leviant 45d910de99 Don't initialize interceptor_metadata_map unless SI_POSIX is set
Differential revision: https://reviews.llvm.org/D64794

llvm-svn: 367188
2019-07-28 08:58:44 +00:00
Pirama Arumuga Nainar 6caa8da072 [profile] In Android, do not mkdir() dirs in GCOV_PREFIX
Summary:
In Android, attempting to mkdir() or even stat() top-level directories
like /data causes noisy selinux denials.  During whole-system coverage
instrumentation, this causes a deluge of noisy messages that drown out
legitimate selinux denials, that should be audited and fixed.

To avoid this, skip creating any directory in GCOV_PREFIX (thereby
assuming that it exists).

- Android platform ensures that the GCOV_PREFIX used in Android is
created and read/writable by all processes.
- This only affects the Android platform (by checking against
__ANDROID_API_FUTURE__) and for apps built with Clang coverage, the
runtime will still create any non-existent parent directories for the
coverage files.

Reviewers: srhines, davidxl

Subscribers: krytarowski, #sanitizers, danalbert, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D65245

llvm-svn: 367064
2019-07-25 22:10:56 +00:00
Julian Lettner 758f6da687 [sanitizer_common] Allow customization of StartSymbolizerSubprocess
Summary:
Make SymbolizerProcess::StartSymbolizerSubprocess virtual and protected
to allow subclasses to customize it via "override and call".

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D65252

llvm-svn: 366967
2019-07-25 00:19:02 +00:00
Kostya Kortchinsky 419f1a4185 [scudo][standalone] Optimization pass
Summary:
This introduces a bunch of small optimizations with the purpose of
making the fastpath tighter:
- tag more conditions as `LIKELY`/`UNLIKELY`: as a rule of thumb we
  consider that every operation related to the secondary is unlikely
- attempt to reduce the number of potentially extraneous instructions
- reorganize the `Chunk` header to not straddle a word boundary and
  use more appropriate types

Note that some `LIKELY`/`UNLIKELY` impact might be less obvious as
they are in slow paths (for example in `secondary.cc`), but at this
point I am throwing a pretty wide net, and it's consistant and doesn't
hurt.

This was mosly done for the benfit of Android, but other platforms
benefit from it too. An aarch64 Android benchmark gives:
- before:
```
  BM_youtube/min_time:15.000/repeats:4/manual_time_mean              445244 us       659385 us            4
  BM_youtube/min_time:15.000/repeats:4/manual_time_median            445007 us       658970 us            4
  BM_youtube/min_time:15.000/repeats:4/manual_time_stddev               885 us         1332 us            4
```
- after:
```
  BM_youtube/min_time:15.000/repeats:4/manual_time_mean       415697 us       621925 us            4
  BM_youtube/min_time:15.000/repeats:4/manual_time_median     415913 us       622061 us            4
  BM_youtube/min_time:15.000/repeats:4/manual_time_stddev        990 us         1163 us            4
```

Additional since `-Werror=conversion` is enabled on some platforms we
are built on, enable it upstream to catch things early: a few sign
conversions had slept through and needed additional casting.

Reviewers: hctim, morehouse, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, javed.absar, kristof.beyls, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64664

llvm-svn: 366918
2019-07-24 16:36:01 +00:00
Max Moroz 374398afb7 [libFuzzer] Fix __sanitizer_print_memory_profile return type in ExtFunctions.def
Summary:
Looks like a typo, as that function actually returns void and is used
as such in libFuzzer code as well.

Reviewers: kcc, Dor1s

Reviewed By: Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D65160

llvm-svn: 366834
2019-07-23 18:26:53 +00:00
Rainer Orth 569f92f1e1 [compiler-rt][builtins] Provide __clear_cache for SPARC
While working on https://reviews.llvm.org/D40900, two tests were failing since __clear_cache
aborted.  While libgcc's __clear_cache is just empty, this only happens because
gcc (in gcc/config/sparc/sparc.c (sparc32_initialize_trampoline, sparc64_initialize_trampoline))
emits flush insns directly.

The following patch mimics that.

Tested on sparcv9-sun-solaris2.11.

Differential Revision: https://reviews.llvm.org/D64496

llvm-svn: 366822
2019-07-23 16:33:54 +00:00
Serge Guelton a30a4a35ec Fix asan infinite loop on undefined symbol
Fix llvm#39641

Recommit of r366413

Differential Revision: https://reviews.llvm.org/D63877

> llvm-svn: 366632

llvm-svn: 366638
2019-07-20 17:44:30 +00:00
Serge Guelton 7a3d4c15a7 Revert "Fix asan infinite loop on undefined symbol"
This reverts commit cbd28cd05b.

Buildbot fail: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/22434/steps/64-bit%20check-asan-dynamic/

llvm-svn: 366634
2019-07-20 13:00:12 +00:00
Serge Guelton cbd28cd05b Fix asan infinite loop on undefined symbol
Fix llvm#39641

Recommit of r366413

Differential Revision: https://reviews.llvm.org/D63877

llvm-svn: 366632
2019-07-20 12:01:18 +00:00
Matthew Voss 407e837540 Revert "Fix asan infinite loop on undefined symbol"
This reverts commit 8e46275488.

This was failing on sanitizer-x86_64-linux and our internal CI.

llvm-svn: 366618
2019-07-19 21:41:07 +00:00
Serge Guelton 8e46275488 Fix asan infinite loop on undefined symbol
Fix llvm#39641

Recommit of r366413

Differential Revision: https://reviews.llvm.org/D63877

llvm-svn: 366588
2019-07-19 15:20:36 +00:00
Julian Lettner be7a7ae0c3 [ASan] Support `{f}puts(NULL)` on Darwin, part 2
Add braces around macro `{ MACRO(); }` to guard against macros that
expand to multiple statements.

llvm-svn: 366488
2019-07-18 20:14:50 +00:00
Matthew Voss 3e93131dd2 Revert "Fix asan infinite loop on undefined symbol"
This reverts commit 63719119c7.

The tests in this commit were failing on a Linux sanitizer bot
and our internal CI.

http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/22401/steps/test%20standalone%20compiler-rt%20with%20symbolizer/logs/stdio

llvm-svn: 366472
2019-07-18 17:43:44 +00:00
Serge Guelton 63719119c7 Fix asan infinite loop on undefined symbol
Fix llvm#39641

Differential Revision: https://reviews.llvm.org/D63877

llvm-svn: 366413
2019-07-18 08:09:31 +00:00
Julian Lettner 9a050f92bb [ASan] Support `{f}puts(NULL)` on Darwin
On Darwin, the man page states that "both fputs() and puts() print
`(null)' if str is NULL."

rdar://48227136

Reviewed By: Lekensteyn

Differential Revision: https://reviews.llvm.org/D64773

llvm-svn: 366342
2019-07-17 16:09:25 +00:00
Julian Lettner d069a19210 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.6
Cleanup ARM64 assembly after removing unnecessary computation of mangled
stack pointer in previous commit.

llvm-svn: 366258
2019-07-16 20:41:04 +00:00
Mitch Phillips b157dcacb5 [GWP-ASan] Add thread ID to PRNG seed.
Summary:
Adds thread ID to PRNG seed for increased entropy. In particular, this allows
multiple runs in quick succession that will have different PRNG seeds, allowing
for better demos/testing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D64453

llvm-svn: 366253
2019-07-16 20:06:17 +00:00
Mitch Phillips 97b4d7a8e1 Removed -mno-omit-leaf-frame-pointer from flags.
Removes -mno-omit-leaf-frame-pointer from Scudo and GWP-ASan's CFlags. Attempt to fix
the sanitizer buildbots.

llvm-svn: 366228
2019-07-16 17:13:02 +00:00
Julian Lettner e6e33cf9f5 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.10
Remove now-unused assembly code for determining xor key on
Linux/AArch64. This is the final commit of this refactoring.

llvm-svn: 366225
2019-07-16 16:22:04 +00:00
Stephan Bergmann e215996a29 Finish "Adapt -fsanitize=function to SANITIZER_NON_UNIQUE_TYPEINFO"
i.e., recent 5745eccef54ddd3caca278d1d292a88b2281528b:

* Bump the function_type_mismatch handler version, as its signature has changed.

* The function_type_mismatch handler can return successfully now, so
  SanitizerKind::Function must be AlwaysRecoverable (like for
  SanitizerKind::Vptr).

* But the minimal runtime would still unconditionally treat a call to the
  function_type_mismatch handler as failure, so disallow -fsanitize=function in
  combination with -fsanitize-minimal-runtime (like it was already done for
  -fsanitize=vptr).

* Add tests.

Differential Revision: https://reviews.llvm.org/D61479

llvm-svn: 366186
2019-07-16 06:23:27 +00:00
Julian Lettner c48162db99 [TSan] Fix asm token error (again)
llvm-svn: 366150
2019-07-15 23:05:14 +00:00
Matt Morehouse 8528b1951c [ASan] Fix >80 character line.
llvm-svn: 366136
2019-07-15 22:07:56 +00:00
Julian Lettner 228f80d92d [TSan] Fix asm token error
llvm-svn: 366131
2019-07-15 21:22:57 +00:00
Julian Lettner 39d888c1e4 [TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.9
Switch over to computing the xor key in C, instead of assembly for
Linux/AArch64.

llvm-svn: 366126
2019-07-15 20:22:27 +00:00
Matthew G McGovern 4885978e23 [sanitizers][windows][mingw32] Mingw32 RTL fixes
RTL interception broke mingw32, this should fix those builds by
removing dependency on windows.h

reviewed in https://reviews.llvm.org/D64694

llvm-svn: 366105
2019-07-15 18:42:14 +00:00
Julian Lettner 4f519b6919 [TSan] Tiny cleanup of UnmangleLongJmpSp for Linux/x86_64
NFC.

llvm-svn: 365982
2019-07-13 00:55:06 +00:00
Nikita Popov a205ebb09c [builtins] Fix assembly in arm sync-ops.h
This assembly is part of a macro that was reformatted in D60351.
The missing space between push and { results in:

    Error: bad instruction `push{r4, r5,r6,lr}'

llvm-svn: 365957
2019-07-12 20:52:02 +00:00
Rainer Orth 4a9a772f44 Enable compiler-rt on SPARC
This patch enables compiler-rt on SPARC targets. Most of the changes are straightforward:

- Add 32 and 64-bit sparc to compiler-rt

- lib/builtins/fp_lib.h needed to check if the int128_t and uint128_t types exist (which they don't on sparc)

There's one issue of note: many asan tests fail to compile on Solaris/SPARC:

fatal error: error in backend: Function "_ZN7testing8internal16BoolFromGTestEnvEPKcb": over-aligned dynamic alloca not supported.

Therefore, while asan is still built, both asan and ubsan-with-asan testing is disabled. The
goal is to check if asan keeps compiling on Solaris/SPARC. This serves asan in gcc,
which doesn't have the problem above and works just fine.

With this patch, sparcv9-sun-solaris2.11 test results are pretty good:

Failing Tests (9):
    Builtins-sparc-sunos :: divtc3_test.c
    Builtins-sparcv9-sunos :: compiler_rt_logbl_test.c
    Builtins-sparcv9-sunos :: divtc3_test.c
[...]
    UBSan-Standalone-sparc :: TestCases/TypeCheck/misaligned.cpp
    UBSan-Standalone-sparcv9 :: TestCases/TypeCheck/misaligned.cpp

The builtin failures are due to Bugs 42493 and 42496. The tree contained a few additonal
patches either currently in review or about to be submitted.

Tested on sparcv9-sun-solaris2.11.

Differential Revision: https://reviews.llvm.org/D40943

llvm-svn: 365880
2019-07-12 08:30:17 +00:00
Rainer Orth 56f2f1cdb3 [Sanitizers] Fix SanitizerCommon-Unit :: ./Sanitizer-*-Test/MemoryMappingLayout.DumpListOfModules on Solaris
The MemoryMappingLayout.DumpListOfModules currently FAILs on Solaris:

  [ RUN      ] MemoryMappingLayout.DumpListOfModules
  /vol/llvm/src/compiler-rt/local/lib/sanitizer_common/tests/sanitizer_procmaps_test.cc:52: Failure
  Value of: found
    Actual: false
  Expected: true
  [  FAILED  ] MemoryMappingLayout.DumpListOfModules (22 ms)

The problem is that the test expects the executable name from modules[i].full_name(),
however the pr_mapname field of struct prmap is just the entry in /proc/<pid>/object,
which is "a.out" instead of "Sanitizer-i386-Test".  Fortunately, the real name can
be determined by looking in proc/<pid>/path where "a.out" is a symlink to the
real path.

Tested on x86_64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D64559

llvm-svn: 365879
2019-07-12 08:27:50 +00:00
Kostya Kortchinsky 8f18a4c980 [scudo][standalone] NFC corrections
Summary:
A few corrections:
- rename `TransferBatch::MaxCached` to `getMaxCached` to conform with
  the style guide;
- move `getBlockBegin` from `Chunk::` to `Allocator::`: I believe it
  was a fallacy to have this be a `Chunk` method, as chunks'
  relationship to backend blocks are up to the frontend allocator. It
  makes more sense now, particularly with regard to the offset. Update
  the associated chunk test as the method isn't available there
  anymore;
- add a forgotten `\n` to a log string;
- for `releaseToOs`, instead of starting at `1`, start at `0` and
  `continue` on `BatchClassId`: in the end it's identical but doesn't
  assume a particular class id for batches;
- change a `CHECK` to a `reportOutOfMemory`: it's a clearer message

Reviewers: hctim, morehouse, eugenis, vitalybuka

Reviewed By: hctim

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64570

llvm-svn: 365816
2019-07-11 19:55:53 +00:00
Julian Lettner 9d41429ff7 [TSan] Move DECLARE_REAL macro outside of namespace
This should fix the "undefined reference to
tsan::interception::real_setjmp" linker errors.

llvm-svn: 365806
2019-07-11 18:26:33 +00:00
Mitch Phillips af3dc759e7 Explicitly define __STDC_FORMAT_MACROS for PRIu64
Summary:
Builds are failing on RHEL machines because of PRIu64.

lvm/projects/compiler-rt/lib/gwp_asan/guarded_pool_allocator.cpp:420:50: error: expected ')'

`snprintf(ThreadBuffer, kThreadBufferLen, "%" PRIu64, ThreadID);`
inttypes.h in RHEL uses PRIu64 macros only when __STDC_FORMAT_MACROS is defined.

Author: DTharun

Reviewers: hctim

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D64388

llvm-svn: 365801
2019-07-11 18:07:03 +00:00
Kostya Kortchinsky aeb3826228 [scudo][standalone] Merge Spin & Blocking mutex into a Hybrid one
Summary:
We ran into a problem on Fuchsia where yielding threads would never
be deboosted, ultimately resulting in several threads spinning on the
same TSD, and no possibility for another thread to be scheduled,
dead-locking the process.

While this was fixed in Zircon, this lead to discussions about if
spinning without a break condition was a good decision, and settled on
a new hybrid model that would spin for a while then block.

Currently we are using a number of iterations for spinning that is
mostly arbitrary (based on sanitizer_common values), but this can
be tuned in the future.

Since we are touching `common.h`, we also use this change as a vehicle
for an Android optimization (the page size is fixed in Bionic, so use
a fixed value too).

Reviewers: morehouse, hctim, eugenis, dvyukov, vitalybuka

Reviewed By: hctim

Subscribers: srhines, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64358

llvm-svn: 365790
2019-07-11 15:32:26 +00:00
Petr Hosek 274ad9c371 [ASan] Use __sanitizer_fill_shadow for FastPoisonShadow on Fuchsia
This is the optimized implementation for Fuchsia provided by the libc.

Differential Revision: https://reviews.llvm.org/D64166

llvm-svn: 365739
2019-07-11 07:30:11 +00:00
Kamil Rytarowski 983d7ddd0b Add NetBSD LSan support
Summary:
Combine few relatively small changes into one:

 - implement internal_ptrace() and internal_clone() for NetBSD
 - add support for stoptheworld based on the ptrace(2) API
 - define COMPILER_RT_HAS_LSAN for NetBSD
 - enable tests for NetBSD/amd64

Inspired by the original implementation by Christos Zoulas in netbsd/src for GCC.

The implementation is in theory CPU independent through well defined macros
across all NetBSD ports, however only the x86_64 version was tested.

Reviewers: mgorny, dvyukov, vitalybuka, joerg, jfb

Reviewed By: vitalybuka

Subscribers: dexonsmith, jfb, srhines, kubamracek, llvm-commits, christos

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64057

llvm-svn: 365735
2019-07-11 06:22:35 +00:00
Julian Lettner 6a155f52bc [TSan] Fix linker error on Linux/AArch64
llvm-svn: 365707
2019-07-10 22:50:07 +00:00
Julian Lettner 098e0a79fa [TSan] Fix compiler error on Linux/AArch64
llvm-svn: 365693
2019-07-10 19:46:08 +00:00
Julian Lettner 1a2c880924 [TSan] Attempt to fix linker error on Linux/AArch64
llvm-svn: 365681
2019-07-10 18:37:57 +00:00
Julian Lettner ba9eb48318 Reland "[TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.8"
Fix compilation errors related to `SANITIZER_GO` `#ifdef`s.

Refine longjmp key management.  For Linux, re-implement key retrieval in
C (instead of assembly).  Removal of `InitializeGuardPtr` and a final
round of cleanups will be done in the next commit.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D64092

> llvm-svn: 365513

llvm-svn: 365560
2019-07-09 21:27:37 +00:00
Julian Lettner c11c43b64f [TSan] Refine longjmp key management on Darwin
NFC.

llvm-svn: 365554
2019-07-09 20:47:37 +00:00
Peter Collingbourne 1366262b74 hwasan: Improve precision of checks using short granule tags.
A short granule is a granule of size between 1 and `TG-1` bytes. The size
of a short granule is stored at the location in shadow memory where the
granule's tag is normally stored, while the granule's actual tag is stored
in the last byte of the granule. This means that in order to verify that a
pointer tag matches a memory tag, HWASAN must check for two possibilities:

* the pointer tag is equal to the memory tag in shadow memory, or
* the shadow memory tag is actually a short granule size, the value being loaded
  is in bounds of the granule and the pointer tag is equal to the last byte of
  the granule.

Pointer tags between 1 to `TG-1` are possible and are as likely as any other
tag. This means that these tags in memory have two interpretations: the full
tag interpretation (where the pointer tag is between 1 and `TG-1` and the
last byte of the granule is ordinary data) and the short tag interpretation
(where the pointer tag is stored in the granule).

When HWASAN detects an error near a memory tag between 1 and `TG-1`, it
will show both the memory tag and the last byte of the granule. Currently,
it is up to the user to disambiguate the two possibilities.

Because this functionality obsoletes the right aligned heap feature of
the HWASAN memory allocator (and because we can no longer easily test
it), the feature is removed.

Also update the documentation to cover both short granule tags and
outlined checks.

Differential Revision: https://reviews.llvm.org/D63908

llvm-svn: 365551
2019-07-09 20:22:36 +00:00
Max Moroz 2d73994046 [libFuzzer] Include FuzzedDataProvider.h in the test without "utils" subdir.
Summary:
This way the test would better match the intended usage of the header,
plus it makes some additional testing (e.g. in CI) a bit easier to set up.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D64440

llvm-svn: 365544
2019-07-09 19:28:14 +00:00
Julian Lettner 95176d72c7 Revert "[TSan] Improve handling of stack pointer mangling in {set,long}jmp, pt.8"
This reverts commit 521f77e635.

llvm-svn: 365534
2019-07-09 18:42:01 +00:00