Commit Graph

10934 Commits

Author SHA1 Message Date
Kuba Mracek 2d8a66f8d3 Mark two tests, which require too much available VM, as "UNSUPPORTED: ios".
llvm-svn: 343650
2018-10-03 00:30:03 +00:00
Evgeniy Stepanov edede5daff [hwasan] Fix top PC in error stack trace being off by 1 instruction.
Summary:
GetStackTrace treats top PC as a return address from an error reporting
function, and adjusts it down by 1 instruction. This is not necessary in
a signal handler, so adjust PC up to compensate.

Reviewers: kcc, vitalybuka, jfb

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D52802

llvm-svn: 343638
2018-10-02 21:52:02 +00:00
Petr Hosek a70ece45eb [lib/fuzzer] Fix logging for Fuchsia
This change fixes two aspects of logging on Fuchsia:

1. When logging to stdout, the output file descriptor should not be
closed in ExecuteCommand, as it prevent fdio_spawn_etc from succeeding
in subsequent calls.
2. When logging to a file in anything other than standalone Zircon, the
log file needs to be created in mutable storage. The best indicator
where this is relative to the Fuchsia component will be given by
'-artifact_prefix=...', so save the log file relative to that path.

Patch by: aarongreen
Differential Revision: https://reviews.llvm.org/D52562

llvm-svn: 343607
2018-10-02 17:21:04 +00:00
David Major 6d6c9150f9 Reland r342652 "[winasan] Unpoison the stack in NtTerminateThread"
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.

Differential Revision: https://reviews.llvm.org/D52091

llvm-svn: 343606
2018-10-02 17:17:12 +00:00
Vitaly Buka 83e57e2123 [sanitizer] Include inlined frames into __sanitizer_symbolize_pc output
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.

New clients can read until they get empty string.

Reviewers: eugenis, morehouse

Subscribers: kubamracek, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D52743

llvm-svn: 343605
2018-10-02 17:01:18 +00:00
Kristina Brooks 59500f7a0b [Arm builtins] Remove non-necessary IS check
This patch removes the instruction set check to make the msr APSR_nzcvq,
ip instruction only execute if Thumb2 is used.
The APSR is a subset of the bits of the CPSR
(B.1.3.3 of the Arm v7 A and R ARM [1]) and is only available for A and
R profiles.
However in section B.9.3.11 of the same document we see that:

"In the A and R profiles, APSR_nzcvq is the same as CPSR_f"
"ARM recommends the APSR forms when only the N, Z, C, V, Q, and GE[3:0]
bits are being written."

This patch also make those files assemble for Armv8-M Mainline
architecture profile.

The builtins were cross-compiled for Arm, Aarch64 and Armv6-M, Armv7-M
and Armv7E-M targets.
Cross-compiled tests were executed for Arm target.

[1]: https://developer.arm.com/docs/ddi0406/latest/arm-architecture-reference-manual-armv7-a-and-armv7-r-edition

Patch by hug-dev (Hugues de Valon).

Differential Revision: https://reviews.llvm.org/D51854

llvm-svn: 343601
2018-10-02 16:32:32 +00:00
Jessica Paquette 9cfc8eca70 Revert "[sanitizer] Include inlined frames into __sanitizer_symbolize_pc output"
This reverts r343554.

It was breaking some bots:
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA/49997/

llvm-svn: 343600
2018-10-02 16:28:52 +00:00
Stephen Hines 80e7d65a42 [sanitizer] Use -Wl,-z,global on Android for sanitizers except UBsan
Summary:
This essentially reverts r337010 since it breaks UBSan, which is used
for a few platform libraries. The "-z global" flag is now added for
Scudo as well. The only other sanitizer shared libraries are for asan
and hwasan, which have also been reinstated to use the global flag.

Reviewers: cryptoad, eugenis

Reviewed By: cryptoad

Subscribers: kubamracek, mgorny, delcypher, #sanitizers, nickdesaulniers, chh, kongyi, pirama, llvm-commits

Differential Revision: https://reviews.llvm.org/D52770

llvm-svn: 343599
2018-10-02 16:19:42 +00:00
Vitaly Buka ae1bbea890 [sanitizer] Include inlined frames into __sanitizer_symbolize_pc output
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.

New clients can read until they get empty string.

Reviewers: eugenis, morehouse

Subscribers: kubamracek, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D52743

llvm-svn: 343554
2018-10-02 00:29:41 +00:00
Vitaly Buka a477e59b03 [sanitizer] Small buffer tests for __sanitizer_symbolize_pc
llvm-svn: 343534
2018-10-01 20:39:49 +00:00
Douglas Yung 22d4948a50 Remove a path-length limitation for the xray logfile.
Reviewers: MaskRay

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D52728

Patch by Jeremy Morse!

llvm-svn: 343524
2018-10-01 20:03:53 +00:00
Kamil Rytarowski 162aac511a Switch syscall(2)/__syscall(2) calls to libc calls on NetBSD
Summary:
When possible, switch syscall(2)/__syscall(2) calls
to direct calls of internal libc symbols.

Add a new function to detect address of a libc
symbol of a function that could be intercepted.
With the address detector in GetRealLibcAddress(),
an optional interceptor of libc call will be bypassed.

The original approach with syscall(2)/__syscall(2)
wasn't portable across supported ABIs and CPU
architectures. The indirect syscall interface is
also a candidate for removal in future revisions
of NetBSD, as the C language ABI is not a good
domain for serialization of arbitrary functions
arguments.

Switch the following functions to libc calls:
 - internal_mmap()
 - internal_munmap()
 - internal_mprotect()
 - internal_close()
 - internal_open()
 - internal_read()
 - internal_write()
 - internal_ftruncate()
 - internal_stat()
 - internal_lstat()
 - internal_fstat()
 - internal_dup2()
 - internal_readlink()
 - internal_unlink()
 - internal_rename()
 - internal_sched_yield()
 - internal__exit()
 - internal_sleep()
 - internal_execve()
 - NanoTime()
 - internal_clock_gettime()
 - internal_waitpid()
 - internal_getpid()
 - internal_getppid()
 - internal_getdents()
 - internal_lseek()
 - internal_sigaltstack()
 - internal_fork()
 - internal_sigprocmask()
 - internal_sysctl()
 - internal_sigemptyset()
 - internal_sigfillset()
 - GetTid()
 - TgKill()

This revision leaves room for refactoring in subsequent commits.

Reviewers: vitalybuka, kcc, joerg

Reviewed By: vitalybuka

Subscribers: mgorny, fedor.sergeev, jfb, loverszhaokai, devnexen, kubamracek, llvm-commits, ro, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D51419

llvm-svn: 343523
2018-10-01 19:50:12 +00:00
David Carlier 50a46a5961 Unbreak linux cfi build
llvm-svn: 343512
2018-10-01 18:14:02 +00:00
David Carlier dc0624844f [Cfi] Compiling cfi library on FreeBSD and NetBSD
Making the library slighty more portable.

Reviewers: vitalybuka, krytarowski

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D51682

llvm-svn: 343510
2018-10-01 18:01:55 +00:00
Dan Liew b6a07f5886 [NFC] Fix `-Wcast-qual` warnings in Darwin `internal_sysctl(...)` implementation.
Summary: The warnings were introduced in r341187 and r341192.

Reviewers: kubamracek, george.karpenkov, krytarowski

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52717

llvm-svn: 343496
2018-10-01 16:51:01 +00:00
Kamil Rytarowski a14b908381 Switch sanitizer_procmaps_bsd to internal_sysctl
Summary:
Stop using directly sysctl(3) routines in sanitizer_procmaps_bsd
and replace it with internal_sysctl().

This will allow to install interceptors for sysctl(3).

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D52646

llvm-svn: 343370
2018-09-29 07:45:03 +00:00
Hans Wennborg 83cd9b32dd Revert r342652 "[winasan] Unpoison the stack in NtTerminateThread"
This seems to cause the thread's exit code to be clobbered, breaking
Chromium tests.

Also revert follow-up r342654.

> In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
>
> Differential Revision: https://reviews.llvm.org/D52091

llvm-svn: 343322
2018-09-28 14:41:25 +00:00
Fangrui Song 13a6750c2e [XRay] Fix fdr-thread-order.cc when current directory contains fdr-thread-order.cc
Summary:
Currently,

    cd test/xray/TestCases/Posix
    $build/bin/llvm-lit fdr-thread-order.cc

fails because `rm fdr-thread-order.*` deletes the .cc file.

This patch uses:

* %t as temporary directory name containing log files
* %t.exe as executable name

It does not delete %t after the test finishes for debugging convenience.
This matches the behavior of tests of various other LLVM components.

Log files will not clog up because the temporary directory (unique among
test files but the same among multiple invocations of a test) is cleaned
at the beginning of the test.

Reviewers: dberris, mboerger, eizan

Reviewed By: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D52638

llvm-svn: 343295
2018-09-28 04:20:08 +00:00
Fangrui Song 58f209f5d7 [XRay] Guard local variables with `static` and struct with unnamed namespaces
Summary:
This is for coding standard conformance, and for fixing an ODR violation
issue: __xray::ThreadLocalData is defined twice and differently in
xray_fdr_logging.cc and xray_basic_logging.cc

Reviewers: dberris, mboerger, eizan

Reviewed By: dberris

Subscribers: delcypher, jfb, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D52639

llvm-svn: 343289
2018-09-28 02:05:52 +00:00
Fangrui Song 0ce4b37bd0 [XRay] Fix argv0-log-file-name.cc race when tests are executed parallelly
`rm xray-log.*` may delete log files of other tests and cause them to
fail, when tests are executed parallelly.

llvm-svn: 343282
2018-09-27 23:59:57 +00:00
Dean Michael Berris c84c46fc0c [XRay] Add LD_LIBRARY_PATH to env variables for Unit Tests
Summary:
This change allows us to use the library path from which the LLVM
libraries are installed, in case the LLVM installation generates shared
libraries.

This should address llvm.org/PR39070.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52597

llvm-svn: 343280
2018-09-27 23:15:05 +00:00
Kostya Kortchinsky 67392feb49 [sanitizer] Disable failing Android test after D52371
Summary:
The default values used for Space/Size for the new SizeClassMap do not work
with Android. The Compact map appears to be in the same boat.
Disable the test on Android for now to turn the bots green, but there is no
reason Compact & Dense should not have an Android test.
Added a FIXME, I will revisit this soon.

Reviewers: eugenis

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52623

llvm-svn: 343252
2018-09-27 19:15:40 +00:00
Roman Lebedev f11d564629 [compiler-rt][ubsan][NFC] Slight test cleanup in preparation for D50902.
Reviewers: vsk, vitalybuka, filcab

Reviewed By: vitalybuka

Subscribers: kubamracek, dberris, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D52590

llvm-svn: 343250
2018-09-27 19:07:47 +00:00
Kostya Kortchinsky 7685301d79 [sanitizer] Introduce a new SizeClassMap with minimal amount of cached entries
Summary:
_Note_: I am not attached to the name `DenseSizeClassMap`, so if someone has a
better idea, feel free to suggest it.

The current pre-defined `SizeClassMap` hold a decent amount of cached entries,
either in cheer number of, or in amount of memory cached.

Empirical testing shows that more compact per-class arrays (whose sizes are
directly correlated to the number of cached entries) are beneficial to
performances, particularly in highly threaded environments.

The new proposed `SizeClassMap` has the following properties:
```
c00 => s: 0 diff: +0 00% l 0 cached: 0 0; id 0
c01 => s: 16 diff: +16 00% l 4 cached: 8 128; id 1
c02 => s: 32 diff: +16 100% l 5 cached: 8 256; id 2
c03 => s: 48 diff: +16 50% l 5 cached: 8 384; id 3
c04 => s: 64 diff: +16 33% l 6 cached: 8 512; id 4
c05 => s: 80 diff: +16 25% l 6 cached: 8 640; id 5
c06 => s: 96 diff: +16 20% l 6 cached: 8 768; id 6
c07 => s: 112 diff: +16 16% l 6 cached: 8 896; id 7

c08 => s: 128 diff: +16 14% l 7 cached: 8 1024; id 8
c09 => s: 144 diff: +16 12% l 7 cached: 7 1008; id 9
c10 => s: 160 diff: +16 11% l 7 cached: 6 960; id 10
c11 => s: 176 diff: +16 10% l 7 cached: 5 880; id 11
c12 => s: 192 diff: +16 09% l 7 cached: 5 960; id 12
c13 => s: 208 diff: +16 08% l 7 cached: 4 832; id 13
c14 => s: 224 diff: +16 07% l 7 cached: 4 896; id 14
c15 => s: 240 diff: +16 07% l 7 cached: 4 960; id 15

c16 => s: 256 diff: +16 06% l 8 cached: 4 1024; id 16
c17 => s: 320 diff: +64 25% l 8 cached: 3 960; id 49
c18 => s: 384 diff: +64 20% l 8 cached: 2 768; id 50
c19 => s: 448 diff: +64 16% l 8 cached: 2 896; id 51

c20 => s: 512 diff: +64 14% l 9 cached: 2 1024; id 48
c21 => s: 640 diff: +128 25% l 9 cached: 1 640; id 49
c22 => s: 768 diff: +128 20% l 9 cached: 1 768; id 50
c23 => s: 896 diff: +128 16% l 9 cached: 1 896; id 51

c24 => s: 1024 diff: +128 14% l 10 cached: 1 1024; id 48
c25 => s: 1280 diff: +256 25% l 10 cached: 1 1280; id 49
c26 => s: 1536 diff: +256 20% l 10 cached: 1 1536; id 50
c27 => s: 1792 diff: +256 16% l 10 cached: 1 1792; id 51

c28 => s: 2048 diff: +256 14% l 11 cached: 1 2048; id 48
c29 => s: 2560 diff: +512 25% l 11 cached: 1 2560; id 49
c30 => s: 3072 diff: +512 20% l 11 cached: 1 3072; id 50
c31 => s: 3584 diff: +512 16% l 11 cached: 1 3584; id 51

c32 => s: 4096 diff: +512 14% l 12 cached: 1 4096; id 48
c33 => s: 5120 diff: +1024 25% l 12 cached: 1 5120; id 49
c34 => s: 6144 diff: +1024 20% l 12 cached: 1 6144; id 50
c35 => s: 7168 diff: +1024 16% l 12 cached: 1 7168; id 51

c36 => s: 8192 diff: +1024 14% l 13 cached: 1 8192; id 48
c37 => s: 10240 diff: +2048 25% l 13 cached: 1 10240; id 49
c38 => s: 12288 diff: +2048 20% l 13 cached: 1 12288; id 50
c39 => s: 14336 diff: +2048 16% l 13 cached: 1 14336; id 51

c40 => s: 16384 diff: +2048 14% l 14 cached: 1 16384; id 48
c41 => s: 20480 diff: +4096 25% l 14 cached: 1 20480; id 49
c42 => s: 24576 diff: +4096 20% l 14 cached: 1 24576; id 50
c43 => s: 28672 diff: +4096 16% l 14 cached: 1 28672; id 51

c44 => s: 32768 diff: +4096 14% l 15 cached: 1 32768; id 48
c45 => s: 40960 diff: +8192 25% l 15 cached: 1 40960; id 49
c46 => s: 49152 diff: +8192 20% l 15 cached: 1 49152; id 50
c47 => s: 57344 diff: +8192 16% l 15 cached: 1 57344; id 51

c48 => s: 65536 diff: +8192 14% l 16 cached: 1 65536; id 48
c49 => s: 81920 diff: +16384 25% l 16 cached: 1 81920; id 49
c50 => s: 98304 diff: +16384 20% l 16 cached: 1 98304; id 50
c51 => s: 114688 diff: +16384 16% l 16 cached: 1 114688; id 51

c52 => s: 131072 diff: +16384 14% l 17 cached: 1 131072; id 48
c53 => s: 64 diff: +0 00% l 0 cached: 8 512; id 4
Total cached: 864928 (152/432)
```

It holds a bit less of 1MB of cached entries at most, and the cache fits in a 
page.

The plan is to use this map by default for Scudo once we make sure that there
is no unforeseen impact for any of current use case.

Benchmarks give the most increase in performance (with Scudo) when looking at
highly threaded/contentious environments. For example, rcp2-benchmark
experiences a 10K QPS increase (~3%), and a decrease of 50MB for the max RSS
(~10%). On platforms like Android where we only have a couple of caches,
performance remain similar.

Reviewers: eugenis, kcc

Reviewed By: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52371

llvm-svn: 343246
2018-09-27 18:20:42 +00:00
Jordan Rupprecht 777bc9f924 [compiler-rt] [builtins] Restore tests from r342917 (disabled in r343095) on Windows.
Summary:
-lm is needed for these tests on Linux, but the lit config for this package automatically adds it for Linux and excludes it for Windows. So we should be able to get these tests running again by just dropping -lm and let the lit config add it when possible.

I was under the impression that -lm worked across platforms because it exists in other tests without and 'UNSUPPORTED: windows' commands (e.g. divsc3_test.c), but those are actually excluded because they 'REQUIRES: c99-complex' which is excluded from windows platforms (also by the local lit config).

I don't have easy access to a windows machine to verify this patch, but I can trigger a build bot run on clang-x64-ninja-win7 shortly after submitting.

Reviewers: hans

Subscribers: dberris, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D52563

llvm-svn: 343245
2018-09-27 18:13:01 +00:00
Simon Atanasyan ab4f15e20c [compiler-rt][cmake][mips] Use -mabi option to select target architecture
The `--target` option is rejected by GCC starting from version 7.0.
It's shorter and more portable to use `mabi` option to select MIPS ABI
for testing target architecture. Clang supports that starting from r343169.

llvm-svn: 343182
2018-09-27 07:17:00 +00:00
Fangrui Song 77708b2390 [Coverage] Disable Linux/coverage-missing.cc in AddressSanitizer-i386-linux testsuite
sancov.py relies on objdump -d to obtain the number of instrumented PCs.
The i386 %dynamiclib will now include .plt entries that are not recognized by objdump,

"sancov.py: found 0 instrumented PCs in *.so",

causing AddressSanitizer-i386-linux to fail.

Change it back to x86-target-arch after %sancov switches to a more robust approach.

llvm-svn: 343178
2018-09-27 06:37:15 +00:00
Evgeniy Stepanov f42aa29b68 [sanitizer] AndroidGetApiLevel for static executables.
A version of AndroidGetApiLevel for static executables that is
completely compile-time.

The generic version uses dl_iterate_phdr which, even though it works
in static executables, crashes if called before libc is initialized.

llvm-svn: 343153
2018-09-26 23:48:13 +00:00
Martin Storsjo a89731ca71 [asan] [windows] Don't use CheckFailed in dynamic runtime thunk mode
Since SVN r342651, CheckFailed isn't exported from asan-dynamic. See
comments in https://reviews.llvm.org/D52279#1246222 for a longer
discussion on this issue.

Differential Revision: https://reviews.llvm.org/D52566

llvm-svn: 343136
2018-09-26 19:49:35 +00:00
David Major e705624cff [winasan] Pin the ASan DLL to prevent unloading
Differential Revision: https://reviews.llvm.org/D52505

llvm-svn: 343123
2018-09-26 16:28:39 +00:00
Hans Wennborg 5d6f3d9f45 Disable tests from r342917 on Windows; -lm won't work there
llvm-svn: 343095
2018-09-26 11:23:00 +00:00
Martin Storsjo 50de4518c5 [CMake] [MinGW] Build address sanitizer for MinGW if building with clang
Differential Revision: https://reviews.llvm.org/D51885

llvm-svn: 343074
2018-09-26 06:53:01 +00:00
Martin Storsjo fed729f28e [ASan] [MinGW] Only try to export MSVC specific C++ symbols if building with a MSVC like compiler
Differential Revision: https://reviews.llvm.org/D51878

llvm-svn: 343073
2018-09-26 06:52:55 +00:00
Martin Storsjo 32df11e3e3 Try to fix cosmetics to keep lines below 80 chars. NFC.
This should fix following buildbot errors:
http://lab.llvm.org:8011/builders/clang-ppc64le-linux/builds/20371

llvm-svn: 343031
2018-09-25 21:27:07 +00:00
Martin Storsjo 45205a2834 [Sanitizers] [MinGW] Produce undecorated symbols for /export: directives when in MinGW mode
In MinGW mode, the /export: directives in object files are interpreted
differently; the symbols are not assumed to be decorated.

Since the GNU linker doesn't support the /alternatename and /include
directives, there's no such distinction for them in lld. This assumes
that the resulting sanitizer libraries will be linked by lld.

Differential Revision: https://reviews.llvm.org/D51882

llvm-svn: 343015
2018-09-25 19:50:39 +00:00
Martin Storsjo 693f3a573c [ASan] [Windows] Avoid including windows.h in asan_malloc_win.cc
Instead provide manual declarations of the used types, to avoid
pulling in conflicting declarations of some of the functions that
are to be overridden.

Differential Revision: https://reviews.llvm.org/D51914

llvm-svn: 343014
2018-09-25 19:50:36 +00:00
Martin Storsjo fd9aa7e078 [Sanitizers] [MinGW] Check for __i386__ in addition to _M_IX86 for i386 specific details
The MinGW headers do define _M_IX86 (contrary to clang-cl and
MSVC where it is a compiler predefined macro), but the headers that
define it aren't included here.

Also check these defines for setting the symbol prefix, instead of
inconsistently using _WIN64 for that.

Differential Revision: https://reviews.llvm.org/D51883

llvm-svn: 343013
2018-09-25 19:50:32 +00:00
Calixte Denizet cb4cd5ccdc [Profile] Fix gcov tests
Summary: The gcda need to be delete before running the binary to avoid to have an increasing "# of Runs" when a test is failing

Reviewers: vitalybuka, eugenis, marco-c

Reviewed By: marco-c

Subscribers: delcypher, llvm-commits, #sanitizers, sylvestre.ledru, marco-c

Differential Revision: https://reviews.llvm.org/D52456

llvm-svn: 342963
2018-09-25 11:12:15 +00:00
Evgeniy Stepanov 090f0f9504 [hwasan] Record and display stack history in stack-based reports.
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.

The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.

Developed in collaboration with Kostya Serebryany.

Reviewers: kcc

Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52249

llvm-svn: 342923
2018-09-24 23:03:34 +00:00
Evgeniy Stepanov 20c4999e8b Revert "[hwasan] Record and display stack history in stack-based reports."
This reverts commit r342921: test failures on clang-cmake-arm* bots.

llvm-svn: 342922
2018-09-24 22:50:32 +00:00
Evgeniy Stepanov 9043e17edd [hwasan] Record and display stack history in stack-based reports.
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.

The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.

Developed in collaboration with Kostya Serebryany.

Reviewers: kcc

Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52249

llvm-svn: 342921
2018-09-24 21:38:42 +00:00
Jordan Rupprecht dc48c4fff4 [compiler-rt] [builtins] Add logb/logbf/logbl methods to compiler-rt to avoid libm dependencies when possible.
Summary:
The complex division builtins (div?c3) use logb methods from libm to scale numbers during division and avoid rounding issues. However, these come from libm, meaning anyone that uses --rtlib=compiler-rt also has to include -lm. Implement logb* methods for standard ieee 754 floats so we can avoid -lm on those platforms, falling back to the old behavior (using either logb() or `__builtin_logb()`) when not supported.

These new methods are defined internally as `__compiler_rt_logb` so as not to conflict with the libm definitions in any way.

This fixes just the libm methods mentioned in PR32279 and PR28652. libc is still required, although that seems to not be an issue.

Note: this is proposed as an alternative to just adding -lm: D49330.

Reviewers: efriedma, compnerd, scanon, echristo

Reviewed By: echristo

Subscribers: jsji, echristo, nemanjai, dberris, mgorny, kbarton, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49514

llvm-svn: 342917
2018-09-24 20:39:19 +00:00
Calixte Denizet d32508a932 [profile] Revert commit https://reviews.llvm.org/rL342718
llvm-svn: 342913
2018-09-24 18:24:29 +00:00
Dan Liew c0ba5dc47b When running the ios/iossim prepare script show the script output when it returns with a non-zero exit code.
Summary:
Previously we'd just show the exception and not the output from the
executed script. This is unhelpful in the case that the script actually
reports some useful information on the failure.

Now we print the output and re-raise the exception.

Reviewers: kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52350

llvm-svn: 342869
2018-09-24 09:30:33 +00:00
Dan Liew 071fdc1b01 Fix the configuration of the Primary allocator for Darwin ARM64 by
changing the value of `SANITIZER_MMAP_RANGE_SIZE` to something more
sensible. The available VMA is at most 64GiB and not 256TiB that
was previously being used.

This change gives us several wins:

* Drastically improves LeakSanitizer performance on
  Darwin ARM64 devices. On a simple synthentic benchmark
  this took leak detection time from ~30 seconds to 0.5 seconds
  due to the `ForEachChunk(...)` method enumerating a much smaller
  number of regions. Previously we would pointlessly iterate
  over a large portion of the SizeClassAllocator32's ByteMap
  that would could never be set due it being configured for a much
  larger VM space than is actually availble.

* Decreases the memory required for the Primary allocator.
  Previously the ByteMap inside the the allocator used
  an array of pointers that took 512KiB of space. Now the required
  space for the array is 128 bytes.

rdar://problem/43509428

Differential Revision: https://reviews.llvm.org/D51173

llvm-svn: 342868
2018-09-24 09:28:35 +00:00
Dean Michael Berris f578aaa058 [XRay] Clean up XRay build configuration
Summary:
This change spans both LLVM and compiler-rt, where we do the following:

- Add XRay to the LLVMBuild system, to allow for distributing the XRay
  trace loading library along with the LLVM distributions.

- Use `llvm-config` better in the compiler-rt XRay implementation, to
  depend on the potentially already-distributed LLVM XRay library.

While this is tested with the standalone compiler-rt build, it does
require that the LLVMXRay library (and LLVMSupport as well) are
available during the build. In case the static libraries are available,
the unit tests will build and work fine. We're still having issues with
attempting to use a shared library version of the LLVMXRay library since
the shared library might not be accessible from the standard shared
library lookup paths.

The larger change here is the inclusion of the LLVMXRay library in the
distribution, which allows for building tools around the XRay traces and
profiles that the XRay runtime already generates.

Reviewers: echristo, beanz

Subscribers: mgorny, hiraditya, mboerger, llvm-commits

Differential Revision: https://reviews.llvm.org/D52349

llvm-svn: 342859
2018-09-24 05:28:01 +00:00
Vlad Tsyrklevich 00705a7f95 SafeStack: Fix flaky test (PR39001)
Summary:
pthread_join() can return before a thread finishes exit()ing in the
kernel and a subsequent tgkill() can report the thread still alive.
Update the pthread-cleanup.c test to sleep and retry if it hits this
possible flake.

Thanks to Jeremy Morse for reporting.

Reviewers: jmorse, eugenis, vitalybuka

Reviewed By: jmorse, vitalybuka

Subscribers: delcypher, jfb, llvm-commits, #sanitizers, kcc

Differential Revision: https://reviews.llvm.org/D52330

llvm-svn: 342763
2018-09-21 19:02:32 +00:00
Kostya Kortchinsky 7bdb0b9493 [sanitizer] Move __sanitizer_set_death_callback to a more appropriate file
Summary:
`__sanitizer_set_death_callback` is a public interface function wrapping
`SetUserDieCallback`. Said function is defined in `sanitizer_termination.cc`,
which is not included in all the RT. Moving the interface function to that
file allows to not have a spurious public fuinction in RT that don't use it.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52363

llvm-svn: 342747
2018-09-21 17:05:56 +00:00
Dean Michael Berris 0cb22386e0 [XRay][compiler-rt] Update use of internal_mmap
Summary:
The implementation of `internal_mmap(...)` deviates from the contract of
`mmap(...)` -- i.e. error returns are actually the equivalent of `errno`
results. We update how XRay uses `internal_mmap(...)` to better handle
these error conditions.

In the process, we change the default pointers we're using from `char*`
to `uint8_t*` to prevent potential usage of the pointers in the string
library functions that expect to operate on `char*`.

We also take the chance to "promote" sizes of individual `internal_mmap`
requests to at least page size bytes, consistent with the expectations
of calls to `mmap`.

Reviewers: cryptoad, mboerger

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D52361

llvm-svn: 342745
2018-09-21 16:34:42 +00:00
Jeremy Morse b52bd9eed6 [libfuzzer] Fix a test's run line
r342698 removed the 'UNSUPPORTED: windows' flags from a number of fuzzer
tests, however too many lines were removed from 'fuzzer-flags.test',
including the run-line that generates the test binary, which breaks that
test for me (and the clang-x64-ninja-win7 buildbot). This patch just
re-adds that line.

llvm-svn: 342720
2018-09-21 10:50:15 +00:00
Calixte Denizet 8f7aab5c61 [profile] Fix the tests for patch in https://reviews.llvm.org/D49916.
Summary:
The goal of D49916 is to remove some hit counters from lines like '}'.
So as a consequence, the tests must be fixed.
FYI, after holidays, I'll add more tests.

Reviewers: marco-c, davidxl

Reviewed By: marco-c

Subscribers: delcypher, llvm-commits, #sanitizers, sylvestre.ledru

Differential Revision: https://reviews.llvm.org/D49917

llvm-svn: 342718
2018-09-21 09:17:14 +00:00
Jonathan Metzman 0744d3c5a1 [fuzzer] Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
Summary:
Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
to get externally defined functions (eg: LLVMFuzzerInitialize,
LLVMFuzzerCustomMutator, etc) working again.

Also enable tests that depended on these functions (on windows)

Reviewers: rnk, morehouse

Reviewed By: rnk, morehouse

Subscribers: rnk, morehouse, mgorny

Differential Revision: https://reviews.llvm.org/D51700

llvm-svn: 342698
2018-09-20 23:24:48 +00:00
David Carlier 362e6095a6 [Hwasan] interceptor macro / extra comma removal
gcc being pedantic, removing the unnecessary comma.

Reviewers: eugenis, kcc

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D52305

llvm-svn: 342680
2018-09-20 19:30:51 +00:00
Calixte Denizet 6dde0d4283 [profile] Fix tests in compiler-rt for patch in gcov (https://reviews.llvm.org/D49659)
Summary: Some tests are broken if patch in D49659 is accepted so this patch fixes them.

Reviewers: marco-c

Reviewed By: marco-c

Subscribers: dberris, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49721

llvm-svn: 342661
2018-09-20 16:21:09 +00:00
David Major 2ebabe8f6c Fix test after r342652
llvm-svn: 342654
2018-09-20 15:22:21 +00:00
David Major 468f53b58c [winasan] Unpoison the stack in NtTerminateThread
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.

Differential Revision: https://reviews.llvm.org/D52091

llvm-svn: 342652
2018-09-20 14:59:33 +00:00
Kostya Kortchinsky caa9619bce [sanitizer] Make __sanitizer::CheckFailed not public
Summary:
As far as I can tell, there is no reason why `__sanitizer::CheckFailed` should
be exported. Looking back in time, it was added with the FIXME with the
following by @timurrrr:

```
[*San/RTL] Fix minor breakage
Grumbling: this hasn't been caught by running 'make check-{a,l,t}san check-sanitizer'
```

I can't find any detail about the breakage, all tests seem to work for me, so
maybe Windows (@rnk?) or something I have no setup for.

The reason to make it private (past the FIXME) is that Scudo defines its own
(without callback) and I am trying to make the .so be loadable with the UBsan
one (that has its own public `CheckFailed`) with as little drama as possible.

Reviewers: eugenis, rnk

Reviewed By: eugenis, rnk

Subscribers: kubamracek, delcypher, #sanitizers, timurrrr, rnk, llvm-commits

Differential Revision: https://reviews.llvm.org/D52279

llvm-svn: 342651
2018-09-20 14:50:06 +00:00
David Major c4624d9e3f [winasan] Reduce hotpatch prefix check to 8 bytes
Same idea as r310419: The 8 byte nop is a suffix of the 9 byte nop, and we need at most 6 bytes.

Differential Revision: https://reviews.llvm.org/D51788

llvm-svn: 342649
2018-09-20 14:28:50 +00:00
Dean Michael Berris 1f60207984 [XRay][compiler-rt] FDRLogWriter Abstraction
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.

The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.

In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.

We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module. We also add
the terminfo library detection along with inclusion of the appropriate
compiler flags for header include lookup.

Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.

Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52220

llvm-svn: 342617
2018-09-20 05:22:37 +00:00
Yi Kong 0c4cf6436a [builtins] Add __emutls_unregister_key function
This is called by Bionic on dlclose to delete the emutls pthread key.

The return value of pthread_key_delete is unchecked and behaviour of
multiple calls to the method is dependent on the implementation of
pthread_key_delete.

Differential Revision: https://reviews.llvm.org/D52251

llvm-svn: 342608
2018-09-20 01:02:13 +00:00
Evgeniy Stepanov 09e7f243f1 Revert "[XRay][compiler-rt] FDRLogWriter Abstraction" and 1 more.
Revert the following 2 commits to fix standalone compiler-rt build:
* r342523 [XRay] Detect terminfo library
* r342518 [XRay][compiler-rt] FDRLogWriter Abstraction

llvm-svn: 342596
2018-09-19 22:29:56 +00:00
Kostya Kortchinsky 851a7c9b2b [sanitizer][fuchsia] Fix VMAR leak
Summary:
Destroy and close a range's vmar if all its memory was unmapped.

This addresses some performance regression due to the proliferation of vmars
when Secondary backed allocations are concerned with Scudo on Fuchsia.

When a Secondary backed allocation was freed, the associated
`ReservedAddressRange` was going away after unmapping the entirety of the
mapping, but without getting rid of the associated vmar properly (which
was created specifically for that mapping). This resulted in an increase of
defunct vmars, that in turn slowed down further new vmar allocations.

This appears to solve ZX-2560/ZX-2642, at least on QEMU.

Reviewers: flowerhack, mcgrathr, phosek, mseaborn

Reviewed By: mcgrathr

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52242

llvm-svn: 342584
2018-09-19 19:50:35 +00:00
Dan Liew bd810dbd27 [UBSan] Fix typo in CMake conditional that checked if the architecture
of a darwin platform was in the list of `UBSAN_SUPPORTED_ARCH`.

This is a follow up to r341306.

The typo meant that if an architecture was a prefix to another
architecture in the list (e.g. `armv7` is a prefix of `armv7k`) then
this would trigger a match which is not the intended behaviour.

rdar://problem/41126835

llvm-svn: 342553
2018-09-19 15:27:00 +00:00
Arnaud A. de Grandmaison 86a5e43656 [compiler-rt][TSan] Add TSan runtime support for Go on linux-aarch64.
Summary:
This patch adds TSan runtime support for Go on linux-aarch64
platforms. This enables people working on golang to implement their
platform/language part of the TSan support.

Basic testing is done with lib/tsan/go/buildgo.sh. Additional testing will be
done as part of the work done in the Go project.

It is intended to support other VMA sizes, except 39 which does not
have enough bits to support the Go heap requirements.

Patch by Fangming Fang <Fangming.Fang@arm.com>.

Reviewers: kubamracek, dvyukov, javed.absar

Subscribers: mcrosier, dberris, mgorny, kristof.beyls, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52167

llvm-svn: 342541
2018-09-19 11:56:20 +00:00
Dean Michael Berris ff93d3a8aa [XRay] Detect terminfo library
Instead of assuming `-ltinfo` works, check whether there's terminfo
support on the host where LLVMSupport is compiled.

Follow-up to D52220.

llvm-svn: 342523
2018-09-19 01:35:52 +00:00
Dean Michael Berris b64f71b029 [XRay][compiler-rt] FDRLogWriter Abstraction
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.

The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.

In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.

We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module.

Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.

Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52220

llvm-svn: 342518
2018-09-18 23:59:32 +00:00
Saleem Abdulrasool 73c39c0328 build: clean up some unnecessary cached variables
The CMAKE_<LANG>_ARCHIVE_FINISH rule doesn't need to be cleared for Darwin
static libraries.  Avoid resetting the variables in the SIP case.  If
CMAKE_RANLIB is cached, then CMake's Ninja generator will invoke ranlib during
installation, not due to the CMAKE_<LANG>_ARCHIVE_FINISH rule.

llvm-svn: 342511
2018-09-18 20:33:01 +00:00
Kristina Brooks 22db696549 [builtins] Fix c?zdi2 on sparc64/Linux and ignore riscv32
On sparc64/Linux, sparc64 isn't defined; the canonical way of
checking for sparc64 is sparc && arch64, which also works on the
BSDs and Solaris. Since this problem does not occur on 32-bit
architectures, riscv32 can be ignored. This fixes and refines rL324593.

Patch by jrtc27 (James Clarke)

Differential Revision: https://reviews.llvm.org/D43146

llvm-svn: 342504
2018-09-18 18:56:52 +00:00
Saleem Abdulrasool f5ebd70a67 build: fix standalone builds for compiler-rt on Darwin
When building static fat libraries, we need to ensure that we use libtool rather
than llvm-ar to create the library.  Duplicate the rules from LLVM to ensure
that we correctly build the fat libraries when building compiler-rt standalone.
This also requires that we duplicate the workaround for the `DYLD_LIBRARY_PATH`
for SIP.  Additionally, ensure that we set the `CMAKE_*_ARCHIVE_FINISH` variable
to ensure that we do not try to use `ranlib` on that target.

llvm-svn: 342425
2018-09-17 23:25:36 +00:00
Matt Morehouse 1e1f3c8298 [libFuzzer] Avoid fuzzer symbols being hidden.
Fixes https://bugzilla.mozilla.org/show_bug.cgi?id=1490845#c5.

Patch By: Mike Hommey

llvm-svn: 342423
2018-09-17 23:08:15 +00:00
Dan Liew fb310c0af9 [UBSan] Partially fix `test/ubsan/TestCases/Misc/log-path_test.cc` so that it can run on devices.
Summary:
In order for this test to work the log file needs to be removed from both
from the host and device. To fix this the `rm` `RUN` lines have been
replaced with `RUN: rm` followed by `RUN: %device_rm`.

Initially I tried having it so that `RUN: %run rm` implicitly runs `rm`
on the host as well so that only one `RUN` line is needed. This
simplified writing the test however that had two large drawbacks.

* It's potentially very confusing (e.g. for use of the device scripts outside
  of the lit tests) if asking for `rm` to run on device also causes files
  on the host to be deleted.

* This doesn't work well with the glob patterns used in the test.
  The host shell expands the `%t.log.*` glob pattern and not on the
  device so we could easily miss deleting old log files from previous
  test runs if the corresponding file doesn't exist on the host.

So instead deletion of files on the device and host are explicitly
separate commands.

The command to delete files from a device is provided by a new
substitution `%device_rm` as suggested by Filipe Cabecinhas.

The semantics of `%device_rm` are that:

* It provides a way remove files from a target device when
 the host is not the same as the target. In the case that the
 host and target are the same it is a no-op.

* It interprets shell glob patterns in the context of the device
  file system instead of the host file system.
  This solves the globbing problem provided the argument is quoted so
  that lit's underlying shell doesn't try to expand the glob pattern.

* It supports the `-r` and `-f` flags of the `rm` command,
  with the same semantics.

Right now an implementation of `%device_rm` is provided only for
ios devices. For all other devices a lit warning is emitted and
the `%device_rm` is treated as a no-op. This done to avoid changing
the behaviour for other device types but leaves room for others
to implement `%device_rm`.

The ios device implementation uses the `%run` wrapper to do the work
of removing files on a device.

The `iossim_run.py` script has been fixed so that it just runs `rm`
on the host operating system because the device and host file system
are the same.

rdar://problem/41126835

Reviewers: vsk, kubamracek, george.karpenkov, eugenis

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51648

llvm-svn: 342391
2018-09-17 13:33:44 +00:00
Petr Hosek 040ab65c53 [sanitizer_common] Fuchsia now supports .preinit_array
Support for .preinit_array has been implemented in Fuchsia's libc,
add Fuchsia to the list of platforms that support this feature.

Differential Revision: https://reviews.llvm.org/D52155

llvm-svn: 342357
2018-09-17 05:22:26 +00:00
Dean Michael Berris 1a23d3bbce [XRay] Simplify FDR buffer management
Summary:
This change makes XRay FDR mode use a single backing store for the
buffer queue, and have indexes into that backing store instead. We also
remove the reliance on the internal allocator implementation in the FDR
mode logging implementation.

In the process of making this change we found an inconsistency with the
way we're returning buffers to the queue, and how we're setting the
extents. We take the chance to simplify the way we're managing the
extents of each buffer. It turns out we do not need the indirection for
the extents, so we co-host the atomic 64-bit int with the buffer object.
It also seems that we've not been returning the buffers for the thread
running the flush functionality when writing out the files, so we can
run into a situation where we could be missing data.

We consolidate all the allocation routines now into xray_allocator.h,
where we used to have routines defined in xray_buffer_queue.cc.

Reviewers: mboerger, eizan

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52077

llvm-svn: 342356
2018-09-17 03:09:01 +00:00
Dean Michael Berris d5577aea07 [XRay] Fix FDR initialization
Follow-up to D51606.

llvm-svn: 342355
2018-09-17 02:49:17 +00:00
Petr Hosek 9e8b4de3b3 [XRay] Remove the unused variable
This broke the fdr-single-thread test after FDRLoggingOptions struct
has been removed in r342318.

llvm-svn: 342320
2018-09-15 06:25:17 +00:00
Petr Hosek d197ebf3ed [XRay] Remove the deprecated __xray_log_init API
This API has been deprecated three months ago and shouldn't be used
anymore, all clients should migrate to the new string based API.

Differential Revision: https://reviews.llvm.org/D51606

llvm-svn: 342318
2018-09-15 02:55:42 +00:00
Matt Morehouse 0039792d7b [libFuzzer] Disable value-profile-strncmp.test on aarch64.
Test no longer finds the BINGO on clang-cmake-aarch64-full bot, and I
can't reproduce on our ARM machine.

llvm-svn: 342255
2018-09-14 18:24:02 +00:00
Ulrich Weigand 9ed8fd5df8 [asan] Fix test case failure on SystemZ
Since we changed our inlining parameters, this test case was failing
on SystemZ, as the two tests were now both inlined into the main
function, which the test didn't expect.  Fixed by adding a few more
noinline attributes.

llvm-svn: 342236
2018-09-14 13:36:55 +00:00
Dean Michael Berris 05cf443463 [XRay][clang] Emit "never-instrument" attribute
Summary:
Before this change, we only emit the XRay attributes in LLVM IR when the
-fxray-instrument flag is provided. This may cause issues with thinlto
when the final binary is being built/linked with -fxray-instrument, and
the constitutent LLVM IR gets re-lowered with xray instrumentation.

With this change, we can honour the "never-instrument "attributes
provided in the source code and preserve those in the IR. This way, even
in thinlto builds, we retain the attributes which say whether functions
should never be XRay instrumented.

This change addresses llvm.org/PR38922.

Reviewers: mboerger, eizan

Subscribers: mehdi_amini, dexonsmith, cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D52015

llvm-svn: 342200
2018-09-14 01:59:12 +00:00
Kostya Serebryany c173a703b5 [hwasan] use a single Printf per line when printing a report (more friendly to android logging)
llvm-svn: 342164
2018-09-13 19:14:22 +00:00
Dean Michael Berris 90a46bdec2 [XRay] Bug fixes for FDR custom event and arg-logging
Summary:
This change has a number of fixes for FDR mode in compiler-rt along with
changes to the tooling handling the traces in llvm.

In the runtime, we do the following:

- Advance the "last record" pointer appropriately when writing the
  custom event data in the log.

- Add XRAY_NEVER_INSTRUMENT in the rewinding routine.

- When collecting the argument of functions appropriately marked, we
  should not attempt to rewind them (and reset the counts of functions
  that can be re-wound).

In the tooling, we do the following:

- Remove the state logic in BlockIndexer and instead rely on the
  presence/absence of records to indicate blocks.

- Move the verifier into a loop associated with each block.

Reviewers: mboerger, eizan

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D51965

llvm-svn: 342122
2018-09-13 09:25:42 +00:00
Vlad Tsyrklevich aadfe4661d [TSan] Update test values
Similarly to before, D51985 again reduced the number of registers
required for the read/write routines causing this test to fail on
sanitizer-x86_64-linux-autoconf.

llvm-svn: 342092
2018-09-12 22:16:14 +00:00
Kostya Serebryany ee9bb8761c [hwasan] On every use-after-free print a developer note: the index of this heap object in the thread's deallocation ring buffer. Mostly useful to hwasan developers, will hopefully let us know the good size of the deallocation ring buffer
llvm-svn: 342014
2018-09-12 00:58:15 +00:00
Kostya Serebryany 2362bf7d10 [hwasan] tests for a buffer overflow with a large allocation
llvm-svn: 342011
2018-09-12 00:27:34 +00:00
Martin Storsjo e199a2d434 [libFuzzer] [Windows] Include windows.h and psapi.h with lowercase
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase, and matches how these
headers are included elsewhere in compiler-rt.

Also include these headers with angle brackets, as they are system
headers.

Differential Revision: https://reviews.llvm.org/D51913

llvm-svn: 341983
2018-09-11 19:11:54 +00:00
Vedant Kumar 727d89526e [gcov] Fix branch counters with switch statements (fix PR38821)
Right now, the counters are added in regards of the number of successors
for a given BasicBlock: it's good when we've only 1 or 2 successors (at
least with BranchInstr). But in the case of a switch statement, the
BasicBlock after switch has several predecessors and we need know from
which BB we're coming from.

So the idea is to revert what we're doing: add a PHINode in each block
which will select the counter according to the incoming BB.  They're
several pros for doing that:

- we fix the "switch" bug
- we remove the function call to "__llvm_gcov_indirect_counter_increment"
  and the lookup table stuff
- we replace by PHINodes, so the optimizer will probably makes a better
  job.

Patch by calixte!

Differential Revision: https://reviews.llvm.org/D51619

llvm-svn: 341977
2018-09-11 18:38:34 +00:00
David Carlier 395aa78986 [Msan] Fix unit test linkage issue on FreeBSD
Fix typical relocation linkage issue.

Reviwewers: dim, emaste

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D51681

llvm-svn: 341924
2018-09-11 10:35:32 +00:00
Martin Storsjo b2b0f859d4 [ASan] [Windows] Remove const from _msize function declaration parameter
This function isn't declared with a const parameter anywhere; neither
in MSVC (neither in ucrt or in older msvcrt versions) nor in mingw-w64.

Differential Revision: https://reviews.llvm.org/D51876

llvm-svn: 341903
2018-09-11 06:07:32 +00:00
Evgeniy Stepanov 7e6c32aa45 [hwasan] Re-enable print-memory-usage-android test.
The problem was not in a non-rooted device, but in tagged local
variable address passed to a system call, see comments in the code.

llvm-svn: 341875
2018-09-10 22:22:02 +00:00
Martin Storsjo 6dc8968124 [Windows] Include BaseTsd.h with lowercase
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase. This header actually also
is named with a lowercase name in the Windows SDK as well.

Differential Revision: https://reviews.llvm.org/D51877

llvm-svn: 341857
2018-09-10 20:28:06 +00:00
Kostya Serebryany 1ee01bbf37 [hwasan] rename two .cc tests into .c
llvm-svn: 341739
2018-09-08 01:27:10 +00:00
Evgeniy Stepanov 8f0d9020d4 [hwasan] Disable print-memory-usage-android test.
Requires a rooted device => fails on sanitizer-x86_64-linux-android bot.

llvm-svn: 341738
2018-09-08 01:16:47 +00:00
Richard Smith 4357ca653a Switch to using a reserved identifier for this internal compiler-rt function.
llvm-svn: 341736
2018-09-08 00:17:37 +00:00
Evgeniy Stepanov d263cb8ea1 [hwasan] Export memory stats through /proc/$PID/maps.
Adds a line to /proc/$PID/maps with more or less up-to-date memory
stats of the process.

llvm-svn: 341735
2018-09-08 00:11:12 +00:00
Dean Michael Berris edf0f6a79b [XRay] XRAY_NEVER_INSTRUMENT more functions, consolidate allocators
Summary:
In this change we apply `XRAY_NEVER_INSTRUMENT` to more functions in the
profiling implementation to ensure that these never get instrumented if
the compiler used to build the library is capable of doing XRay
instrumentation.

We also consolidate all the allocators into a single header
(xray_allocator.h) which sidestep the use of the internal allocator
implementation in sanitizer_common.

This addresses more cases mentioned in llvm.org/PR38577.

Reviewers: mboerger, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D51776

llvm-svn: 341647
2018-09-07 10:16:14 +00:00
Alexander Potapenko f1510528dd [MSan] Define %clang_kmsan for KMSAN tests
llvm-svn: 341638
2018-09-07 09:17:12 +00:00
Jonathan Metzman 3ab0ee4d9b [libfuzzer] Enable trace-malloc-unballanced.test on Windows.
Summary:
Enables trace-malloc-unbalanced.test on Windows, fixing two problems it had with Windows before.
The first fix is specifying python instead of relying on a script's shebang since they can't be used on Windows.
The second fix is making the regex tolerate windows' implementation of the "%p" format string.

Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: morehouse

Differential Revision: https://reviews.llvm.org/D51760

llvm-svn: 341632
2018-09-07 04:28:54 +00:00
Kostya Serebryany 6fdfd6a34f [hwasan] relax the rint-memory-usage.c test further
llvm-svn: 341625
2018-09-07 01:36:34 +00:00
Jonathan Metzman 6167c4ee74 [libfuzzer] Use cl driver mode for tests and enable another test on Windows
Summary:
When targeting MSVC: compile using clang's cl driver mode (this is needed for
libfuzzer's exit_on_src_pos feature). Don't use -lstdc++ when linking,
it isn't needed and causes a warning.
On Windows: Fix exit_on_src_pos.test by making sure debug info isn't
overwritten during compilation of second binary by using .exe extension.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: aprantl, JDevlieghere

Differential Revision: https://reviews.llvm.org/D51757

llvm-svn: 341622
2018-09-07 01:11:31 +00:00
Kostya Serebryany 93ce8b24d5 [hwasan] make the print-memory-usage.c less agressive: do not assume that malloc can't happen before main
llvm-svn: 341615
2018-09-07 00:38:31 +00:00
Evgeniy Stepanov 5b332abd66 [hwasan] Fix malloc overflow detection.
Check size limit before rounding up, otherwise malloc((size_t)-1)
would happily allocate 0 bytes.

Steal a nice test case from scudo.

llvm-svn: 341612
2018-09-07 00:27:11 +00:00
Kostya Serebryany f9ec62cb3a [hwasan] change the thread list so that main_thread can also be removed
llvm-svn: 341610
2018-09-07 00:16:55 +00:00
Evgeniy Stepanov fe4808eba7 [hwasan] Remove allocator interceptors.
Summary:
When building without COMPILER_RT_HWASAN_WITH_INTERCEPTORS, skip
interceptors for malloc/free/etc and only export their versions with
__sanitizer_ prefix.

Also remove a hack in mallinfo() interceptor that does not apply to
hwasan.

Reviewers: kcc

Subscribers: kubamracek, krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D51711

llvm-svn: 341598
2018-09-06 22:53:08 +00:00
Kostya Serebryany f1f556c179 [hwasan] fix pthread_exit
llvm-svn: 341594
2018-09-06 22:13:43 +00:00
Kostya Serebryany 950a1a3936 [hwasan] introduce __hwasan_print_memory_usage
llvm-svn: 341592
2018-09-06 22:08:41 +00:00
Jonathan Metzman c239482207 [libfuzzer] Correct typo in comment (test commit).
Differential Revision: https://reviews.llvm.org/D51739

llvm-svn: 341569
2018-09-06 17:50:01 +00:00
Max Moroz a39da2f8ca [libfuzzer] Fix fuzzer-oom.test on windows and reenable it.
Summary: Patch by Jonathan Metzman (@metzman).

Reviewers: Dor1s, morehouse

Reviewed By: Dor1s

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D51735

llvm-svn: 341562
2018-09-06 16:48:18 +00:00
Alexander Potapenko 7f270fcf0a [MSan] store origins for variadic function parameters in __msan_va_arg_origin_tls
Add the __msan_va_arg_origin_tls TLS array to keep the origins for variadic function parameters.
Change the instrumentation pass to store parameter origins in this array.

This is a reland of r341528.

test/msan/vararg.cc doesn't work on Mips, PPC and AArch64 (because this
patch doesn't touch them), XFAIL these arches.
Also turned out Clang crashed on i80 vararg arguments because of
incorrect origin type returned by getOriginPtrForVAArgument() - fixed it
and added a test.

llvm-svn: 341554
2018-09-06 15:14:36 +00:00
Alexander Potapenko ac6595bd53 [MSan] revert r341528 to unbreak the bots
llvm-svn: 341541
2018-09-06 12:19:27 +00:00
Tim Northover 8f141016ee ARM: wrap call to __clzsi2 so that the name is correct on MachO.
MachO symbols are prefixed with an extra '_' (that's 3 in total for this
function), so assembly calls have to go through a wrapper to insert any prefix
needed.

llvm-svn: 341540
2018-09-06 12:13:46 +00:00
Alexander Potapenko 1a10ae0def [MSan] store origins for variadic function parameters in __msan_va_arg_origin_tls
Add the __msan_va_arg_origin_tls TLS array to keep the origins for
variadic function parameters.
Change the instrumentation pass to store parameter origins in this array.

llvm-svn: 341528
2018-09-06 08:50:11 +00:00
Max Moroz 48d1398b36 [libfuzzer] Temporarily disable unittests failing on Windows.
Summary:
Patch by Jonathan Metzman (@metzman).


Reviewers: morehouse, Dor1s

Reviewed By: Dor1s

Subscribers: kcc, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D51712

llvm-svn: 341514
2018-09-06 02:54:47 +00:00
Kostya Serebryany 68902c7050 [hwasan] when verbose_threads==1, print the memory usage per thread
llvm-svn: 341507
2018-09-05 23:52:31 +00:00
Kostya Serebryany b29d42ee31 [hwasan] simplify the code, NFC
llvm-svn: 341501
2018-09-05 23:22:38 +00:00
Evgeniy Stepanov 6afce6a438 [hwasan] Don't handle signals on Android.
Sigtrap is used for error reporting, but all other signals are better
left for the platform.

In particular, sanitizer signal handlers do not dump registers or
memory which makes debugging harder for no good reason.

llvm-svn: 341500
2018-09-05 22:46:19 +00:00
Matt Morehouse 02f3ad81c3 [libfuzzer] Replace memmem with strstr.
Summary: Memmem is not available on Windows.

Patch By: metzman

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: george.karpenkov, morehouse

Differential Revision: https://reviews.llvm.org/D51692

llvm-svn: 341495
2018-09-05 21:03:43 +00:00
Kostya Serebryany 3daf3e7072 [hwasan] deflake a test
llvm-svn: 341480
2018-09-05 16:09:53 +00:00
Max Moroz 651abaf536 Enable symbolize-deadlock.test on Windows
Summary:
Remove unneeded #include of <unistd.h> from SymbolizeDeadlock.cpp to
enable compilation on Windows and enable symbolize-deadlock.test on
Windows.

Patch by Jonathan Metzman (@metzman).

Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: kcc, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D51685

llvm-svn: 341473
2018-09-05 14:43:38 +00:00
David Carlier c3d1d107ff [Xray] Darwin providing defined value for weak symbols to fix linkage issue
- Temporary hack to make the buildbot failure stop on Darwin.

llvm-svn: 341445
2018-09-05 05:18:34 +00:00
Brian Cain 416296b8f7 Add glibc_prereq to platform limits mmsghdr
sendmmsg requires glibc >= 2.14.

Fixes PR38589.

Review: https://reviews.llvm.org/D51538
llvm-svn: 341442
2018-09-05 02:15:54 +00:00
Kostya Serebryany ee0e2f8323 [hwasan] revert r341435 as it breaks the bot on aarch64
llvm-svn: 341441
2018-09-05 01:29:08 +00:00
Kostya Serebryany c12cc9b85c [hwasan] fix colored output
llvm-svn: 341440
2018-09-05 01:27:48 +00:00
Kostya Serebryany 5d05be84b7 [hwasan] print thread IDs when reporting a bug (also had to fix pthread_create on Linux)
llvm-svn: 341438
2018-09-05 01:16:50 +00:00
Kostya Serebryany 9fbedcad71 [hwasan] use real TLS on linux to store the current thread -- this way we can call t->Destroy in __hwasan_thread_exit, same as on Android
llvm-svn: 341435
2018-09-05 00:17:23 +00:00
Kostya Serebryany 2768b52117 [hwasan] simplify the code, NFC
llvm-svn: 341432
2018-09-05 00:01:45 +00:00
Kostya Serebryany c551220de8 [hwasan] remove stale Thread:: data members. While doing so noticed that GetThreadStackAndTls was always called with 'at_initialization=true', fixed that.
llvm-svn: 341431
2018-09-04 23:57:09 +00:00
Kostya Serebryany 3d8f49a6bf [hwasan] add a unique id to a thread and add debug prints for thread creation/destruction
llvm-svn: 341428
2018-09-04 23:26:08 +00:00
Kostya Serebryany 470db78115 [sanitizer] optimize internal_memset for the most performance critical case (16-byte-aligned)
llvm-svn: 341420
2018-09-04 22:43:30 +00:00
Matt Morehouse 24568789c4 [libFuzzer] Enable tests on Windows
Summary:
Enable tests on Windows and make check-fuzzer pass on it.  Make
check-fuzzer pass on Windows by fixing libFuzzer, fixing tests, and by
disabling tests on Windows. Most of these are disabled temporarily as
support for the tests and the features they test will be added
incrementally.  Other tests will not be enabled since they require
things that are not on Windows (eg: afl_driver tests).  Every test
that was explicitly disabled on Windows has a comment explaining why
(unless obvious like merge-posix.test).

The lit.cfg file was modified to support running tests on windows.
fuzzer-dirs.test was fixed by making the Windows implementation print
the same error message as the posix version.
merge-control-file.test was fixed by making the test binary end with
the ".exe" extension (on all platforms).

Patch By: metzman

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: srhines, mgorny

Differential Revision: https://reviews.llvm.org/D51549

llvm-svn: 341385
2018-09-04 17:08:47 +00:00
Matt Morehouse facfb70d90 [TSan] Update assembly test again.
Previous commit incorrectly updated the read1 case.

llvm-svn: 341384
2018-09-04 17:04:30 +00:00
Matt Morehouse 876d382cf3 [TSan] Update assembly code check.
The new assembly makes our benchmark faster, so it should be safe to
update the check.

llvm-svn: 341381
2018-09-04 16:34:26 +00:00
Dan Liew 257f375f0c [asan] Clean up some confusing code in
`test/asan/TestCases/Darwin/segv_read_write.c`

* The `fd` arg passed to `mmap()` should be `-1`. It is not defined
what passing `0` does on Darwin.

* The comment about the shadow memory doesn't make any sense to me,
so I'm removing it.

Differential Revision: https://reviews.llvm.org/D44579

llvm-svn: 341307
2018-09-03 10:33:32 +00:00
Dan Liew 7294c0563c [UBSan] Add CMake and lit support for configuring and running UBSan
tests for ios, watchos, tvos, and their simulator counterparts.

This commit does not make the tests actually pass. This will be handled
in later commits.

rdar://problem/41126835

Differential Revision: https://reviews.llvm.org/D51270

llvm-svn: 341306
2018-09-03 10:30:10 +00:00
Dan Liew d96d5e9046 Give a better error message when trying to run the iossim tests and `SANITIZER_IOSSIM_TEST_DEVICE_IDENTIFIER` is not set in the environment.
Summary: Give a better error message when trying to run the iossim tests and `SANITIZER_IOSSIM_TEST_DEVICE_IDENTIFIER` is not set in the environment.

Reviewers: kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51272

llvm-svn: 341300
2018-09-03 08:40:19 +00:00
Dan Liew 5eb532bb5d [UBSan] Propagate `UBSAN_OPTIONS` environment variable when running ios simulator tests.
rdar://problem/41126835

Reviewers: kubamracek, vsk, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51273

llvm-svn: 341299
2018-09-03 08:37:42 +00:00
Dan Liew 4218a00697 [UBSan] Add missing `%run` prefixes to Pointer tests.
Summary: rdar://problem/41126835

Reviewers: vsk, kubamracek

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51271

llvm-svn: 341298
2018-09-03 08:33:24 +00:00
David Carlier fb58ea1b80 Fix buildbot test
llvm-svn: 341286
2018-09-02 10:52:43 +00:00
David Carlier e858aa6565 [Sanitizer] openbsd build fix
sysctl has a different signature under OpenBSD

Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D51572

llvm-svn: 341285
2018-09-02 09:08:50 +00:00
Evgeniy Stepanov 5abf7d90ac [hwasan] Fix new[] with zero size.
Fixes "allocator is out of memory trying to allocate 0x0 bytes" by
always allocating at least one byte.

llvm-svn: 341229
2018-08-31 17:49:49 +00:00
Kamil Rytarowski 43b15cddac Try to unbreak internal_sysctl() for MacOSX
Cast the 5th argument to (void *), removing cast.

llvm-svn: 341192
2018-08-31 09:35:33 +00:00
Kamil Rytarowski 1125e78045 Try to fix internal_sysctl() for MacOSX
Cast the first argument to (int *) removing const.

llvm-svn: 341187
2018-08-31 08:51:29 +00:00
Kamil Rytarowski 6c3f626378 Improve portability of internal_sysctl()
Add an explicit cast from uptr to size_t to prevent potential type mismatch.

llvm-svn: 341183
2018-08-31 08:24:23 +00:00
Kamil Rytarowski 935203306f Add internal_sysctl() used by FreeBSD, NetBSD, OpenBSD and MacOSX
Summary:
Switch local sysctl(2) calls to internal_sysctl().

This is a preparation for introduction of interceptors for
the sysctl*() family of functions and switching `internal_sysctl*()`
to libc calls bypassing interceptors.

No functional change intended with this revision.

Reviewers: vitalybuka, joerg, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D51425

llvm-svn: 341181
2018-08-31 08:10:06 +00:00
Kostya Serebryany f0965c3af4 [hwasan] move code around to remove an opaque class definition (HwasanThreadLocalMallocStorage) that had incorrect size and used 10x more than needed RAM (500K instead of 50K per thread)
llvm-svn: 341170
2018-08-31 06:08:48 +00:00
Kostya Serebryany bca94773b7 [hwasan] simplify the code, NFC
llvm-svn: 341166
2018-08-31 05:55:18 +00:00
Kostya Serebryany 60746d91c1 [hwasan] more heap-buffer-overflow tests
llvm-svn: 341162
2018-08-31 05:05:01 +00:00
Kostya Serebryany 53cb061b32 [hwasan] report heap-buffer-overflow location, once, not once per each live thread
llvm-svn: 341160
2018-08-31 03:29:09 +00:00
Kostya Serebryany 1e2ec57803 [hwasan] properly report heap-buffer-overflow
llvm-svn: 341159
2018-08-31 03:18:31 +00:00
Kostya Serebryany e775a3ee42 [hwasan] fix the build on Linux Ubuntu 14.04. We don't need __hwasan_shadow on Linux at all
llvm-svn: 341158
2018-08-31 02:38:23 +00:00
Kostya Serebryany f8be3fa5b6 [hwasan] make malloc(0) return nullptr, add basic address description for stack addresses
llvm-svn: 341156
2018-08-31 01:38:00 +00:00
Kostya Serebryany 347b989cef [hwasan] fix the linux-only pthread_create interceptor and reinstate the two threaded tests
llvm-svn: 341143
2018-08-30 23:22:26 +00:00
Kostya Serebryany e6507f02a0 [hwasan] use thread-local ring buffers to properly report heap-use-after-free
llvm-svn: 341133
2018-08-30 22:11:56 +00:00
Kostya Serebryany 8f0f36b83f [hwasan] temporarily remove two tests to silence the bots
llvm-svn: 341129
2018-08-30 22:07:09 +00:00
Kostya Serebryany c359d9b8fa [hwasan] simplify the thread hangling: instead of the ThreadRegistry (too heavy) simply maintain a linked list of Threads
llvm-svn: 341111
2018-08-30 20:15:39 +00:00
David Carlier bdab89b228 [Xray] Darwin fix variable typo
llvm-svn: 341090
2018-08-30 18:12:47 +00:00
Matt Morehouse 7e042bb1d1 [libFuzzer] Port to Windows
Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.

Patch By: metzman

Reviewers: morehouse, rnk

Reviewed By: morehouse, rnk

Subscribers: #sanitizers, delcypher, morehouse, kcc, eraman

Differential Revision: https://reviews.llvm.org/D51022

llvm-svn: 341082
2018-08-30 15:54:44 +00:00
David Carlier e4c372c4a4 [Xray] Darwin improving slightly the support
using sysctl to get the tic frequency data.
still linkage issue for X-ray_init not resolved.

Reviewers: dberris, kubamracek

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D51399

llvm-svn: 341019
2018-08-30 05:55:27 +00:00
Petr Hosek 6518929569 [sanitizer] Transition to new _zx_vmar_... calls
Now that all _zx_vmar_... calls have been updated, we can undo the
change made in r337801 and switch over to the new calls.

Differential Revision: https://reviews.llvm.org/D51468

llvm-svn: 341011
2018-08-30 01:27:26 +00:00
Kostya Serebryany a4132df0ee [hwasan] add a simple threaded UAF test, make it work on x86 (need to disable tagging in malloc with inside pthread_create)
llvm-svn: 341007
2018-08-30 00:44:55 +00:00
Kostya Serebryany d0cd2db23b [hwasan] add basic ThreadRegistry plumbing, also rename HwasanThread to Thread
llvm-svn: 341005
2018-08-30 00:13:20 +00:00
Kostya Serebryany a050a8da03 [hwasan] remove even more stale code
llvm-svn: 340989
2018-08-29 22:54:52 +00:00
Kostya Serebryany 87bc568eca [hwasan] remove more stale code
llvm-svn: 340985
2018-08-29 22:47:53 +00:00
Kostya Serebryany 1e4498869d [hwasan] get rid of some macros, remove the fixed shadow mapping
llvm-svn: 340983
2018-08-29 22:42:16 +00:00
Kostya Serebryany 5d26313ce0 [hwasan] formatting change, NFC
llvm-svn: 340980
2018-08-29 22:23:34 +00:00
Kostya Serebryany ab62b59a76 [hwasan] rename some variables and functions for better readability, NFC
llvm-svn: 340979
2018-08-29 22:21:22 +00:00
Max Moroz 8c95b48ba2 [libFuzzer] Remove mutation stats and weighted mutation selection.
Summary:
This was an experimental feature. After evaluating it with:

1) https://github.com/google/fuzzer-test-suite/tree/master/engine-comparison

2) enabling on real world fuzz targets running at ClusterFuzz and OSS-Fuzz

The following conclusions were made:

1) With fuzz targets that have reached a code coverage plateau, the feature does
   not improve libFuzzer's ability to discover new coverage and may actually
   negatively impact it.

2) With fuzz targets that have not yet reached a code coverage plateau, the
   feature might speed up new units discovery in some cases, but it is quite
   rare and hard to confirm with a high level on confidence.

Revert of https://reviews.llvm.org/D48054 and https://reviews.llvm.org/D49621.

Reviewers: metzman, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D51455

llvm-svn: 340976
2018-08-29 21:53:15 +00:00
Kostya Serebryany fd48b7d558 [hwasan] simplify the realloc implementation: always allocate/deallocate on realloc. This may slowdown some realloc-heavy code, but at least at this point a want simpler code. Also added a test
llvm-svn: 340973
2018-08-29 21:28:14 +00:00
Kostya Serebryany 51e1a83850 Add a thread-local ring buffer of heap allocations
Summary:
We need this in order to properly report heap-use-after-free,
since we don't have a quarantine.

This is a first part of the code, more like a proof of concept.
But I'd like to commit at as is and proceed with refactoring,
adding a ThreadRegistry, and extending the functionality.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51394

llvm-svn: 340971
2018-08-29 21:07:07 +00:00
Kostya Serebryany d709a36084 [sanitizer] reapply r340884 'Add a RingBuffer class to sanitizer_common', with fixes for Windows
llvm-svn: 340969
2018-08-29 21:00:01 +00:00
Kostya Kortchinsky 65e1bcf2b2 [sanitizer] Revert D50940
Summary:
The previous version of the patch makes some code unable to distinguish
failure to map address 0 and error.
Revert to turn the bots back to green while figuring out a new approach.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51451

llvm-svn: 340957
2018-08-29 19:41:28 +00:00
Matt Morehouse cf311cfc20 Revert "[libFuzzer] Port to Windows"
This reverts r340949 due to bot breakage again.

llvm-svn: 340954
2018-08-29 18:40:41 +00:00
Matt Morehouse 245ebd71ef [libFuzzer] Port to Windows
Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.

Reviewers: morehouse, rnk

Reviewed By: morehouse, rnk

Subscribers: #sanitizers, delcypher, morehouse, kcc, eraman

Differential Revision: https://reviews.llvm.org/D51022

llvm-svn: 340949
2018-08-29 18:08:34 +00:00
Hans Wennborg 61cbae90a5 Revert r340884 "Add a RingBuffer class to sanitizer_common"
The test doesn't pass on Windows, where sizeof(long) == 4 also
on 64-bit, and so it isn't a multiple of sizeof(void*).

This also reverts the follow-up r340886.

> Summary: a constrained RingBuffer optimized for fast push
>
> Reviewers: eugenis
>
> Reviewed By: eugenis
>
> Subscribers: kubamracek, mgorny, delcypher, #sanitizers, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D51196

llvm-svn: 340924
2018-08-29 12:40:29 +00:00
Kamil Rytarowski d1c1e036ae Introduce new type for inteceptors UINTMAX_T
Summary:
The UINTMAX_T type will be used in new interceptors.

While there, correct the type of strtoumax(3) from INTMAX_T to UINTMAX_T.

Original patch from Yang Zheng.

Reviewers: vitalybuka, kcc, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, tomsun.0.7, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D51106

llvm-svn: 340907
2018-08-29 09:11:17 +00:00
Kostya Serebryany 67d0488934 [sanitizer] fix a test
llvm-svn: 340886
2018-08-28 23:50:59 +00:00
Kostya Serebryany 66eefee7ed Add a RingBuffer class to sanitizer_common
Summary: a constrained RingBuffer optimized for fast push

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kubamracek, mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51196

llvm-svn: 340884
2018-08-28 23:32:56 +00:00
Matt Morehouse bab8556f01 Revert "[libFuzzer] Port to Windows"
This reverts commit r340860 due to failing tests.

llvm-svn: 340867
2018-08-28 19:07:24 +00:00
Matt Morehouse c6fff3b6f5 [libFuzzer] Port to Windows
Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.

Patch By: metzman

Reviewers: morehouse, rnk

Reviewed By: morehouse, rnk

Subscribers: morehouse, kcc, eraman

Differential Revision: https://reviews.llvm.org/D51022

llvm-svn: 340860
2018-08-28 18:34:32 +00:00
Vlad Tsyrklevich 03f89d3c5e SafeStack: Fix thread liveness check on *BSD
Summary:
The Linux/BSD system call interfaces report errors differently, use the
internal_iserror() function to correctly check errors on either.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: delcypher, llvm-commits, #sanitizers, krytarowski, kcc, devnexen

Differential Revision: https://reviews.llvm.org/D51368

llvm-svn: 340856
2018-08-28 18:30:03 +00:00
Dean Michael Berris 6b1e125db9 [XRay][compiler-rt] Remove uses of internal allocator in profiling mode
Summary:
This change removes further cases where the profiling mode
implementation relied on dynamic memory allocation. We're using
thread-local aligned (uninitialized) memory instead, which we initialize
appropriately with placement new.

Addresses llvm.org/PR38577.

Reviewers: eizan, kpw

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D51278

llvm-svn: 340814
2018-08-28 10:41:10 +00:00
Dean Michael Berris bb6aa92c31 [XRay][compiler-rt] Stash flags as well in x86_64 trampoline
Summary:
This change saves and restores the full flags register in x86_64 mode.
This makes running instrumented signal handlers safer, and avoids flags
set during the execution of the event handlers from polluting the
instrumented call's flags state.

Reviewers: kpw, eizan, jfb

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D51277

llvm-svn: 340812
2018-08-28 10:32:50 +00:00
Vitaly Buka dd4c14e4dd [lsan] Check that leak sanitizer works in the forked process
Regression test for PR38698

llvm-svn: 340769
2018-08-27 19:15:05 +00:00
Petr Hosek 118dc299f9 [sanitizer][fuzzer] Transition back to ZX_TIME_INFINITE
Now that all Zircon calls have been transitioned to take time as signed
value, we can transition back to ZX_TIME_INFINITE, undoing the change
made in r337802.

Differential Revision: https://reviews.llvm.org/D51266

llvm-svn: 340764
2018-08-27 17:51:52 +00:00
Vitaly Buka d833acdb5f Revert "[lsan] Do not check for leaks in the forked process"
Users need leak reports in forks.

This reverts commit r334036.

llvm-svn: 340758
2018-08-27 17:26:28 +00:00
Kostya Kortchinsky 9920489a2a [scudo] Replace eraseHeader with compareExchangeHeader for Quarantined chunks
Summary:
The reason for the existence of `eraseHeader` was that it was deemed faster
to null-out a chunk header, effectively making it invalid, rather than marking
it as available, which incurred a checksum computation and a cmpxchg.

A previous use of `eraseHeader` was removed with D50655 due to a race.

Now we remove the second use of it in the Quarantine deallocation path and
replace is with a `compareExchangeHeader`.

The reason for this is that greatly helps debugging some heap bugs as the chunk
header is now valid and the chunk marked available, as opposed to the header
being invalid. Eg: we get an invalid state error, instead of an invalid header
error, which reduces the possibilities. The computational penalty is negligible.

Reviewers: alekseyshl, flowerhack, eugenis

Reviewed By: eugenis

Subscribers: delcypher, jfb, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D51224

llvm-svn: 340633
2018-08-24 18:21:32 +00:00
David Carlier 60251a5eb7 [Sanitizer] implementing remaining function under OSX
- GetRandom and GetnumberOfCPUs using sys call for the former.
- enabling unit tests for the other oses.

Reviewers: kubamracek

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D50937

llvm-svn: 340621
2018-08-24 16:53:06 +00:00
Kostya Serebryany a7c3846a2e [hwasan] implement detection of realloc-after-free
llvm-svn: 340593
2018-08-24 01:44:17 +00:00
Kostya Serebryany c5f98d2ab2 [hwasan] implement detection of double-free (invalid-free)
llvm-svn: 340591
2018-08-24 01:12:26 +00:00
Kuba Mracek 9f690839d6 [sanitizer] Don't call task_for_pid(mach_task_self). NFC.
Calling task_for_pid with mach_task_self is just returning mach_task_self anyway, but it also triggers system warnings (task_for_pid is only supposed to be used by high-privileged executables). NFC.

rdar://problem/39198248

Differential Revision: https://reviews.llvm.org/D51119

llvm-svn: 340587
2018-08-23 22:55:58 +00:00
Kostya Kortchinsky 14b838a1ca [sanitizer] Change Mmap*NoAccess to return nullptr on error
Summary:
`MmapNoAccess` & `MmapFixedNoAccess` return directly the result of
`internal_mmap`, as opposed to other Mmap functions that return nullptr.

This inconsistency leads to some confusion for the callers, as some check for
`~(uptr)0` (`MAP_FAILED`) for failure (while it can fail with `-ENOMEM` for
example).

Two potential solutions: change the callers, or make the functions return
`nullptr` on failure to follow the precedent set by the other functions.
The second option looked more appropriate to me.

Correct the callers that were wrongly checking for `~(uptr)0` or
`MAP_FAILED`.

TODO for follow up CLs:
- There are a couple of `internal_mmap` calls in XRay that check for
  MMAP_FAILED as a result as well (cc: @dberris); they should use
  `internal_iserror`;


Reviewers: eugenis, alekseyshl, dberris, kubamracek

Reviewed By: alekseyshl

Subscribers: kristina, kubamracek, delcypher, #sanitizers, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D50940

llvm-svn: 340576
2018-08-23 21:13:39 +00:00
Kostya Serebryany e2efbbe571 [hwasan] make error reporting look more like in asan, print the memory tag around the buggy access, simplify one test
llvm-svn: 340470
2018-08-22 22:55:16 +00:00
Kostya Serebryany bb4cf6e1e1 [hwasan] remove stale data field
llvm-svn: 340442
2018-08-22 19:47:19 +00:00
Kuba Mracek 986f03c2ea [tsan] Adjust setjmp/longjmp handling on Darwin for macOS Mojave
On macOS Mojave, the OS started using the XOR-by-a-secret-key scheme (same as glibc is alread doing) for storing the SP value in setjmp environment. We need to adjust for that to keep supporting setjmp/longjmp on latest Darwin. The patch is basically doing the same what we're already doing for glibc.

rdar://problem/43542596

Differential Revision: https://reviews.llvm.org/D51064

llvm-svn: 340350
2018-08-21 22:35:52 +00:00
Kuba Mracek 22d1a2789a Fixup for r340342: Avoid Block_release'ing the block since we're no longer making a copy.
rdar://problem/42242579

llvm-svn: 340347
2018-08-21 22:03:28 +00:00
Kuba Mracek faef7d034a [tsan] Avoid calling Block_copy in the "sync" GCD interceptors
The synchronous dispatch functions in GCD (dispatch_sync, dispatch_barrier_sync), don't make a copy of the passed block. To maintain binary compatibility, we should avoid doing that as well in TSan, as there's no reason to do that. The synchronous dispatch functions will not return before the block is actually executed.

rdar://problem/42242579

Differential Revision: https://reviews.llvm.org/D50920

llvm-svn: 340342
2018-08-21 21:24:22 +00:00
Vlad Tsyrklevich f6c701054e SafeStack: Use correct unsafe stack sizes
Summary:
When deallocating thread stacks, we use one thread's unsafe stack size
to deallocate another thread's unsafe stack; however, the two sizes may
differ. Record an unsafe stack's size in the thread stack linked list.

Reviewers: pcc, eugenis

Reviewed By: eugenis

Subscribers: delcypher, llvm-commits, #sanitizers, kcc

Differential Revision: https://reviews.llvm.org/D51016

llvm-svn: 340308
2018-08-21 17:29:01 +00:00
Evgeniy Stepanov 4f0e10fff9 [hwasan] Add a (almost) no-interceptor mode.
Summary:
The idea behind this change is to allow sanitization of libc. We are prototyping on Bionic,
but the tool interface will be general enough (or at least generalizable) to support any other libc.

When libc depends on libclang_rt.hwasan, the latter can not interpose libc functions.
In fact, majority of interceptors become unnecessary when libc code is instrumented.

This change gets rid of most hwasan interceptors and provides interface for libc to notify
hwasan about thread creation and destruction events. Some interceptors (pthread_create)
are kept under #ifdef to enable testing with uninstrumented libc. They are expressed in
terms of the new libc interface.

The new cmake switch, COMPILER_RT_HWASAN_WITH_INTERCEPTORS, ON by default, builds testing
version of the library with the aforementioned pthread_create interceptor.
With the OFF setting, the library becomes more of a libc plugin.

Reviewers: vitalybuka, kcc, jfb

Subscribers: srhines, kubamracek, mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D50922

llvm-svn: 340216
2018-08-20 21:49:15 +00:00
Kostya Kortchinsky 7515e75bc2 [sanitizer] Use private futex operations for BlockingMutex
Summary:
Use `FUTEX_PRIVATE_FLAG` in conjunction with the wait & wake operations
employed by `BlockingMutex`. As far as I can tell, the mutexes are
process-private, and there is an actual performance benefit at employing the
private operations. There should be no downside to switching to it.

Reviewers: eugenis, alekseyshl, dvyukov

Reviewed By: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D50910

llvm-svn: 340178
2018-08-20 14:57:58 +00:00
Fangrui Song b2f8ba0dec [msan] Remove XFAIL: freebsd from test/msan/tls_reuse.cc
This passes now.

llvm-svn: 340132
2018-08-18 23:25:55 +00:00
Kuba Mracek 2b93dfe0ad [sanitizer] When setting up shadow memory on iOS, fix handling the return value of task_info on older OS versions
task_vm_info is a "revisioned" structure, new OS versions add fields to the end, and compatibility is based on the reported size. On older OS versions, min_address/max_address is not filled back. Let's handle that case. Unfortunately, we can't really write a test (as the failure only happens when on a specific OS version).

Differential Revision: https://reviews.llvm.org/D50275

llvm-svn: 340058
2018-08-17 17:53:14 +00:00