Commit Graph

10839 Commits

Author SHA1 Message Date
Dean Michael Berris f8d5969943 [XRay][compiler-rt] Update test to use similar structure
This is a follow-up to D50037.

llvm-svn: 338349
2018-07-31 04:47:37 +00:00
Dean Michael Berris 3bd20d4605 [XRay][compiler-rt] Profiling Mode: Include file header in buffers
Summary:
This change provides access to the file header even in the in-memory
buffer processing. This allows in-memory processing of the buffers to
also check the version, and the format, of the profile data.

Reviewers: eizan, kpw

Reviewed By: eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D50037

llvm-svn: 338347
2018-07-31 04:16:54 +00:00
Reid Kleckner cae1b9fef2 Pacify sanitizer lint script that still does not run on Windows
llvm-svn: 338334
2018-07-31 00:08:26 +00:00
Reid Kleckner a5ed43c1c9 [asan/win] Use SRW locks to fix a race in BlockingMutex
Summary:
Before my change, BlockingMutex used Windows critial sections. Critical
sections can only be initialized by calling InitializeCriticalSection,
dynamically.

The primary sanitizer allocator expects to be able to reinterpret zero
initialized memory as a BlockingMutex and immediately lock it.
RegionInfo contains a mutex, and it placement new is never called for
it. These objects are accessed via:
  RegionInfo *GetRegionInfo(uptr class_id) const {
    DCHECK_LT(class_id, kNumClasses);
    RegionInfo *regions = reinterpret_cast<RegionInfo *>(SpaceEnd());
    return &regions[class_id];
  }
The memory comes from the OS without any other initialization.

For various reasons described in the comments, BlockingMutex::Lock would
check if the object appeared to be zero-initialized, and it would lazily
call the LinkerInitialized constructor to initialize the critical
section. This pattern is obviously racy, and the code had a bunch of
FIXMEs about it.

The best fix here is to use slim reader writer locks, which can start
out zero-initialized. They are available starting in Windows Vista. I
think it's safe to go ahead and use them today.

Reviewers: kcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D49893

llvm-svn: 338331
2018-07-30 23:32:33 +00:00
Roman Lebedev f07c5a0e9c [NFC] Rename test/ubsan/TestCases/{ImplicitCast => ImplicitConversion}
Just to be consistent with the rest.
I should have done that in the commit itself, but the filepaths
is one thing i forgot to verify :S

llvm-svn: 338307
2018-07-30 21:11:34 +00:00
Roman Lebedev 0c234517fd [compiler-rt] integer-truncation-blacklist.c: XFAIL on android/ios
The Builder sanitizer-x86_64-linux-android is failing
starting with rL338287 / D48959.

It runs the tests via android_compile.py, so i'm not sure this
is actually *this* issue:
  https://code.google.com/p/address-sanitizer/issues/detail?id=316
but this seems oddly similar to the other XFAIL'ed cases...

Right now that seems to be the only failing builder,
so i *think* it makes sense to try to just blacklist it for now.

llvm-svn: 338296
2018-07-30 20:05:24 +00:00
Roman Lebedev eb4a9bc343 [compiler-rt][ubsan] Implicit Conversion Sanitizer - integer truncation - compiler-rt part
Summary:
This is a compiler-rt part.
The clang part is D48958.

See [[ https://bugs.llvm.org/show_bug.cgi?id=21530 | PR21530 ]], https://github.com/google/sanitizers/issues/940.

Reviewers: #sanitizers, samsonov, vsk, rsmith, pcc, eugenis, kcc, filcab

Reviewed By: #sanitizers, vsk, filcab

Subscribers: llvm-commits, eugenis, filcab, kubamracek, dberris, #sanitizers, regehr

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D48959

llvm-svn: 338287
2018-07-30 18:58:30 +00:00
Azharuddin Mohammed ec6bdf8d4f [cmake] [ARM] Exclude any VFP builtins if VFP is not supported
Summary:
rL325492 disables FPU features when using soft floating point
(-mfloat-abi=soft), which is used internally when building for arm. This causes
errors with builtins that utililize VFP instructions.

With this change we check if VFP is enabled (by checking if the preprocessor
macro __VFP_FP__ is defined), and exclude such builtins if it is not enabled.

Reviewers: rengolin, samsonov, compnerd, smeenai, javed.absar, peter.smith

Reviewed By: peter.smith

Subscribers: delcypher, peter.smith, mgorny, kristof.beyls, chrib, llvm-commits

Differential Revision: https://reviews.llvm.org/D47217

llvm-svn: 338284
2018-07-30 18:18:59 +00:00
Dean Michael Berris f1ceb0b376 [XRay][compiler-rt] FDR Mode: Use mmap instead of internal allocator
Summary:
This change moves FDR mode to use `internal_mmap(...)` from
sanitizer_common instead of the internal allocator interface. We're
doing this to sidestep the alignment issues we encounter with the
`InternalAlloc(...)` functions returning pointers that have some magic
bytes at the beginning.

XRay copies bytes into the buffer memory, and does not require the magic
bytes tracking the other sanitizers use when allocating/deallocating
buffers.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49972

llvm-svn: 338228
2018-07-30 05:56:42 +00:00
Fangrui Song 8c40e40a85 [asan] Fix typo
llvm-svn: 338225
2018-07-30 00:25:16 +00:00
Fangrui Song c0ca8089a2 Fix Asan-i386-calls-Test AddressSanitizer.ShadowGapTest on FreeBSD
0x22000000 happens to be on the left of a heap allocation and the error
message is different (heap-buffer-overflow).
FreeBSD NetBSD have larger SHADOW_OFFSET (0x40000000) but let's try not
using #ifdef here.

llvm-svn: 338208
2018-07-28 23:41:50 +00:00
Jonas Hahnfeld a7c9fe3762 [test] Use printf instead of C++ iostream, NFC.
This test fails with libc++ when built with MemorySanitizer. This
is because we link to an uninstrumented version of the library
so msan detects a nested error when calling std::cout << "...".
This can be easily avoided by using good old printf.

Differential Revision: https://reviews.llvm.org/D49867

llvm-svn: 338053
2018-07-26 18:23:40 +00:00
Dmitry Vyukov 04f97cf2f0 [tsan] Fix gcc pedantic warning
Fix gcc (7.2.0) pedantic warning
warning: extra ‘;’ [-Wpedantic]

Author: jasonl220 (Jason Lovett)
Review: https://reviews.llvm.org/D49817
llvm-svn: 338023
2018-07-26 13:02:54 +00:00
Fangrui Song cb1107ed14 [sanitizer] Include signal.h instead of sys/signal.h
llvm-svn: 338004
2018-07-26 04:50:33 +00:00
Max Moroz f55b8c6e91 [libFuzzer] Disable handle-unstable.test for ARM and delete duplicated test file.
Summary:
This change should fix the failures mentioned in
https://reviews.llvm.org/D49684#1175245

Reviewers: kevinwkt, morehouse, metzman

Reviewed By: kevinwkt, morehouse

Subscribers: kristof.beyls, delcypher, chrib, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49810

llvm-svn: 337949
2018-07-25 18:34:20 +00:00
Jeremy Morse 63c153edb2 [XRay tests] Don't filter test-critical calls
By default, xray filters events that takes less than 5uS from its log.
In this existing test, should printf complete very quickly this will
lead to test-critical function calls being filtered (i.e. print_parent_tid).
Given that we're not testing the filtering feature, disable it for this
test.

llvm-svn: 337929
2018-07-25 14:48:53 +00:00
David Carlier 62e06ff583 [Asan][Msan] Unit tests Disable some tests for FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D49784

llvm-svn: 337913
2018-07-25 12:38:23 +00:00
Petr Hosek 47e5fcba57 [profile] Support profiling runtime on Fuchsia
This ports the profiling runtime on Fuchsia and enables the
instrumentation. Unlike on other platforms, Fuchsia doesn't use
files to dump the instrumentation data since on Fuchsia, filesystem
may not be accessible to the instrumented process. We instead use
the data sink to pass the profiling data to the system the same
sanitizer runtimes do.

Differential Revision: https://reviews.llvm.org/D47208

llvm-svn: 337881
2018-07-25 03:01:35 +00:00
Matt Morehouse 5377b5b275 [sanitizer] Update symbolizer test.
llvm-svn: 337872
2018-07-24 23:50:42 +00:00
Max Moroz 8db0befc6d [libFuzzer] Handle unstable edges by disregarding unstable edges
Summary:
Added a new mode within flag -handle_unstable for new unstable handling algorithm that does the following:
    When an edge is shown as unstable, copy to UnstableCounters the value 0.
    During ApplyUnstableCounters we copy back the value 0 to ModuleInline8bitCounters if the edge was unstable.

This way we would be ignoring completely features that were collected through non-determinism.
Unstable hits would be counted as if it never hit.

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49684

llvm-svn: 337853
2018-07-24 21:02:44 +00:00
Kuba Mracek dc36389ea8 [tsan] Fix crash in objc_sync_enter/objc_sync_exit when using an Obj-C tagged pointer
Objective-C tagged pointers (either bottom-most or top-most bit is 1) are valid Obj-C objects but are not valid pointers. Make sure we don't crash on them when used in objc_sync_enter/objc_sync_exit. Instead, let's synchronize on a global object.

Differential Revision: https://reviews.llvm.org/D49707

llvm-svn: 337837
2018-07-24 16:19:06 +00:00
Igor Kudrin 356c2aeffe [profile] Fix finding the first and last directory separators on Windows.
Until now, our code preferred backslashes to slashes, whereas Windows
allows using both types of directory separators in one path string.

Differential Revision: https://reviews.llvm.org/D49664

llvm-svn: 337826
2018-07-24 13:06:19 +00:00
Igor Kudrin 63600c7487 [profile] Fix a possible memory leak in parseFilenamePattern().
Differential Revision: https://reviews.llvm.org/D49666

llvm-svn: 337823
2018-07-24 12:28:53 +00:00
Petr Hosek b8ab7e811b [sanitizer][fuzzer] Temporarily transition to ZX_TIME_INFINITE_OLD
This is a preparation for breaking change when all Zircon calls that
take time as an argument will start using signed valued. We will
transition back to ZX_TIME_INFITINE after all the changes to these
symbols are done and become part of the Fuchsia SDK.

Differential Revision: https://reviews.llvm.org/D49694

llvm-svn: 337802
2018-07-24 02:34:24 +00:00
Petr Hosek e2da642697 [sanitizer] Transition from _zx_vmar_... to _zx_vmar_..._old calls
This is a preparation for breaking changes to _zx_vmar_... calls.
We will transition back to _zx_vmar_... after all the changes to
these symbols are done and become part of the Fuchsia SDK.

Differential Revision: https://reviews.llvm.org/D49697

llvm-svn: 337801
2018-07-24 02:28:54 +00:00
Petr Hosek c14d513e0d Revert "[Fuzzer] Update path to libc++ headers"
This reverts commit r337775 since r337727 has been reverted in r337782.

llvm-svn: 337784
2018-07-24 00:34:55 +00:00
Petr Hosek 2cf5d81f04 [Fuzzer] Update path to libc++ headers
The path to headers which are installed into libc++ build directory
has changed in r337727 which broke the libFuzzer build.

Differential Revision: https://reviews.llvm.org/D49705

llvm-svn: 337775
2018-07-23 23:38:28 +00:00
Matthew Voss 1486d5c6e2 Reapply "[XRay] Remove scheduling dependency in fork_basic_logging.cc"
Summary:
Continuation of https://reviews.llvm.org/D49501

Second part of the test has an scheduling order when there shouldn't be.



Reviewers: dberris, ormris

Reviewed By: dberris, ormris

Subscribers: TWeaver

Differential Revision: https://reviews.llvm.org/D49559

llvm-svn: 337745
2018-07-23 21:22:02 +00:00
Max Moroz 1e954f78d1 [libFuzzer] Handle unstable edges by using minimum hit counts
Summary:
Created unstable_handle flag that takes 1 or 2, depending on the handling type.
Modified RunOne to accommodate the following heuristic:
    Use the first CollectFeatures to count how many features there are.
    If no new features, CollectFeatures like before.
    If there is new feature, we run CB 2 more times,
        Check which edges are unstable per input and we store the least amount of hit counts for each edge.
        Apply these hit counts back to inline8bitcounters so that CollectFeatures can work as intended.
Modified UnstableCounters to 8int_t and created a bitset UnstableSet to tell which edges are unstable.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, morehouse

Reviewed By: Dor1s, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49525

llvm-svn: 337696
2018-07-23 14:20:52 +00:00
Peter Collingbourne acf005676e Change the cap on the amount of padding for each vtable to 32-byte (previously it was 128-byte)
We tested different cap values with a recent commit of Chromium. Our results show that the 32-byte cap yields the smallest binary and all the caps yield similar performance.
Based on the results, we propose to change the cap value to 32-byte.

Patch by Zhaomo Yang!

Differential Revision: https://reviews.llvm.org/D49405

llvm-svn: 337622
2018-07-20 21:43:20 +00:00
H.J. Lu 0cb55919ec Mark REAL(swapcontext) with indirect_return attribute on x86
When shadow stack from Intel CET is enabled, the first instruction of all
indirect branch targets must be a special instruction, ENDBR.

lib/asan/asan_interceptors.cc has

...
  int res = REAL(swapcontext)(oucp, ucp);
...

REAL(swapcontext) is a function pointer to swapcontext in libc.  Since
swapcontext may return via indirect branch on x86 when shadow stack is
enabled, as in this case,

int res = REAL(swapcontext)(oucp, ucp);
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  This function may be
returned via an indirect branch.

Here compiler must insert ENDBR after call, like

call *bar(%rip)
endbr64

I opened an LLVM bug:

https://bugs.llvm.org/show_bug.cgi?id=38207

to add the indirect_return attribute so that it can be used to inform
compiler to insert ENDBR after REAL(swapcontext) call.  We mark
REAL(swapcontext) with the indirect_return attribute if it is available.

This fixed:

https://bugs.llvm.org/show_bug.cgi?id=38249

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D49608

llvm-svn: 337603
2018-07-20 19:24:11 +00:00
Dan Liew c358e51e9b On Darwin switch from the `VM_MEMORY_ANALYSIS_TOOL` VM tag to
`VM_MEMORY_SANITIZER`.

It turns out that `VM_MEMORY_ANALYSIS_TOOL` is already reserved for
use by other tools so switch to a tag reserved for use by the Sanitizers.

rdar://problem/41969783

Differential Revision: https://reviews.llvm.org/D49603

llvm-svn: 337579
2018-07-20 17:07:35 +00:00
Kostya Kortchinsky cccd21d42c [scudo] Simplify internal names (NFC)
Summary:
There is currently too much redundancy in the class/variable/* names in Scudo:
- we are in the namespace `__scudo`, so there is no point in having something
  named `ScudoX` to end up with a final name of `__scudo::ScudoX`;
- there are a lot of types/* that have `Allocator` in the name, given that
  Scudo is an allocator I figure this doubles up as well.

So change a bunch of the Scudo names to make them shorter, less redundant, and
overall simpler. They should still be pretty self explaining (or at least it
looks so to me).

The TSD part will be done in another CL (eg `__scudo::ScudoTSD`).

Reviewers: alekseyshl, eugenis

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49505

llvm-svn: 337557
2018-07-20 15:07:17 +00:00
Dmitry Vyukov 97cf5f7f40 esan: fix shadow setup
r337531 changed return type of MmapFixedNoReserve, but esan wasn't updated.
As the result esan shadow setup always fails.
We probably need to make MmapFixedNoAccess signature consistent
with MmapFixedNoReserve. But this is just to unbreak tests.
 

llvm-svn: 337550
2018-07-20 13:40:08 +00:00
David Carlier 12be7b7bf7 [Xray] fix c99 warning build about flexible array semantics
Reviewers: dberris

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D49590

llvm-svn: 337536
2018-07-20 09:22:22 +00:00
Dmitry Vyukov f52726aae9 sanitizers: consistently check result of MmapFixedNoReserve
MmapFixedNoReserve does not terminate process on failure.
Failure to check its result and die will always lead to harder
to debug crashes later in execution. This was observed in Go
processes due to some address space conflicts.

Consistently check result of MmapFixedNoReserve.
While we are here also add warn_unused_result attribute
to prevent such bugs in future and change return type to bool
as that's what all callers want.

Reviewed in https://reviews.llvm.org/D49367

llvm-svn: 337531
2018-07-20 08:33:41 +00:00
Joel E. Denny a47cb644c7 [libFuzzer] Use separate test directory for each config
Previously, check-all failed many tests for me.  It was running the
X86_64DefaultLinuxConfig, X86_64LibcxxLinuxConfig, and
X86_64StaticLibcxxLinuxConfig configs out of
llvm-build/projects/compiler-rt/test/fuzzer.  Now, it runs them out of
separate subdirectories there, and most tests pass.

Reviewed By: morehouse, george.karpenkov

Differential Revision: https://reviews.llvm.org/D49249

llvm-svn: 337521
2018-07-20 02:39:01 +00:00
Kostya Serebryany 44edc281d9 [libFuzzer] when -print_coverage=1 is given, print more stats (the number of seeds that hit every given function)
llvm-svn: 337501
2018-07-19 22:00:48 +00:00
Teresa Johnson 408f50dbb4 [profile] Fix typo in test
Noticed that this was causing a compiler warning in the test.

llvm-svn: 337493
2018-07-19 20:18:37 +00:00
Teresa Johnson 73053b221f [profile] Add interface to get profile filename
Summary:
Add __llvm_profile_get_filename interface to get the profile filename,
which can be used for identifying which profile file belongs to an app
when multiple binaries are instrumented and dumping profiles into the
same directory. The filename includes the path.

Reviewers: davidxl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49529

llvm-svn: 337482
2018-07-19 19:03:50 +00:00
Teresa Johnson d4143bc772 [compiler-rt] Only set lto_flags if lto_supported
Fix bot failure from r37465. Move the new lto_flags append under the
check for whether lto_supported. Otherwise TestingConfig may not have
that member.

llvm-svn: 337467
2018-07-19 16:12:15 +00:00
Teresa Johnson 293b0f43ac [compiler-rt] Add NewPM testing to CFI tests
Summary:
Executes both LTO and ThinLTO CFI tests an additional time using the new
pass manager. I only bothered to add with gold and not lld as testing
with one linker should be sufficient. I didn't add for APPLE or WIN32
since I don't have a way to test those.

Depends on D49429.

Reviewers: pcc

Subscribers: dberris, mgorny, mehdi_amini, delcypher, dexonsmith, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49432

llvm-svn: 337465
2018-07-19 15:32:48 +00:00
Dean Michael Berris 41cea19b4f [XRay][compiler-rt] Profiling: No files when empty
This change makes it so that the profiling mode implementation will only
write files when there are buffers to write. Before this change, we'd
always open a file even if there were no profiles collected when
flushing.

llvm-svn: 337443
2018-07-19 09:20:19 +00:00
Dean Michael Berris e0099f1c6d [XRay][compiler-rt] FDR Mode: Add extents metadata to buffer
When providing raw access to the FDR mode buffers, we used to not
include the extents metadata record. This oversight means that
processing the buffers in-memory will lose important information that
would have been written in files.

This change exposes the metadata record by serializing the data
similarly to how we would do it when flushing to files.

llvm-svn: 337441
2018-07-19 05:17:32 +00:00
David Carlier cfc1d1d46e [Xray] Fix allocator build, MAP_NORESERVE flag is not always supported
MAP_NORESERVE is not supported or a no-op on BSD.

Reviewers: dberris

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D49494

llvm-svn: 337440
2018-07-19 05:08:59 +00:00
Kostya Serebryany 7a6295f2e1 [libFuzzer] fix the bot (the no-assert build)
llvm-svn: 337437
2018-07-19 03:16:12 +00:00
Kostya Serebryany 79bd14772f [libFuzzer] quick hack to fix the bot
llvm-svn: 337436
2018-07-19 01:54:28 +00:00
Kostya Serebryany 6b87e0c18f [libFuzzer] first experimental attempt at DFT-based mutations (DFT=data-flow-trace)
llvm-svn: 337434
2018-07-19 01:23:32 +00:00
Matthew Voss ac1f9263d1 Remove scheduling dependency from XRay :: Posix/fork_basic_logging.cc
Summary:
We've been seeing intermittent failures on our internal bots and we suspect
this may be due to the OS scheduling the child process to run before the parent
process.

This version ensures that the parent and child can be run in either order.

Reviewers: Maknee, dberris

Reviewed By: dberris

Subscribers: delcypher, #sanitizers, Maknee, llvm-commits

Differential Revision: https://reviews.llvm.org/D49501

llvm-svn: 337432
2018-07-19 00:25:00 +00:00
Peter Collingbourne 4a653fa7f1 Rename __asan_gen_* symbols to ___asan_gen_*.
This prevents gold from printing a warning when trying to export
these symbols via the asan dynamic list after ThinLTO promotes them
from private symbols to external symbols with hidden visibility.

Differential Revision: https://reviews.llvm.org/D49498

llvm-svn: 337428
2018-07-18 22:23:14 +00:00
Petr Hosek 4915d3a1ec [Fuzzer] Improve crash unwinding on Fuchsia
Fuchsia doesn't have signals; instead it expects processes to have a
dedicated exception thread that binds to the process' exception port and
waits for exception packets to be delivered. On the other hand,
libFuzzer and sanitizer_common use expect to collect crash information
via libunwind from the same thread that caused the exception.

The long term fix is to improve support for remote unwinding in
libunbwind, plumb this through sanitizer_common and libFuzzer, and
handle the exception exclusively on the exception thread. In the
meantime, this revision has the exception thread "resurrect" the
crashing thread by:

* saving its general purpose register state onto the crashing thread's
  stack,
* setting the crashing thread's program counter to an assembly trampoline
  with the CFI information needed by libunwind, and
* resuming the crashed thread.

Patch By: aarongreen

Differential Revision: https://reviews.llvm.org/D48509

llvm-svn: 337418
2018-07-18 19:20:47 +00:00
Max Moroz 42b54e8117 [libFuzzer] Create single template for visiting Inline8bitCounters
Summary:
Created IterateInline8bitCounters, a single template for visiting  Inline8bitCounters (nested for loop)
Made InitializeUnstableCounters and UpdateUnstableCounters both send a lambda to IterateInline8bitCounters.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49453

llvm-svn: 337403
2018-07-18 17:03:27 +00:00
Dean Michael Berris 4719c52455 [XRay][compiler-rt] Segmented Array: Simplify and Optimise
Summary:
This is a follow-on to D49217 which simplifies and optimises the
implementation of the segmented array. In this patch we co-locate the
book-keeping for segments in the `__xray::Array<T>` with the data it's
managing. We take the chance in this patch to actually rename `Chunk` to
`Segment` to better align with the high-level description of the
segmented array.

With measurements using benchmarks landed in D48879, we've identified
that calls to `pthread_getspecific` started dominating the cycles, which
led us to revert the change made in D49217 to use C++ thread_local
initialisation instead (it reduces the cost by a huge margin, since we
save one PLT-based call to pthread functions in the hot path). In
particular, this is in `__xray::getThreadLocalData()`.

We also took the opportunity to remove the least-common-multiple based
calculation and instead pack as much data into segments of the array.
This greatly simplifies the API of the container which hides as much of
the implementation details as possible. For instance, we calculate the
number of elements we need for the each segment internally in the Array
instead of making it part of the type.

With the changes here, we're able to get a measurable improvement on the
performance of profiling mode on top of what D48879 already provides.

Depends on D48879.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49363

llvm-svn: 337343
2018-07-18 02:08:39 +00:00
Dean Michael Berris 9d6b7a5f2b [XRay][compiler-rt] Simplify Allocator Implementation
Summary:
This change simplifies the XRay Allocator implementation to self-manage
an mmap'ed memory segment instead of using the internal allocator
implementation in sanitizer_common.

We've found through benchmarks and profiling these benchmarks in D48879
that using the internal allocator in sanitizer_common introduces a
bottleneck on allocating memory through a central spinlock. This change
allows thread-local allocators to eliminate contention on the
centralized allocator.

To get the most benefit from this approach, we also use a managed
allocator for the chunk elements used by the segmented array
implementation. This gives us the chance to amortize the cost of
allocating memory when creating these internal segmented array data
structures.

We also took the opportunity to remove the preallocation argument from
the allocator API, simplifying the usage of the allocator throughout the
profiling implementation.

In this change we also tweak some of the flag values to reduce the
amount of maximum memory we use/need for each thread, when requesting
memory through mmap.

Depends on D48956.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49217

llvm-svn: 337342
2018-07-18 01:53:39 +00:00
Dean Michael Berris 1e3feb49e3 [XRay][compiler-rt] FDR Mode: Allow multiple runs
Summary:
Fix a bug in FDR mode which didn't allow for re-initialising the logging
in the same process. This change ensures that:

- When we flush the FDR mode logging, that the state of the logging
  implementation is `XRAY_LOG_UNINITIALIZED`.

- Fix up the thread-local initialisation to use aligned storage and
  `pthread_getspecific` as well as `pthread_setspecific` for the
  thread-specific data.

- Actually use the pointer provided to the thread-exit cleanup handling,
  instead of assuming that the thread has thread-local data associated
  with it, and reaching at thread-exit time.

In this change we also have an explicit test for two consecutive
sessions for FDR mode tracing, and ensuring both sessions succeed.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49359

llvm-svn: 337341
2018-07-18 01:31:30 +00:00
Max Moroz 061b4af998 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337324
2018-07-17 20:37:40 +00:00
Martin Storsjo 946a81f5b0 [builtins] Implement the __chkstk function for ARM for MinGW
This function is available for linking in from kernel32.dll, but
it's not allowed to link that function from there in Windows Store
apps.

Differential Revision: https://reviews.llvm.org/D49055

llvm-svn: 337313
2018-07-17 19:14:47 +00:00
Matt Morehouse 43a2296976 libFuzzer: prevent irrelevant strings from leaking into auto-dictionary
This is a fix for bug 37047.

https://bugs.llvm.org/show_bug.cgi?id=37047

Implemented by basically reversing the logic. Previously all strings
were considered, with some operations excluded. Now strings are excluded
by default, and only strings during the CB considered.

Patch By: pdknsk

Differential Revision: https://reviews.llvm.org/D48800

llvm-svn: 337296
2018-07-17 16:12:00 +00:00
Matt Morehouse ef521ffe24 [libFuzzer] Avoid STL in MSan test.
Summary:
STL can cause MSan false positives if lib[std]c++ isn't instrumented
with MSan.

Reviewers: kcc

Reviewed By: kcc

Subscribers: Dor1s, llvm-commits

Differential Revision: https://reviews.llvm.org/D49404

llvm-svn: 337224
2018-07-16 23:22:54 +00:00
Max Moroz 5697c59c7f Revert r337194 (https://reviews.llvm.org/D48891) due to compilation errors.
llvm-svn: 337206
2018-07-16 20:05:18 +00:00
Alex Lorenz 88feedd7c7 [cmake][libFuzzer] fixup r337193 to ensure msan/dfsan are not added
to test deps for libfuzzer when they're not supported by the platform

llvm-svn: 337203
2018-07-16 19:41:49 +00:00
Petr Hosek b05be5f4b4 [CMake] Use cxx-headers as a depedency for C++ headers
We no longer pass CLANG_DEFAULT_CXX_STDLIB to the runtimes build
as it was causing issues so we can no longer use this variable. We
instead use cxx-headers as a dependency whenever this is available
since both XRay and libFuzzer are built as static libraries so this
is sufficient.

Differential Revision: https://reviews.llvm.org/D49346

llvm-svn: 337199
2018-07-16 18:48:20 +00:00
Max Moroz 8a5083df53 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337194
2018-07-16 17:50:46 +00:00
Matt Morehouse 0876a889f7 [libFuzzer] Add msan and dfsan to test deps.
Required now that we have tests using MSan and DFSan.

llvm-svn: 337193
2018-07-16 17:37:15 +00:00
Max Moroz 08dad54924 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337187
2018-07-16 16:01:31 +00:00
Max Moroz 1d369a5d01 Revert r337175 (https://reviews.llvm.org/D49212) due to unintentional format changes.
llvm-svn: 337180
2018-07-16 15:15:34 +00:00
Chandler Carruth 4abbd13ab2 Fix warnings in a static assert added to CFI.
llvm-svn: 337178
2018-07-16 15:01:26 +00:00
Henry Zhu ebc68ff65f Test commit
llvm-svn: 337176
2018-07-16 14:54:29 +00:00
Max Moroz 2156d885e0 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337175
2018-07-16 14:54:23 +00:00
Marco Castelluccio 1f0b194b4a [gcov] Add a test showing differences in line counts when building with or without exceptions enabled.
Test for https://bugs.llvm.org/show_bug.cgi?id=38066.

llvm-svn: 337174
2018-07-16 14:40:33 +00:00
Filipe Cabecinhas 3bc3fb477a [cfi] Don't pass a uint16_t to memset. Make sure the 16-bit constant is appropriate for us.
Reviewers: eugenis, pcc, kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49252

llvm-svn: 337170
2018-07-16 13:41:54 +00:00
Marco Castelluccio 0a75de4bfe Add a test with __gcov_flush called before terminating the program.
Test for https://bugs.llvm.org/show_bug.cgi?id=38067.

llvm-svn: 337150
2018-07-16 09:13:46 +00:00
Dean Michael Berris 734e3928b3 [XRay][compiler-rt] Use `SANITIZER_CXX_ABI_LIBRARY` for XRay unit tests
Summary:
Fix a TODO in CMake config for XRay tests to use the detected C++ ABI
library in the tests.

Also make the tests depend on the llvm-xray target when built in-tree.

Reviewers: kpw, eizan

Reviewed By: eizan

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D49358

llvm-svn: 337142
2018-07-16 01:08:56 +00:00
Petr Hosek d533fae6ef [CMake] Change the flag to use compiler-rt builtins to boolean
This changes the name and the type to what it was prior to r333037
which matches the name of the flag used in other runtimes: libc++,
libc++abi and libunwind. We don't need the type to be a string since
there's only binary choice between libgcc and compiler-rt unlike in
the case of C++ library where there're multiple options.

Differential Revision: https://reviews.llvm.org/D49325

llvm-svn: 337116
2018-07-15 03:05:20 +00:00
Max Moroz 038771a25a [UBSan] Followup for silence_unsigned_overflow flag to handle negate overflows.
Summary:
That flag has been introduced in https://reviews.llvm.org/D48660 for
suppressing UIO error messages in an efficient way. The main motivation is to
be able to use UIO checks in builds used for fuzzing as it might provide an
interesting signal to a fuzzing engine such as libFuzzer.

See https://github.com/google/oss-fuzz/issues/910 for more information.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49324

llvm-svn: 337068
2018-07-13 22:49:06 +00:00
Vlad Tsyrklevich ddf998b608 Fix for Darwin buildbot failure due to r337037
Duplicate __get_unsafe_stack_bottom instead of using an alias for
platforms that don't suppport it like Darwin.

llvm-svn: 337044
2018-07-13 20:56:48 +00:00
Vlad Tsyrklevich be9a9fd3dd SafeStack: Add builtins to read unsafe stack top/bottom
Summary:
Introduce built-ins to read the unsafe stack top and bottom. The unsafe
stack top is required to implement garbage collection scanning for
Oilpan. Currently there is already a built-in 'get_unsafe_stack_start'
to read the bottom of the unsafe stack, but I chose to duplicate this
API because 'start' is ambiguous (e.g. Oilpan uses WTF::GetStackStart to
read the safe stack top.)

Reviewers: pcc

Reviewed By: pcc

Subscribers: llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49152

llvm-svn: 337037
2018-07-13 19:48:35 +00:00
Petr Hosek 2d46cfedf2 [CMake] Don't use CLANG_DEFAULT_* values
These are not necessarily correct, just because Clang is configured
to use libc++ or compiler-rt as defaults doesn't mean that the host
compiler uses these as defaults as well. A more correct solution
would be to test the host compiler to check what the default actually
is as attempted in D46857.

Part of PR-38025.

Differential Revision: https://reviews.llvm.org/D49272

llvm-svn: 337033
2018-07-13 19:16:30 +00:00
Kostya Kortchinsky 4a547fb62d [sanitizer] Use -Wl,-z,global on Android
Summary:
Use `-Wl,-z,global` for all Sanitizer shared libraries on
Android. We want them to be in the global group
(https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md#changes-to-library-search-order)
to avoid any alloc/dealloc mismatch between the libc allocator & said library.

`audioserver` was one of the binary that exhibited the problem with Scudo,
this seems to fix it.

[edited for accuracy]

Reviewers: eugenis, alekseyshl

Reviewed By: eugenis

Subscribers: kubamracek, srhines, mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49198

llvm-svn: 337010
2018-07-13 15:31:05 +00:00
Marco Castelluccio d47bde0d7a Add a test with multiple BBs on the same line
Test for https://bugs.llvm.org/show_bug.cgi?id=38065.

llvm-svn: 336996
2018-07-13 13:01:43 +00:00
Dan Liew 95d9d22e00 Remove `tsan/Darwin/gcd-after-null.mm` test.
Summary:
This test invokes undocumented behaviour that could change in
the future. Given this, it's probably best to just remove the
test.

rdar://problem/42022283

Reviewers: kubamracek

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49269

llvm-svn: 336977
2018-07-13 07:37:01 +00:00
Dean Michael Berris 10141261e1 [XRay][compiler-rt] Add PID field to llvm-xray tool and add PID metadata record entry in FDR mode
Summary:
llvm-xray changes:
- account-mode - process-id  {...} shows after thread-id
- convert-mode - process {...} shows after thread
- parses FDR and basic mode pid entries
- Checks version number for FDR log parsing.

Basic logging changes:
- Update header version from 2 -> 3

FDR logging changes:
- Update header version from 2 -> 3
- in writeBufferPreamble, there is an additional PID Metadata record (after thread id record and tsc record)

Test cases changes:
- fdr-mode.cc, fdr-single-thread.cc, fdr-thread-order.cc modified to catch process id output in the log.

Reviewers: dberris

Reviewed By: dberris

Subscribers: hiraditya, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49153

llvm-svn: 336974
2018-07-13 05:38:22 +00:00
Dean Michael Berris 5d92d3e5be [XRay][compiler-rt] Profiling Mode: Flush logs on exit
Summary:
This change adds support for writing out profiles at program exit.

Depends on D48653.

Reviewers: kpw, eizan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D48956

llvm-svn: 336969
2018-07-13 04:04:18 +00:00
George Karpenkov 01c762b9fd [fuzzer] [tests] Increase the number of iterations for three-bytes.test
The test is flaky otherwise on some of our macOS machines in the test fleet.

Differential Revision: https://reviews.llvm.org/D49277

llvm-svn: 336966
2018-07-13 01:21:50 +00:00
Dan Liew 0de57a676c [CMake] Fix a typo in the variable used to retrieve source file names
for the `RTHwasan_dynamic` target.

Differential Revision: https://reviews.llvm.org/D49178

llvm-svn: 336944
2018-07-12 20:55:02 +00:00
Marco Castelluccio 2bb551c6bf Simplify instrprof-dlopen-dlclose-gcov.test to avoid failures on Aarch64.
The test for a function with an if block in a single line (https://bugs.llvm.org/show_bug.cgi?id=38065) will be moved to a separate test.

llvm-svn: 336942
2018-07-12 20:28:09 +00:00
Matt Morehouse 4543816150 [SanitizerCoverage] Add associated metadata to 8-bit counters.
Summary:
This allows counters associated with unused functions to be
dead-stripped along with their functions.  This approach is the same one
we used for PC tables.

Fixes an issue where LLD removes an unused PC table but leaves the 8-bit
counter.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, hiraditya, kcc

Differential Revision: https://reviews.llvm.org/D49264

llvm-svn: 336941
2018-07-12 20:24:58 +00:00
Matt Morehouse 24d617cbee [libFuzzer] If LLD available, require it to build first.
Since we now have a test that requires LLD, make sure it is built before
that test runs.

llvm-svn: 336932
2018-07-12 18:52:10 +00:00
Matt Morehouse d507c96ff1 [libFuzzer] Use lld-available for gc-sections.test.
The lld feature is never available for libFuzzer tests, so
gc-sections.test never actually runs.

llvm-svn: 336926
2018-07-12 18:09:03 +00:00
Dan Liew 4625480acb [CMake] Remove unnecesary list of source files for Xray unit tests.
The list duplicates information already available in the parent
directory so use that instead. It is unclear to me why we need
to spell out the dependencies explicitly but fixing that should
be done in a separate patch.

Differential Revision: https://reviews.llvm.org/D49177

llvm-svn: 336905
2018-07-12 13:48:18 +00:00
Dan Liew c5d759a926 [CMake] Rename `SANITIZER_HEADERS` to `SANITIZER_IMPL_HEADERS` under `lib/sanitizer_common`.
The variable name `SANITIZER_HEADERS` is already used for the list of
public headers in `include/CMakeLists.txt`.  Although the previous
implementation worked it's probably best to avoid shadowing global
variables to avoid confusion.

Differential Revision: https://reviews.llvm.org/D49176

llvm-svn: 336904
2018-07-12 13:36:44 +00:00
Dean Michael Berris 4366b0122a [XRay][compiler-rt] Fixup: require x86_64 for profiling mode tests
This constrains the build environments we are testing/supporting for the runtime
tests until we can be sure xray works in more platforms.

llvm-svn: 336878
2018-07-12 01:54:29 +00:00
Joel E. Denny a09b9317f5 [FileCheck] Add -allow-deprecated-dag-overlap to another compiler-rt test
See https://reviews.llvm.org/D47106 for details.

llvm-svn: 336859
2018-07-11 22:07:31 +00:00
Joel E. Denny da660d6a70 [FileCheck] Add -allow-deprecated-dag-overlap to failing compiler-rt tests
See https://reviews.llvm.org/D47106 for details.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D47326

llvm-svn: 336845
2018-07-11 20:26:44 +00:00
Zaara Syeda f3fd38effd Remove ppc64 BE XFAILs now that gcov profiling works, after starting a clean
build this time.

llvm-svn: 336839
2018-07-11 19:17:43 +00:00
Marco Castelluccio e125955890 Link to the correct bug number about the Mac failure for instrprof-shared-gcov-flush.test.
llvm-svn: 336820
2018-07-11 15:44:15 +00:00
Zaara Syeda 47c2ddf6b3 Revert 336811, there are still some problems with the tests.
llvm-svn: 336819
2018-07-11 15:37:19 +00:00
Zaara Syeda 495adf52db Remove ppc64 BE XFAILs now that gcov profiling works.
llvm-svn: 336811
2018-07-11 14:55:19 +00:00
Marco Castelluccio e21cfa7e7f Fix reading 32 bit gcov tag values on little-endian machines
Summary:
The write buffer contains signed chars, which means the shift operations caused values such as the arc tag value (0x01a10000) to be read incorrectly (0xffa10000).

This fixes a regression from https://reviews.llvm.org/D49132.

Reviewers: uweigand, davidxl

Reviewed By: uweigand

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49161

llvm-svn: 336775
2018-07-11 09:30:25 +00:00
Dean Michael Berris 8299e4b8db [XRay] basic mode PID and TID always fetch
Summary: XRayRecords now includes a PID field. Basic handlers fetch pid and tid each time they are called instead of caching the value. Added a testcase that calls fork and checks if the child TID is different from the parent TID to verify that the processes' TID are different in the trace.

Reviewers: dberris, Maknee

Reviewed By: dberris, Maknee

Subscribers: kpw, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49025

llvm-svn: 336769
2018-07-11 07:14:27 +00:00
Stephen Hines 6812c28ef3 Add libcxxabi option back for sanitizer use.
Summary:
A prior refactoring accidentally dropped the case for using libc++abi as
the out-of-tree C++ runtime library for sanitizers. This patch restores
that functionality, which is used by Android, which can't depend on the
full libc++ for these libraries.

Reviewers: phosek, EricWF

Reviewed By: phosek

Subscribers: meikeb, kongyi, chh, mgorny, delcypher, llvm-commits, #sanitizers, pirama

Differential Revision: https://reviews.llvm.org/D49157

llvm-svn: 336749
2018-07-11 00:50:03 +00:00
Matt Morehouse 79d55d30c3 [libFuzzer] Disable dataflow.test on AArch64.
Summary:
After my recent change to allow MSan + libFuzzer, the
ExplodeDFSanLabelsTest.cpp test started to overflow the stack with
recursive function SetBytesForLabel() on an AArch64 bot.  Perhaps that
bot has a smaller stack size, or maybe AArch64 has larger stack frames
for this particular function.

Reviewers: kcc, javed.absar

Reviewed By: kcc

Subscribers: kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D49150

llvm-svn: 336725
2018-07-10 20:45:36 +00:00
Matt Morehouse 0e904e8806 Revert "[Fuzzer] Afl driver changing iterations handling"
This reverts rL334510 due to breakage of afl_driver's command line
interface.

Patch By: Jonathan Metzman

Differential Revision: https://reviews.llvm.org/D49141

llvm-svn: 336719
2018-07-10 19:58:42 +00:00
Vlad Tsyrklevich f90ad5b327 Limit ASan non-executable-pc test case to x86 bots
The test case fails on the big-endian PPC bot, probably because PowerPC
uses function descriptors. More over other architectures don't support
NX mappings. (This test case was not being exercised prior to r336633.)

llvm-svn: 336714
2018-07-10 19:34:46 +00:00
Ulrich Weigand dd4d2ac607 [gcov] Fix fallout from r336693
Fix building GCDAProfiling.c with pre-C99 compilers.
This caused a build bot failure.

llvm-svn: 336706
2018-07-10 17:08:05 +00:00
Ulrich Weigand f4556f634f [asan] Disable non-execute test on s390
Processors before z14 don't support non-execute protection,
so they will start execution random memory contents, causing
the test to randomly fail or succeed.

llvm-svn: 336705
2018-07-10 16:55:27 +00:00
Dan Liew 9f6302e950 Try to fix broken build due to r336663.
It turns out that the `${XRAY_HEADERS}` CMake variable was already
in use and was used for public headers. It seems that
`lib/xray/tests/CMakeLists.txt` was depending on this.

To fix rename the new `${XRAY_HEADERS}` to `${XRAY_IMPL_HEADERS}`.

llvm-svn: 336699
2018-07-10 16:22:05 +00:00
Ulrich Weigand f0642bfe44 Remove s390x XFAILs now that gcov profiling works.
llvm-svn: 336695
2018-07-10 16:09:24 +00:00
Ulrich Weigand e508cd2a04 [gcov] Fix gcov profiling on big-endian machines
Two fixes required to handle big-endian systems:

- 64-bit counter values are stored in a mixed-endian format in the
gcov files: a 32-bit low-part followed by a 32-bit high part. Note that
this is already implemented correctly on the LLVM side, see
GCOVBuffer::readInt64.

- The tag values (e.g. arcs tag, object summary tag, ...) are aways
written as the same sequence of bytes independent of byte order. But
when *reading* them back in, the code reads them as 32-bit values in
host byte order. For the comparisons to work correctly, this should
instead always read them as little-endian values.

Fixes PR 38121.

Reviewed By: marco-c

Differential Revision: https://reviews.llvm.org/D49132

llvm-svn: 336693
2018-07-10 16:08:27 +00:00
Marco Castelluccio 2827420aab Reapply "Make __gcov_flush flush counters for all shared libraries"
This reapplies r336365, after marking tests as failing on various
configurations.

llvm-svn: 336678
2018-07-10 14:12:03 +00:00
Filipe Cabecinhas f6cf891b4d [scudo] Use mkdir -p when creating directories for a test
llvm-svn: 336673
2018-07-10 13:59:59 +00:00
Dan Liew 9af77974e2 [LibFuzzer] Disable MSan test on Darwin which was added by r336619. The MemorySanitizer is
not supported on Darwin currently and so Clang refuses to compile with
`-fsanitize=memory`.

llvm-svn: 336669
2018-07-10 13:35:35 +00:00
Dan Liew b1f95697c1 [CMake] Add compiler-rt header files to the list of sources for targets
when building with an IDE so that header files show up in the UI.
This massively improves the development workflow in IDEs.

To implement this a new function `compiler_rt_process_sources(...)` has
been added that adds header files to the list of sources when the
generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no
changes are made to the list of source files.

The function can be passed a list of headers via the
`ADDITIONAL_HEADERS` argument. For each runtime library a list of
explicit header files has been added and passed via
`ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of
headers was already present but it was stale and has been updated
to reflect the current state of the source tree.

The original version of this patch used file globbing (`*.{h,inc,def}`)
to find the headers but the approach was changed due to this being a
CMake anti-pattern (if the list of headers changes CMake won't
automatically re-generate if globbing is used).

The LLVM repo contains a similar function named `llvm_process_sources()`
but we don't use it here for several reasons:

* It depends on the `LLVM_ENABLE_OPTION` cache variable which is
  not set in standalone compiler-rt builds.
* We would have to `include(LLVMProcessSources)` which I'd like to
  avoid because it would include a bunch of stuff we don't need.

Differential Revision: https://reviews.llvm.org/D48422

llvm-svn: 336663
2018-07-10 13:00:17 +00:00
Filipe Cabecinhas 6a9c719ee1 [compiler-rt] Get rid of "%T" expansions
Summary:
Original patch by Kuba Mracek

The %T lit expansion expands to a common directory shared between all
the tests in the same directory, which is unexpected and unintuitive,
and more importantly, it's been a source of subtle race conditions and
flaky tests. In https://reviews.llvm.org/D35396, it was agreed that it
would be best to simply ban %T and only keep %t, which is unique to each
test. When a test needs a temporary directory, it can just create one
using mkdir %t.

This patch removes %T in compiler-rt.

Differential Revision: https://reviews.llvm.org/D48618

llvm-svn: 336661
2018-07-10 12:53:46 +00:00
Dean Michael Berris b251f62428 [XRay][compiler-rt] Fixup build breakage
Changes:

- Remove static assertion on size of a structure, fails on systems where
  pointers aren't 8 bytes.

- Use size_t instead of deducing type of arguments to
  `nearest_boundary`.

Follow-up to D48653.

llvm-svn: 336648
2018-07-10 08:58:12 +00:00
Dean Michael Berris 0dd4f9f22f [XRay][compiler-rt] xray::Array Freelist and Iterator Updates
Summary:
We found a bug while working on a benchmark for the profiling mode which
manifests as a segmentation fault in the profiling handler's
implementation. This change adds unit tests which replicate the
issues in isolation.

We've tracked this down as a bug in the implementation of the Freelist
in the `xray::Array` type. This happens when we trim the array by a
number of elements, where we've been incorrectly assigning pointers for
the links in the freelist of chunk nodes. We've taken the chance to add
more debug-only assertions to the code path and allow us to verify these
assumptions in debug builds.

In the process, we also took the opportunity to use iterators to
implement both `front()` and `back()` which exposes a bug in the
iterator decrement operation.  In particular, when we decrement past a
chunk size boundary, we end up moving too far back and reaching the
`SentinelChunk` prematurely.

This change unblocks us to allow for contributing the non-crashing
version of the benchmarks in the test-suite as well.

Reviewers: kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D48653

llvm-svn: 336644
2018-07-10 08:25:44 +00:00
Vlad Tsyrklevich 25291f15e7 Add lowercase OS name feature
Summary:
Some tests already make use of OS feature names, e.g. 'linux' and 'freebsd',
but they are not actually currently set by lit.

Reviewers: pcc, eugenis

Reviewed By: eugenis

Subscribers: emaste, krytarowski, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49115

llvm-svn: 336633
2018-07-10 02:02:21 +00:00
Matt Morehouse a34c65e845 [libFuzzer] Make -fsanitize=memory,fuzzer work.
This patch allows libFuzzer to fuzz applications instrumented with MSan
without recompiling libFuzzer with MSan instrumentation.

Fixes https://github.com/google/sanitizers/issues/958.

Differential Revision: https://reviews.llvm.org/D48891

llvm-svn: 336619
2018-07-09 23:51:08 +00:00
Matt Morehouse a5bb6d53f2 Revert "[libFuzzer] Mutation tracking and logging implemented"
This reverts r336597 due to bot breakage.

llvm-svn: 336616
2018-07-09 22:31:26 +00:00
Matt Morehouse d153d46884 [libFuzzer] Mutation tracking and logging implemented
Code now exists to track number of mutations that are used in fuzzing in
total and ones that produce new coverage. The stats are currently being
dumped to the command line.

Patch By: Kode Williams

Differntial Revision: https://reviews.llvm.org/D48054

llvm-svn: 336597
2018-07-09 20:17:52 +00:00
Alex Shlyapnikov 63af91574f [ASan] Minor ASan error reporting cleanup
Summary:
- use proper Error() decorator for error messages
- refactor ASan thread id and name reporting

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49044

llvm-svn: 336573
2018-07-09 17:54:55 +00:00
Michael Zolotukhin 9664ca9dce Revert "Make __gcov_flush flush counters for all shared libraries"
This reverts r336365: the added tests are failing on various
configurations (e.g. on green-dragon).

llvm-svn: 336474
2018-07-07 00:07:00 +00:00
Kostya Serebryany 2bd02db943 libFuzzer: always print line-break for NEW_FUNC/PC output
Summary: This is a minor cosmetic change. When function/path exceed ~1000 characters, the output is truncated before the line-break. I noticed this for NEW_FUNC.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48799

llvm-svn: 336461
2018-07-06 19:47:00 +00:00
Yvan Roux 19d28309af Recommit "[CMake] Run libFuzzer tests with check-all."
Since problematic tests on AArch64 were disabled at r336446.

llvm-svn: 336449
2018-07-06 17:22:02 +00:00
Matt Morehouse 373d6ed7cf [MSan] Add functions to enable/disable interceptor checks.
Summary:
The motivation for this change is to make libFuzzer+MSan possible
without instrumenting libFuzzer.

See https://github.com/google/sanitizers/issues/958.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D48890

llvm-svn: 336447
2018-07-06 17:10:51 +00:00
Yvan Roux becbb87553 [libFuzzer] Disable hanging tests on AArch64
Disable problematic tests which broke AArch64 bots.
Details available in Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=38034

Differential Revision: https://reviews.llvm.org/D49011

llvm-svn: 336446
2018-07-06 17:06:01 +00:00
Kostya Kortchinsky 2efb847b6f [scudo] Add some logs for Android
Summary:
Namely, set the abort message, and allow to write the message to syslog if the
option is enabled.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48902

llvm-svn: 336445
2018-07-06 16:50:12 +00:00
Marco Castelluccio 32d18beb8c Make __gcov_flush flush counters for all shared libraries
Summary:
This will make the behavior of __gcov_flush match the GCC behavior.

I would like to rename __gcov_flush to __llvm_gcov_flush (in case of programs linking to libraries built with different compilers), but I guess we can't for compatibility reasons.

Reviewers: davidxl

Reviewed By: davidxl

Subscribers: samsonov, vitalybuka, pcc, kcc, junbuml, glider, fhahn, eugenis, dvyukov, davidxl, srhines, chh, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48538

llvm-svn: 336365
2018-07-05 15:52:59 +00:00
Yvan Roux 01c0d860f3 Revert "[CMake] Run libFuzzer tests with check-all."
Revert due to AArch64 bots breakage, upstream PR raised to track the
issue: https://bugs.llvm.org/show_bug.cgi?id=38034

llvm-svn: 336341
2018-07-05 12:19:03 +00:00
George Karpenkov aeeac6d41c [libFuzzer] [NFC] Inline static variable to avoid the linker warning.
Differential Revision: https://reviews.llvm.org/D48650

llvm-svn: 336238
2018-07-04 00:37:45 +00:00
Kostya Serebryany 51ddb88300 [libFuzzer] add one more value profile metric, under a flag (experimental)
llvm-svn: 336234
2018-07-03 22:33:09 +00:00
Kostya Serebryany dcac0a3b5e [libFuzzer] remove stale code, as suggested in https://reviews.llvm.org/D48800
llvm-svn: 336230
2018-07-03 21:22:44 +00:00
Kostya Serebryany 0bc9143de8 [libFuzzer] add a tiny and surprisingly hard puzzle
llvm-svn: 336229
2018-07-03 21:17:55 +00:00
Kostya Kortchinsky e64a81475c [scudo] Get rid of builtin-declaration-mismatch warnings
Summary:
The C interceptors were using `SIZE_T` defined in the interception library as
a `__sanitizer::uptr`. On some 32-bit platforms, this lead to the following
warning:
```
warning: declaration of ‘void* malloc(SIZE_T)’ conflicts with built-in declaration ‘void* malloc(unsigned int)’ [-Wbuiltin-declaration-mismatch]
INTERCEPTOR_ATTRIBUTE void *malloc(SIZE_T size) {
```
`__sanitizer::uptr` is indeed defined as an `unsigned long` on those.

So just include `stddef.h` and use `size_t` instead.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48885

llvm-svn: 336221
2018-07-03 19:03:46 +00:00
Kostya Kortchinsky 02827792b5 [scudo] Enable Scudo on PPC64
Summary:
In conjunction with the clang side change D48833, this will enable Scudo on
PPC64. I tested `check-scudo` on a powerpc64le box and everything passes.

Reviewers: eugenis, alekseyshl

Reviewed By: alekseyshl

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48834

llvm-svn: 336213
2018-07-03 16:09:18 +00:00
Kostya Kortchinsky 4bc7f3d4d6 [scudo] Enable Scudo memory hooks for Fuchsia.
Summary:
    It would be useful for Flutter apps, especially, to be able to use
    malloc hooks to debug memory leaks on Fuchsia. They're not able to do
    this right now, so it'd be a nice bonus to throw in with the Scudo
    switchover.


Reviewers: cryptoad, alekseyshl

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D48618

llvm-svn: 336139
2018-07-02 19:48:01 +00:00
Fangrui Song 5f27e0c021 [asan] Fix deadlock issue on FreeBSD, caused by use of .preinit_array in rL325240
Summary:
Without this patch,
clang -fsanitize=address -xc =(printf 'int main(){}') -o a; ./a => deadlock in __asan_init>AsanInitInternal>AsanTSDInit>...>__getcontextx_size>_rtld_bind>rlock_acquire(rtld_bind_lock, &lockstate)

libexec/rtld-elf/rtld.c
  wlock_acquire(rtld_bind_lock, &lockstate);
  if (obj_main->crt_no_init)
    preinit_main(); // unresolved PLT functions cannot be called here

lib/libthr/thread/thr_rtld.c
  uc_len = __getcontextx_size(); // unresolved PLT function in libthr.so.3

check-xray tests currently rely on .preinit_array so we special case in
xray_init.cc

Subscribers: srhines, kubamracek, krytarowski, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48806

llvm-svn: 336067
2018-07-01 17:52:41 +00:00
David Carlier c33f35ff0f [UBsan] Enable subset of unit tests for OpenBSD
Reviewers: kubamracek, krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D48805

llvm-svn: 336053
2018-06-30 21:35:05 +00:00
Fangrui Song da891dc6b4 [asan] Use MADV_NOCORE for use_madv_dontdump on FreeBSD.
Currently in FreeBSD 12.0-CURRENT with trunk clang+compiler-rt, faulty -fsanitize=address executable hangs at 'urdlck' state.

Ka Ho Ng has verified that by backporting this to llvm 6.0.1, with use_madv_dontdump=1, shadow memory is not dumped.

ASAN_OPTIONS=abort_on_error=1:disable_coredump=0:use_madv_dontdump=1 ./a

Reviewers: dimitry, kcc, dvyukov, emaste, khng300

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48257

llvm-svn: 336046
2018-06-30 08:27:48 +00:00
Chih-Hung Hsieh 7222e8e30b [profile] Add llvm_gcov_flush to be called outside a shared library
__gcov_flush is hidden.
For applications to dump profiling data of selected .so files,
they can use dlsym to find and call llvm_gcov_flush in each .so file.

Differential Revision: https://reviews.llvm.org/D45454

llvm-svn: 336019
2018-06-29 21:45:55 +00:00
Evgeniy Stepanov d77e7ba74e [cfi] Use __builtin version of __clear_cache.
__builtin___clear_cache is also present on X86 and does the right thing (i.e. nop) there.

llvm-svn: 335997
2018-06-29 17:22:58 +00:00
Evgeniy Stepanov ac9b6adf77 [scs] Disable negative test in shadowcallstack.
The test checks that scs does NOT work correctly w/o runtime support.
That's a strange thing to test, and it is also flaky, because things
may just work if x18 happens to point to a writable page.

llvm-svn: 335982
2018-06-29 15:16:45 +00:00
Kostya Kortchinsky e92d7878e5 [scudo] Add some runtime tests for the minimal runtime
Summary:
As well as some tests to ensure that various combinations of the clang command
line flags work (shared/static/minimal).

Reviewers: eugenis, alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48553

llvm-svn: 335981
2018-06-29 14:56:25 +00:00
Evgeniy Stepanov 6d310651a2 [cfi] Reset i-cache after copying code in test.
Fixes intermittent crashes on Android.
Also add PROT_READ to the code mapping for no real reason.

llvm-svn: 335943
2018-06-29 00:38:26 +00:00
Evgeniy Stepanov 3702f91287 [asan] Change 3 tests from XFAIL to UNSUPPORTED.
The failure in https://github.com/google/sanitizers/issues/981 is flaky.

llvm-svn: 335941
2018-06-28 23:27:55 +00:00
Evgeniy Stepanov 730c985d56 [asan] Disable 3 tests on Android O+.
https://github.com/google/sanitizers/issues/981

llvm-svn: 335937
2018-06-28 22:54:55 +00:00
Evgeniy Stepanov e40ae2e3a2 [asan] Fix one more test on Android.
This change makes it pass both on libstdc++ and libc++-based toolchains.

The error was:
error: function-like macro '__GLIBC_PREREQ' is not defined
                               ^

llvm-svn: 335936
2018-06-28 22:54:42 +00:00
Evgeniy Stepanov 59ef3dbbc6 [asan] un-XFAIL one test
NDK r16 provides glob.h, which makes this test pass.
Supporting different test outcomes depending on the version of NDK
add unnecessary complexity to the test harness. IMHO, it's OK to require
the latest stable release.

llvm-svn: 335935
2018-06-28 22:54:30 +00:00
Evgeniy Stepanov 71dc39823c [fuzzer] Don't run tests on Android.
Need better build system support.

llvm-svn: 335933
2018-06-28 22:16:59 +00:00
Alex Shlyapnikov fb1644835b [TSan] More detailed error message on failed sahdow memory madvise
Summary:
Report errno value on failed shadow memory madvise attempt and add a
hint message with the possible workaround.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48668

llvm-svn: 335928
2018-06-28 21:38:34 +00:00
David Carlier a8c5041bdf [UBsan] minimal unit tests removing OpenBSD spacial case
The OpenBSD driver now handles sanitisers with ld.lld automatically.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48740

llvm-svn: 335923
2018-06-28 21:10:52 +00:00
Vedant Kumar c8735fbb4c [Darwin] Add an integration test for PGO + symbol exports
rdar://41470205

llvm-svn: 335891
2018-06-28 17:53:43 +00:00
Kuba Mracek df93fe46e5 Skip building TSan on platforms where there are no 64-bit architectures.
llvm-svn: 335873
2018-06-28 16:43:43 +00:00
Haojian Wu d93a1e5271 Correct the test modified in rL335777.
llvm-svn: 335865
2018-06-28 15:24:34 +00:00
Petr Hosek 887f26d470 Support for multiarch runtimes layout
This change adds a support for multiarch style runtimes layout, so in
addition to the existing layout where runtimes get installed to:

lib/clang/$version/lib/$os

Clang now allows runtimes to be installed to:

lib/clang/$version/$target/lib

This also includes libc++, libc++abi and libunwind; today those are
assumed to be in Clang library directory built for host, with the
new layout it is possible to install libc++, libc++abi and libunwind
into the runtime directory built for different targets.

The use of new layout is enabled by setting the
LLVM_ENABLE_RUNTIME_TARGET_DIR CMake variable and is supported by both
projects and runtimes layouts. The runtimes CMake build has been further
modified to use the new layout when building runtimes for multiple
targets.

Differential Revision: https://reviews.llvm.org/D45604

llvm-svn: 335809
2018-06-28 03:11:52 +00:00
George Karpenkov 348dd9ddd1 [libFuzzer] [Tests] [NFC] Change seed for reduce_inputs.test
Differential Revision: https://reviews.llvm.org/D48686

llvm-svn: 335802
2018-06-28 00:39:50 +00:00
George Karpenkov 0cad9cb9a4 [libFuzzer] [NFC] [Tests] Mark signal handling tests as UNSUPPORTED on Darwin
Under load, these tests tend to fail sporadically on our bots.
In my understanding, the signal handling is not guaranteed to happen
within 2 seconds, and the test is inherently flaky.

llvm-svn: 335792
2018-06-27 23:11:24 +00:00
Petr Hosek 61574fdb06 [sanitizer] zx_cprng_draw no longer returns any value
Remove the return value check.

Differential Revision: https://reviews.llvm.org/D48671

llvm-svn: 335790
2018-06-27 21:25:21 +00:00
Vlad Tsyrklevich 600a2df289 Another shot at fixing android r335644 failure
The android buildbot moves the build outputs to a different directory
and rewrites the executable path, the DSO passed as an argument does not
get re-written. Use rpaths to load the DSO the same way the
test/cfi/cross-dso/ tests do and test the DSO name differently.

llvm-svn: 335777
2018-06-27 20:00:55 +00:00
Matt Morehouse 520748f01e [UBSan] Add silence_unsigned_overflow flag.
Summary:
Setting UBSAN_OPTIONS=silence_unsigned_overflow=1 will silence all UIO
reports.  This feature, combined with
-fsanitize-recover=unsigned-integer-overflow, is useful for providing
fuzzing signal without the excessive log output.

Helps with https://github.com/google/oss-fuzz/issues/910.

Reviewers: kcc, vsk

Reviewed By: vsk

Subscribers: vsk, kubamracek, Dor1s, llvm-commits

Differential Revision: https://reviews.llvm.org/D48660

llvm-svn: 335762
2018-06-27 18:24:46 +00:00
Petr Hosek 3209b28aa9 [sanitizer] zx_cprng_draw no longer takes the output argument
The zx_cprng_draw system call no longer takes the output argument.

Differential Revision: https://reviews.llvm.org/D48657

llvm-svn: 335755
2018-06-27 16:49:37 +00:00
Vedant Kumar 7eb8119798 [asan] Relax the null_deref.cc test even further
On Darwin/x86_64, asan may report the crashing line of NullDeref as line
19 (i.e the closing brace of the function), whereas on other targets we
see line 15 ("ptr[10]++").

The optimized debug info here isn't reliable enough to check.

rdar://problem/41526369

llvm-svn: 335747
2018-06-27 16:06:15 +00:00
Vedant Kumar 43eec242e0 [asan] Relax a test which depends on optimized debug info
On some ARM platforms this test depends on debug locations being
present on constant materialization code, which was eliminated in
r335497.

Relax the test to allow two outcomes: the backtrace either contains the
right line numbers, or no line numbers.

llvm-svn: 335741
2018-06-27 14:57:59 +00:00
Dan Liew bb78eef6b6 [CMake] Tidy up the organisation of compiler-rt when configured as a standalone
build with an IDE (e.g. Xcode) as the generator.

Previously the global `USE_FOLDERS` property wasn't set in standalone
builds leading to existing settings of FOLDER not being respected.

In addition to this there were several targets that appeared at the top
level that were not interesting and clustered up the view. These have
been changed to be displayed in "Compiler-RT Misc".

Now when an Xcode project is generated from a standalone compiler-rt
build the project navigator is much less cluttered. The interesting
libraries should appear in "Compiler-RT Libraries" in the IDE.

Differential Revision: https://reviews.llvm.org/D48378

llvm-svn: 335728
2018-06-27 12:56:34 +00:00
Vlad Tsyrklevich 5417225745 Fix another bot broken by r335644
The android sanitizer bot can't resolve the function name in the DSO and
it's not relevant to the test.

llvm-svn: 335693
2018-06-27 00:31:17 +00:00
Vlad Tsyrklevich ee59c5e922 Fix test broken by r335644
llvm-svn: 335657
2018-06-26 20:21:33 +00:00
Vlad Tsyrklevich e745cf9bf3 CFI: Print DSO names for failed cross-DSO icalls
Reviewers: pcc

Reviewed By: pcc

Subscribers: kubamracek, delcypher, llvm-commits, kcc, #sanitizers

Differential Revision: https://reviews.llvm.org/D48583

llvm-svn: 335644
2018-06-26 18:51:04 +00:00
Matt Morehouse 0948bc2086 [libFuzzer] Do not turn unittest warnings into errors.
Some warnings originating from googletest were causing bots to fail
while bulding unit tests.  The sanitizers address this issue by not
using -Werror.  We adopt this approach for libFuzzer.

llvm-svn: 335640
2018-06-26 18:37:37 +00:00
Matt Morehouse 0ab788e271 [libFuzzer] Mark several tests UNSUPPORTED for aarch64.
Now that check-fuzzer runs as part of check-all, some aarch64 bots had
tests failing.

llvm-svn: 335639
2018-06-26 18:37:09 +00:00
Peter Collingbourne f99bc0b2f6 Mark mfcall.cpp as UNSUPPORTED: win32.
llvm-svn: 335573
2018-06-26 02:59:22 +00:00
Peter Collingbourne e44acadf6a Implement CFI for indirect calls via a member function pointer.
Similarly to CFI on virtual and indirect calls, this implementation
tries to use program type information to make the checks as precise
as possible.  The basic way that it works is as follows, where `C`
is the name of the class being defined or the target of a call and
the function type is assumed to be `void()`.

For virtual calls:
- Attach type metadata to the addresses of function pointers in vtables
  (not the functions themselves) of type `void (B::*)()` for each `B`
  that is a recursive dynamic base class of `C`, including `C` itself.
  This type metadata has an annotation that the type is for virtual
  calls (to distinguish it from the non-virtual case).
- At the call site, check that the computed address of the function
  pointer in the vtable has type `void (C::*)()`.

For non-virtual calls:
- Attach type metadata to each non-virtual member function whose address
  can be taken with a member function pointer. The type of a function
  in class `C` of type `void()` is each of the types `void (B::*)()`
  where `B` is a most-base class of `C`. A most-base class of `C`
  is defined as a recursive base class of `C`, including `C` itself,
  that does not have any bases.
- At the call site, check that the function pointer has one of the types
  `void (B::*)()` where `B` is a most-base class of `C`.

Differential Revision: https://reviews.llvm.org/D47567

llvm-svn: 335569
2018-06-26 02:15:47 +00:00
David Carlier 7a0bf90dcf [UBsan] Enable ubsan minimal unit tests on OpenBSD
OpenBSD needs lld linker for sanitisers.
Disabling lint checking as some symbols cannot be defined and block the proper unit tests launch.

Reviewers: lebedev.ri, vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48528

llvm-svn: 335524
2018-06-25 21:10:25 +00:00
Vedant Kumar 487bef376b [ubsan] Mark a test case as unsupported on Windows
__ubsan_on_report isn't defined as weak, and redefining it in a test is
not supported on Windows.

See the error message here: https://reviews.llvm.org/D48446

llvm-svn: 335523
2018-06-25 20:56:23 +00:00
Matt Morehouse 3077c82498 [CMake] Run libFuzzer tests with check-all.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D48200

llvm-svn: 335519
2018-06-25 20:30:03 +00:00
Alex Shlyapnikov d42edb28d0 [HWASan] Initalize shadow earler.
Summary:
Initialize shadow memory before calling more libc functions to allow
for HWASan-instrumented libc.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48551

llvm-svn: 335502
2018-06-25 17:27:13 +00:00
Matt Morehouse 988f2da613 [libFuzzer] Use Vector rather than std::vector.
llvm-svn: 335487
2018-06-25 15:59:24 +00:00
Vedant Kumar c835306ac5 [ubsan] Fix __ubsan_on_report interface definition
Speculative fix for the interface definition of __ubsan_on_report for
the Windows bots:

  http://lab.llvm.org:8011/builders/sanitizer-windows/builds/30528
  lib\ubsan\ubsan_interface.inc(55): error C2065: '__ubsan_on_report':
  undeclared identifier

INTERCEPT_SANITIZER_WEAK_FUNCTION was the wrong macro to use to begin
with because __ubsan_on_report isn't weak. Reading through that macro,
it's still not clear to me why there is an undefined reference, though,
because it appears to define a dummy __ubsan_on_report shim.

llvm-svn: 335383
2018-06-22 20:15:33 +00:00
Vedant Kumar 059d20360a [ubsan] Add support for reporting diagnostics to a monitor process
Add support to the ubsan runtime for reporting diagnostics to a monitor
process (e.g a debugger).

The Xcode IDE uses this by setting a breakpoint on __ubsan_on_report and
collecting diagnostic information via __ubsan_get_current_report_data,
which it then surfaces to users in the editor UI.

Testing for this functionality already exists in upstream lldb, here:
lldb/packages/Python/lldbsuite/test/functionalities/ubsan

Apart from that, this is `ninja check-{a,ub}san` clean.

Differential Revision: https://reviews.llvm.org/D48446

llvm-svn: 335371
2018-06-22 17:21:17 +00:00
Dmitry Vyukov ac5fa6605c tsan: fix deficiency in MutexReadOrWriteUnlock
MutexUnlock uses ReleaseStore on s->clock, which is the right thing to do.
However MutexReadOrWriteUnlock for writers uses Release on s->clock.
Make MutexReadOrWriteUnlock also use ReleaseStore for consistency and performance.
Unfortunately, I don't think any test can detect this as this only potentially
affects performance.

llvm-svn: 335322
2018-06-22 08:27:52 +00:00
Kostya Kortchinsky 307c2eb94f [scudo] Add a minimal runtime for -fsanitize-minimal-runtime compatibility
Summary:
This patch follows D48373.

The point is to be able to use Scudo with `-fsanitize-minimal-runtime`. For that
we need a runtime that doesn't embed the UBSan one. This results in binaries
that can be compiled with `-fsanitize=scudo,integer -fsanitize-minimal-runtime`.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48377

llvm-svn: 335296
2018-06-21 21:48:04 +00:00
George Karpenkov b085ca5095 [libFuzzer] Filter architectures for testing on Apple platforms.
This is done in all other sanitizers, and was missing on libFuzzer.

llvm-svn: 335290
2018-06-21 21:19:43 +00:00
George Karpenkov e1a50ca120 [libFuzzer] Provide more descriptive names for testing targets.
llvm-svn: 335289
2018-06-21 21:19:24 +00:00
Kuba Mracek 14c786a960 [tsan] Use DARWIN_osx_LINK_FLAGS when building unit tests to match ASan behavior.
llvm-svn: 335265
2018-06-21 18:00:51 +00:00
Kuba Mracek bb9dedfa8a [sanitizer] Stop running tests against 32-bit iOS simulator
llvm-svn: 335245
2018-06-21 15:21:24 +00:00
David Carlier 7d796ececd [TSan] fix build and couple of unit tests on FreeBSD
Similarly to Msan adding -pie provokes linkage issue, was spotted with pie_test.cc
Set to XFAIL for couple of unit tests.

Reviewers: vitalybuka, krytarowski, dim

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48317

llvm-svn: 335166
2018-06-20 20:17:44 +00:00
David Carlier 8979d44d31 [Lsan] intercept thr_exit on FreeBSD
Intercepts thr_exit call on FreeBSD.
Disable pthread key workflow.
The pthread key create approach does not function under FreeBSD as the libpthread is not initialised enough at this stage.

Reviewers: vitalybuka, krytarowski, dim

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48268

llvm-svn: 335164
2018-06-20 20:13:25 +00:00
Alex Shlyapnikov 7e067ab1af [Sanitizers] Remove OOM/BadRequest allocator error handling policies.
Summary:
Remove the generic error nadling policies and handle each allocator error
explicitly. Although more verbose, it allows for more comprehensive, precise
and actionable allocator related failure reports.

This finishes up the series of changes of the particular sanitizer
allocators, improves the internal allocator error reporting and removes
now unused policies.

Reviewers: vitalybuka, cryptoad

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48328

llvm-svn: 335147
2018-06-20 17:10:33 +00:00
Kuba Mracek dd51ef2867 Fix-up for r335123: Set default apple_platform to "osx".
llvm-svn: 335133
2018-06-20 14:59:16 +00:00
Kuba Mracek badaa15c88 [sanitizer] Unify and generalize Apple platforms in CMake and lit test configs
There's more platforms than just "ios" and "iossim" that we should support, and adding more lit config variables for each platform isn't great. Let's generalize and have a single value that determines what the platform under test is.

Differential Revision: https://reviews.llvm.org/D48309

llvm-svn: 335123
2018-06-20 13:33:42 +00:00
Reid Kleckner 3f573cab15 [asan] Add flushes to try to fix test
llvm-svn: 335089
2018-06-20 01:09:51 +00:00
Reid Kleckner b3ba1cc82a [asan] Add Windows test for handle_segv and SetUnhandledExceptionFilter
llvm-svn: 335087
2018-06-20 00:45:54 +00:00
Fangrui Song 6fa3bf9588 [XRay] Fix error message. NFC
Reviewers: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48313

llvm-svn: 335055
2018-06-19 15:39:19 +00:00
Kostya Kortchinsky 63c33b1c9b [scudo] Move noinline functions definitions out of line
Summary:
Mark `isRssLimitExceeded` as `NOINLINE`, and move it's definition as well as
the one of `performSanityChecks` out of the class definition, as requested.

Reviewers: filcab, alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48228

llvm-svn: 335054
2018-06-19 15:36:30 +00:00
Fangrui Song cad40b1a39 [XRay] rm GLOB || true -> rm -f
Summary: `rm -f` does not write diagnostic message when there is no file argument.

Reviewers: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48311

llvm-svn: 335025
2018-06-19 07:11:33 +00:00
Francis Visoiu Mistrih 972a462a7e Fix bots after r334981
llvm-svn: 335013
2018-06-19 02:56:24 +00:00
Reid Kleckner b8a5ee2b00 [asan] Avoid deadlock when initializing the symbolizer CHECK fails
llvm-svn: 335007
2018-06-19 00:36:47 +00:00
Chris Bieneman feb4537b44 Fixing os_version_check.c to be actual C source
The initial implementaiton was using the C++ typeof keyword. This
causes the compiler to spew warnings unnecissarilly. This patch removes
the uses of typeof and replaces them with explicit typedefs of the
function types.

llvm-svn: 334981
2018-06-18 20:56:19 +00:00
Alex Shlyapnikov c75d47b52d [TSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404 and other sanitizer allocators.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, no stack, no details, not too
helpful nor informative. To improve the situation, detailed and structured
common errors were defined and reported under the appropriate conditions.

Common tests were generalized a bit to cover a slightly different TSan
stack reporting format, extended to verify errno value and returned
pointer value check is now explicit to facilitate debugging.

Reviewers: dvyukov

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48087

llvm-svn: 334975
2018-06-18 20:03:31 +00:00
Francis Visoiu Mistrih 1197f01bb9 [sanitizer] Guard call to internal_lseek when SANITIZER_MAC is true
r334881 breaks macOS bots because internal_lseek is not defined (neither
used on macOS):
http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/46240/consoleFull.

See discussion from r334881: https://reviews.llvm.org/rL334881

llvm-svn: 334944
2018-06-18 13:53:51 +00:00
Paul Robinson 7555c589af Update copyright year to 2018.
llvm-svn: 334936
2018-06-18 12:22:17 +00:00
Petr Hosek 068246886a [Fuzzer] Set an explicit libc++ dependency when needed
On targets that don't link internal libc++ (Fuchsia and Linux) but
use libc++ as their C++ library and libFuzzer is being built using
the just built compiler together with libc++ as part of runtimes, we
need an explicit dependency from libFuzzer object library to libc++ to
make sure the headers are available by the time we start building
libFuzzer.

Differential Revision: https://reviews.llvm.org/D48261

llvm-svn: 334928
2018-06-18 08:01:57 +00:00
Vitaly Buka 7fd0d388e0 [sanitizer] Suppress unused function warning
llvm-svn: 334923
2018-06-18 05:55:44 +00:00
Vitaly Buka 87ea4487f7 [sanitizer] Fix tsan GO build
llvm-svn: 334914
2018-06-17 17:58:08 +00:00
Vitaly Buka 4647c8b93d [sanitizer] Fix s390 build after r334900
llvm-svn: 334913
2018-06-17 17:40:38 +00:00
Vitaly Buka 755f0e8174 [sanitizer] Use confstr to check libc version in InitTlsSize
Reviewers: Lekensteyn, jakubjelinek

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D48265

llvm-svn: 334912
2018-06-17 17:31:22 +00:00
Jonas Hahnfeld 71befacd6c Revert "[CMake] Use a different source depending on C++ support"
This reverts commit r332924 and followup r332936 silencing a warning.

The change breaks the build on x86 if there is no 32-bit version of the
C++ libraries, see discussion in https://reviews.llvm.org/D47169.

llvm-svn: 334903
2018-06-17 09:51:33 +00:00
Vitaly Buka 4d6b33c510 [fuzzer] Python 3 print fixes
llvm-svn: 334902
2018-06-17 09:11:56 +00:00
Vitaly Buka b213a345e1 [fuzzer] Fix collect_data_flow.py for python 3
llvm-svn: 334901
2018-06-17 08:41:56 +00:00
Vitaly Buka b957f99d57 [sanitizer] Use const char* in internal_simple_strtoll
llvm-svn: 334900
2018-06-17 08:41:45 +00:00
Fangrui Song 87bcd4abef [sanitizer_common] Fix windows build caused by r334881
llvm-svn: 334884
2018-06-16 05:05:19 +00:00
Vitaly Buka b0a3e6f127 [asan] Enable fgets_fputs test on Android
"echo data" didn't work because %run on android executes test on the device
when lit shell command on the host system.

https://github.com/google/sanitizers/issues/952

llvm-svn: 334883
2018-06-16 04:01:08 +00:00
Fangrui Song d40fc6019b [sanitizer_common] Use O_TRUNC for WrOnly access mode.
Summary: Otherwise if the file existed and was larger than the write size before the OpenFile call, the file will not be truncated and contain garbage in trailing bytes.

Reviewers: glider, kcc, vitalybuka

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48250

llvm-svn: 334881
2018-06-16 03:32:59 +00:00
Vitaly Buka 572e442b6d [asan] Move long-object-path.cc test to Linux
Reviewers: Lekensteyn

Reviewed By: Lekensteyn

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D48186

llvm-svn: 334866
2018-06-15 21:25:42 +00:00
Matt Morehouse 668dd03c8d [libFuzzer] Avoid -fuse-ld=lld on gc-sections.
The bot doesn't recognize lld as a linker even though it has the
property lld-available.

llvm-svn: 334864
2018-06-15 21:01:56 +00:00
Matt Morehouse 0ea9a90b3d [SanitizerCoverage] Add associated metadata to pc-tables.
Summary:
Using associated metadata rather than llvm.used allows linkers to
perform dead stripping with -fsanitize-coverage=pc-table.  Unfortunately
in my local tests, LLD was the only linker that made use of this metadata.

Partially addresses https://bugs.llvm.org/show_bug.cgi?id=34636 and fixes
https://github.com/google/sanitizers/issues/971.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: Dor1s, hiraditya, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D48203

llvm-svn: 334858
2018-06-15 20:12:58 +00:00
Petr Hosek 5e6bc090ed [Fuzzer] Don't hardcode target architecture for Fuzzer tests
Don't hardcode the architecture for Fuzzer tests which breaks when
compiler-rt is being compiled for architectures other than x86_64.

Differential Revision: https://reviews.llvm.org/D48207

llvm-svn: 334852
2018-06-15 18:21:02 +00:00
Kostya Kortchinsky 4adf24502e [scudo] Add verbose failures in place of CHECK(0)
Summary:
The current `FailureHandler` mechanism was fairly opaque with regard to the
failure reason due to using `CHECK(0)`. Scudo is a bit different from the other
Sanitizers as it prefers to avoid spurious processing in its failure path. So
we just `dieWithMessage` using a somewhat explicit string.

Adapted the tests for the new strings.

While this takes care of the `OnBadRequest` & `OnOOM` failures, the next step
is probably to migrate the other Scudo failures in the same failes (header
corruption, invalid state and so on).

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: filcab, mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48199

llvm-svn: 334843
2018-06-15 16:45:19 +00:00
Eli Friedman 16c4456022 [compiler-rt] [builtins] Don't build __atomic_* by default.
The locks need to be implemented in a shared library to work correctly,
so they shouldn't be part of libclang_rt.builtins.a, except in
specialized scenarios where the user can prove it will only be linked
once.

Differential Revision: https://reviews.llvm.org/D47606

llvm-svn: 334779
2018-06-14 23:22:53 +00:00
George Karpenkov 9afe1fa7e2 [libFuzzer] [NFC] XFAIL one of the tests on iOS.
llvm-svn: 334775
2018-06-14 22:18:18 +00:00
George Karpenkov d7e1a9488b [libFuzzer] [NFC] Remaining minor fixes to support testing on devices.
XFAIL's and adding %run commands.

llvm-svn: 334774
2018-06-14 22:13:36 +00:00
George Karpenkov d1e0365ce3 [libFuzzer] [NFC] Support multi-arch and multi-OS building and testing
Differential Revision: https://reviews.llvm.org/D47296

llvm-svn: 334768
2018-06-14 20:46:07 +00:00
George Karpenkov 46b0df649f [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 334766
2018-06-14 20:30:04 +00:00
George Karpenkov 320c690df3 [NFC] Generalize flags for linking in shared objects in compiler-rt tests to support using multiple shared objects at once
Differential Revision: https://reviews.llvm.org/D48156

llvm-svn: 334765
2018-06-14 20:29:47 +00:00
Alex Shlyapnikov c9b41354ff [MSan] Linker-initialize static fallback_mutex in msan_allocator.cc
Summary:
static fallback_mutex in msan_allocator.cc does not need the ctor
call and can be linker initialized.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: morehouse, eugenis

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48145

llvm-svn: 334749
2018-06-14 18:30:51 +00:00
Alex Shlyapnikov 241b758690 [ASan] Linker-initialize static ScopedInErrorReport::current_error_.
Summary:
Static ScopedInErrorReport::current_error_ can be linker initialized to
shave one global ctor call on application startup and be __asan_init-safe.

Global constructors in ASan runtime are bad because __asan_init runs
from preinit_array, before any such constructors.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: eugenis, morehouse

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48141

llvm-svn: 334748
2018-06-14 18:22:23 +00:00
Peter Wu 4ff2d87f30 [ASAN] fix typos and disable long-object-path test for win32
Glob patterns seem unsupported for commands executed by the emulated
shell (LIT_USE_INTERNAL_SHELL=1). Disable the test while that is being
addressed (a workaround such as "cd a-*" also does not work).

llvm-svn: 334719
2018-06-14 15:12:46 +00:00
Kostya Kortchinsky 1ee741bdb6 [scudo] Make Secondary linker-initialized compliant
Summary:
As a follow up to D48142 for Scudo, switch the `SpinMutex` to its static
counterpart, and ensure zero-initialization by memset'ing the whole class.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48148

llvm-svn: 334716
2018-06-14 14:33:28 +00:00
Peter Wu 1bbab1e558 [ASAN] fix startup crash in dlsym for long paths since glibc 2.27
Summary:
Error messages for dlsym used to be stored on the stack, but since
commit 2449ae7b ("ld.so: Introduce struct dl_exception") in glibc 2.27
these are now stored on the heap (and thus use the dlsym alloc pool).

Messages look like "undefined symbol: __isoc99_printf\0/path/to/a.out".
With many missing library functions and long object paths, the pool is
quickly exhausted. Implement a simple mechanism to return freed memory
to the pool (clear it in case it is used for calloc).

Fixes https://github.com/google/sanitizers/issues/957

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D47995

llvm-svn: 334703
2018-06-14 10:42:54 +00:00
David Carlier 706ea47bdc [Sanitizer] fix compilation warning
In most of systems, this field is a signed type but in some it is an unsigned.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48118

llvm-svn: 334686
2018-06-14 04:51:26 +00:00
Peter Wu 0569660af3 [ASAN] Fix fputs interception for 32-bit macOS
On 32-bit macOS, "_fputs$UNIX2003" is called instead of "fputs" and the
"fgets_fputs" test fails. Apparently previous versions still passed the
test due to the internal implementation calling "strlen", but that does
not seem to be the case with macOS 10.13.3. Fixes r334450.

llvm-svn: 334670
2018-06-13 23:19:21 +00:00
Alex Shlyapnikov 3fc66075c7 [Sanitizers] Make sanitizer allocator linker-initialize compliant.
Summary:
These four SpinMutex ctors was the only code executed in the ctor for
the static __asan::Allocator instance (same for the other sanitizers
allocators), which is supposed to be fully linker-initialized.

Also, when the global ctor for this allocator instance is executed,
this instance might already be initialized by __asan_init called from
.preinit_array.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: morehouse, eugenis, cryptoad

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48142

llvm-svn: 334660
2018-06-13 21:45:01 +00:00
Dimitry Andric a00c372cd3 Disable MSan tests of prlimit on FreeBSD
Like NetBSD, FreeBSD does not have prlimit(2), so do not attempt to use
it during the MSan tests.

llvm-svn: 334659
2018-06-13 21:37:49 +00:00
Shoaib Meenai b682276f48 [compiler-rt] Use CMAKE_LINKER instead of hardcoding ld
Respect a custom linker path provided by the user if one is present
(otherwise CMAKE_LINKER will have been set to the right value by CMake).

llvm-svn: 334654
2018-06-13 20:48:30 +00:00
Ahmed Bougacha 61631b1b8f [ASAN] Re-enable fgets_fputs test for darwin.
It seems to be passing, at least on the bots I've seen.

llvm-svn: 334626
2018-06-13 17:46:17 +00:00
Alex Shlyapnikov 4bf9b5612a [TSan] Fix madvise(MADV_NOHUGEPAGE) for meta shadow memory
Summary:
Move madvise(MADV_NOHUGEPAGE) for the meta shadow memory after the meta
shadow memory is mapped (currently it silently fails with ENOMEM).

Add a diagnostic message to detect similar problems in the future.

Reviewers: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48097

llvm-svn: 334624
2018-06-13 17:18:41 +00:00
Petr Hosek 7252e2a8bd [XRay] Set an explicit dependency on libc++ when needed
When XRay is being built as part of the just built compiler together
with libc++ as part of the runtimes build, we need an explicit
dependency from XRay to libc++ to make sure that the library is
available by the time we start building XRay.

Differential Revision: https://reviews.llvm.org/D48113

llvm-svn: 334575
2018-06-13 07:08:28 +00:00
Petr Hosek 22f8825dca Reland "Passthrough additional flags to custom libcxx CMake build"
This is needed when we're cross-compiling compiler-rt.

Differential Revision: https://reviews.llvm.org/D47834

llvm-svn: 334570
2018-06-13 05:32:22 +00:00
Peter Wu fa60e6966c [ASAN] disable fgets_fputs test for android and darwin
aarch64/aosp_marlin-userdebug/OPR4.170623.016] builder unexpectedly
failed the fgets test (`assertion "fp" failed`). macOS unexpectedly
passes the fputs test without triggering ASAN.

llvm-svn: 334558
2018-06-12 23:05:03 +00:00
George Karpenkov 077c75bfd1 [libFuzzer] [NFC] Do not use hardcoded relative paths in tests.
For paths without "%t" inside uniqueness is not guaranteed, and
potential collisions might be present.

Differential Revision: https://reviews.llvm.org/D47288

llvm-svn: 334546
2018-06-12 21:14:33 +00:00
George Karpenkov 43a0f71aa4 [libFuzzer] [NFC] Make compiler command generation more readable.
Use config.clang as a more general option than config.c_compiler.

Differential Revision: https://reviews.llvm.org/D47295

llvm-svn: 334545
2018-06-12 21:14:11 +00:00
Matt Morehouse e3300d2ff3 Revert r334458, r334220, r334212, r334139.
Reverts changes to AddCompilerRT.cmake due to breakage of
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/.

llvm-svn: 334528
2018-06-12 19:43:40 +00:00
Walter Lee 0a58f9c953 [asan, myriad] Support environment variables
Provide a buffer that the test harness can write into to provide
values for the environment variables.  Format is a null-separated list
of VAR=value pairs; this is sufficent for our purpose.

Differential Revision: https://reviews.llvm.org/D47960

llvm-svn: 334522
2018-06-12 18:34:12 +00:00
David Carlier 301855fb0d [Fuzzer] Afl driver changing iterations handling
Handling differently the iterations with the type limit and eventually an error message.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D47880

llvm-svn: 334510
2018-06-12 15:47:58 +00:00
Kostya Kortchinsky 76969eaf3d [scudo] Add C++17 aligned new/delete operators support
Summary:
This CL adds support for aligned new/delete operators (C++17). Currently we
do not support alignment inconsistency detection on deallocation, as this
requires a header change, but the APIs are introduced and are functional.

Add a smoke test for the aligned version of the operators.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48031

llvm-svn: 334505
2018-06-12 14:42:40 +00:00
Peter Wu 14db2509ac [ASAN] fix fgets and fgets_fputs tests failure
Some systems (Android) might not have /etc/passwd. Fixes r334450.

llvm-svn: 334487
2018-06-12 09:27:18 +00:00
Dean Michael Berris aea8ece102 [XRay][profiler] Part 5: Profiler File Writing
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements the profile writing mechanism, to allow profiles
collected through the profiler mode to be persisted to files.

Follow-on patches would allow us to load these profiles and start
converting/analysing them through the `llvm-xray` tool.

Depends on D44620.

Reviewers: echristo, kpw, pelikan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45998

llvm-svn: 334472
2018-06-12 04:06:25 +00:00
Dean Michael Berris cfd7eec3d8 [XRay][profiler] Part 4: Profiler Mode Wiring
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements the wiring required to enable us to actually
select the `xray-profiling` mode, and install the handlers to start
measuring the time and frequency of the function calls in call stacks.
The current way to get the profile information is by working with the
XRay API to `__xray_process_buffers(...)`.

In subsequent changes we'll implement profile saving to files, similar
to how the FDR and basic modes operate, as well as means for converting
this format into those that can be loaded/visualised as flame graphs. We
will also be extending the accounting tool in LLVM to support
stack-based function call accounting.

We also continue with the implementation to support building small
histograms of latencies for the `FunctionCallTrie::Node` type, to allow
us to actually approximate the distribution of latencies per function.

Depends on D45758 and D46998.

Reviewers: eizan, kpw, pelikan

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D44620

llvm-svn: 334469
2018-06-12 03:29:39 +00:00
Ryan Prichard 1e216a59f6 [builtins] Delay emutls deallocation for one round
Summary:
With Android/Bionic, delay deallocation to round 2 of 4. It must run after
C++ thread_local destructors have been called, but before the final 2
rounds, because emutls calls free, and jemalloc then needs another 2
rounds to free its thread-specific data.

Fixes https://github.com/android-ndk/ndk/issues/687

Reviewers: cmtice, srhines, jyknight, chh, echristo

Reviewed By: srhines, chh, echristo

Subscribers: echristo, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46978

llvm-svn: 334463
2018-06-12 01:32:26 +00:00
Ryan Prichard 31e4a550a3 [builtins] emutls cleanup: determine header size using sizeof
Summary: Also add a few post-#else/#endif comments

Reviewers: echristo, srhines

Reviewed By: echristo

Subscribers: chh, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47861

llvm-svn: 334462
2018-06-12 01:32:25 +00:00
Matt Morehouse a4374ab222 [CMake] Remove -Wno-maybe-uninitialized from passthrough vars.
Fixes http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/
buildbot.

llvm-svn: 334458
2018-06-12 00:37:06 +00:00
Peter Wu 1c05c95739 [sanitizer] Add fgets, fputs and puts into sanitizer_common
Summary:
Add fgets, fputs and puts to sanitizer_common. This adds ASAN coverage
for these functions, extends MSAN support from fgets to fputs/puts and
extends TSAN support from puts to fputs.

Fixes: https://github.com/google/sanitizers/issues/952

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46545

llvm-svn: 334450
2018-06-11 22:58:04 +00:00
Alex Shlyapnikov ed32baa84b [Sanitizers] %tool_options -> %env_tool_opts in allocator_returns_null.cc
Use proper substitution in the common allocator_returns_null.cc test.

llvm-svn: 334438
2018-06-11 21:04:24 +00:00
Alex Shlyapnikov 406385505d [Sanitizers] Move allocator_returns_null.cc test to common.
Summary:
Add allocator_returns_null.cc test to sanitizer_common and
remove all sanitizer-specific ones except:
- HWASan is not covered by sanitizer_common
- TSan allocator does not have comprehensive error reporting yet

Reviewers: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47971

llvm-svn: 334433
2018-06-11 19:45:59 +00:00
Alex Shlyapnikov 789494026e [Sanitizers] Move pvalloc overflow tests to common.
Summary:
Now all sanitizers with improved allocator error reporting are covered
by these common tests.

Also, add pvalloc-specific checks to LSan.

HWASan is not covered by sanitizer_common, hence its own pvalloc
and other allocator tests.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47970

llvm-svn: 334424
2018-06-11 17:33:53 +00:00
Kostya Kortchinsky 4410e2c43f [scudo] Improve the scalability of the shared TSD model
Summary:
The shared TSD model in its current form doesn't scale. Here is an example of
rpc2-benchmark (with default parameters, which is threading heavy) on a 72-core
machines (defaulting to a `CompactSizeClassMap` and no Quarantine):
- with tcmalloc: 337K reqs/sec, peak RSS of 338MB;
- with scudo (exclusive): 321K reqs/sec, peak RSS of 637MB;
- with scudo (shared): 241K reqs/sec, peak RSS of 324MB.

This isn't great, since the exclusive model uses a lot of memory, while the
shared model doesn't even come close to be competitive.

This is mostly due to the fact that we are consistently scanning the TSD pool
starting at index 0 for an available TSD, which can result in a lot of failed
lock attempts, and touching some memory that needs not be touched.

This CL attempts to make things better in most situations:
- first, use a thread local variable on Linux (intead of pthread APIs) to store
  the current TSD in the shared model;
- move the locking boolean out of the TSD: this allows the compiler to use a
  register and potentially optimize out a branch instead of reading it from the
  TSD everytime (we also save a tiny bit of memory per TSD);
- 64-bit atomic operations on 32-bit ARM platforms happen to be expensive: so
  store the `Precedence` in a `uptr` instead of a `u64`. We lose some
  nanoseconds of precision and we'll wrap around at some point, but the benefit
  is worth it;
- change a `CHECK` to a `DCHECK`: this should never happen, but if something is
  ever terribly wrong, we'll crash on a near null AV if the TSD happens to be
  null;
- based on an idea by dvyukov@, we are implementing a bound random scan for
  an available TSD. This requires computing the coprimes for the number of TSDs,
  and attempting to lock up to 4 TSDs in an random order before falling back to
  the current one. This is obviously slightly more expansive when we have just
  2 TSDs (barely noticeable) but is otherwise beneficial. The `Precedence` still
  basically corresponds to the moment of the first contention on a TSD. To seed
  on random choice, we use the precedence of the current TSD since it is very
  likely to be non-zero (since we are in the slow path after a failed `tryLock`)

With those modifications, the benchmark yields to:
- with scudo (shared): 330K reqs/sec, peak RSS of 327MB.

So the shared model for this specific situation not only becomes competitive but
outperforms the exclusive model. I experimented with some values greater than 4
for the number of TSDs to attempt to lock and it yielded a decrease in QPS. Just
sticking with the current TSD is also a tad slower. Numbers on platforms with
less cores (eg: Android) remain similar.

Reviewers: alekseyshl, dvyukov, javed.absar

Reviewed By: alekseyshl, dvyukov

Subscribers: srhines, kristof.beyls, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47289

llvm-svn: 334410
2018-06-11 14:50:31 +00:00
Peter Wu 1db004ebdd [ASAN] Fix crash on i?86-linux (32-bit) against glibc 2.27 and later
Summary:
Running sanitized 32-bit x86 programs on glibc 2.27 crashes at startup, with:

    ERROR: AddressSanitizer: SEGV on unknown address 0xf7a8a250 (pc 0xf7f807f4 bp 0xff969fc8 sp 0xff969f7c T16777215)
    The signal is caused by a WRITE memory access.
    #0 0xf7f807f3 in _dl_get_tls_static_info (/lib/ld-linux.so.2+0x127f3)
    #1 0xf7a92599  (/lib/libasan.so.5+0x112599)
    #2 0xf7a80737  (/lib/libasan.so.5+0x100737)
    #3 0xf7f7e14f in _dl_init (/lib/ld-linux.so.2+0x1014f)
    #4 0xf7f6eb49  (/lib/ld-linux.so.2+0xb49)

The problem is that glibc changed the calling convention for the GLIBC_PRIVATE
symbol that sanitizer uses (even when it should not, GLIBC_PRIVATE is exactly
for symbols that can change at any time, be removed etc.), see
https://sourceware.org/ml/libc-alpha/2017-08/msg00497.html

Fixes https://github.com/google/sanitizers/issues/954

Patch By: Jakub Jelinek

Reviewed By: vitalybuka, Lekensteyn

Differential Revison: https://reviews.llvm.org/D44623

llvm-svn: 334363
2018-06-10 11:17:47 +00:00
Alex Shlyapnikov b24e6a9905 [MSan] Switch to common aligned allocs tests.
Summary:
Switch to aligned_alloc and posix_memalign tests in sanitizer_common
and delete the MSan-specific ones.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47968

llvm-svn: 334340
2018-06-09 00:47:37 +00:00
Alex Shlyapnikov 643600566d [MSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47793

llvm-svn: 334338
2018-06-08 23:31:42 +00:00
Alex Shlyapnikov 84f8ec5c72 [Sanitizers] Use proper substitution in common allocator test.
Use %env_tool_ops instead of %tool_options.

llvm-svn: 334332
2018-06-08 21:58:22 +00:00
Walter Lee 5a780ee9a9 [asan, myriad] Use local pool for new/delete when ASan run-time is not up
This can happen on Myriad RTEMS so needs to be handled.

Differential Revision: https://reviews.llvm.org/D47916

llvm-svn: 334329
2018-06-08 21:49:38 +00:00
Vitaly Buka 683e2d636d [asan] Re-enable no_asan_gen_globals for 32-bit
PR22682. Bug in gold was fixed 3 years ago.

llvm-svn: 334319
2018-06-08 21:04:44 +00:00
Alex Shlyapnikov dcf0097962 [Sanitizers] Check alignment != 0 for aligned_alloc and posix_memalign
Summary:
Move the corresponding tests to the common folder (as all of the
sanitizer allocators will support this feature soon) and add the checks
specific to aligned_alloc to ASan and LSan allocators.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47924

llvm-svn: 334316
2018-06-08 20:40:35 +00:00
David Carlier 6dedf2b5d8 [Xray] logging forgotten header
pthread.h missing for pthread_key* functions.

Reviewers: dberris

Reviewed By: dberris

Differential Revison: https://reviews.llvm.org/D47933

llvm-svn: 334272
2018-06-08 08:42:37 +00:00
Dean Michael Berris b81372b56b [XRay][compiler-rt] Cleanup some internal XRay utilities
This change uses 'const' for the retryingWriteAll(...) API and removes
unnecessary 'static' local variables in getting the temporary filename.

llvm-svn: 334267
2018-06-08 07:48:03 +00:00
Dean Michael Berris af4d583ea8 [XRay][compiler-rt] Remove reliance on C++ ABI features
Summary:
This fixes http://llvm.org/PR32274.

This change adds a test to ensure that we're able to link XRay modes and
the runtime to binaries that don't need to depend on the C++ standard
library or a C++ ABI library. In particular, we ensure that this will work
with C programs compiled+linked with XRay.

To make the test pass, we need to change a few things in the XRay
runtime implementations to remove the reliance on C++ ABI features. In
particular, we change the thread-safe function-local-static
initialisation to use pthread_* instead of the C++ features that ensure
non-trivial thread-local/function-local-static initialisation.

Depends on D47696.

Reviewers: dblaikie, jfb, kpw, eizan

Reviewed By: kpw

Subscribers: echristo, eizan, kpw, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46998

llvm-svn: 334262
2018-06-08 04:00:07 +00:00
Alex Shlyapnikov a2670df602 [HWASan] Disable allocator_returns_null.cc test on Android.
Disable test added in D47798 on Android and investigate why "new"
behaves differently there.

llvm-svn: 334255
2018-06-08 00:56:49 +00:00
Alex Shlyapnikov af12729185 [HWASan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47798

llvm-svn: 334248
2018-06-07 23:33:33 +00:00
Kostya Serebryany 111d0b2aec [libFuzzer] When printing NEW_FUNC, use 1-base indexing.
Summary: Otherwise we print things like [0/1] which is visually confusing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47837

llvm-svn: 334234
2018-06-07 21:15:24 +00:00
Eric Fiselier 84be761332 [CMake] Fix Libc++ Modules build.
When building the dylib, the C++ headers are fundamentally non-module.
They require special versions of the headers in order to provide C++03 and
legacy ABI definitions. This causes ODR issues when modules are enabled
during both the build and the usage of the libc++ headers.

This patch fixes the build error by disabling modules when building the
libc++ sources.

llvm-svn: 334220
2018-06-07 19:57:43 +00:00
Douglas Yung 20fa3d2504 Silence a -Wconstant-logical-operand warning.
llvm-svn: 334214
2018-06-07 18:51:47 +00:00
Petr Hosek 157a5c2f09 [Fuzzer] Update the header path for fdio/spawn.h on Fuchsia
The path now includes lib/ prefix.

Differential Revision: https://reviews.llvm.org/D47866

llvm-svn: 334213
2018-06-07 18:41:35 +00:00
Petr Hosek 6367bca339 [CMake] Filter out -z,defs when building custom libc++
-z,defs is incompatible with sanitizers so we need to filter it out
from the linker flags before passing them to the libc++ build.

Differential Revision: https://reviews.llvm.org/D47865

llvm-svn: 334212
2018-06-07 18:27:27 +00:00
Kamil Rytarowski b9c78e41ba Disable recursive interceptors in signal(3)/MSan
Summary:
signal(3) on NetBSD calls internally sigaction(2).

Without disabling the recursive interceptor, there are
false positives about uninitialized memory reads inside libc.

This change fixes execution of such programs as sh(1) and
vmstat(1) in the NetBSD userland.

Sponsored by <The NetBSD Foundation>

Reviewers: eugenis, vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47650

llvm-svn: 334168
2018-06-07 07:55:20 +00:00
Kostya Serebryany 67af99235f [libFuzzer] make the corpus elements aware of their data flow traces
llvm-svn: 334158
2018-06-07 01:40:20 +00:00
Kostya Serebryany 7a88f19099 [libFuzzer] simplify a test, hopefully to fix the bot
llvm-svn: 334156
2018-06-07 01:18:43 +00:00
Vitaly Buka 661d81ef61 [sanitizer] Don't use internal_unlink on Windows
llvm-svn: 334152
2018-06-07 00:26:06 +00:00
Kostya Serebryany 4d9fd7a266 [libFuzzer] remove an experimental flag -use_feature_frequency
llvm-svn: 334146
2018-06-06 23:24:41 +00:00
Petr Hosek 3a32b41024 [Fuzzer] Use private libc++ even for Fuchsia
On Fuchsia, we use libc++ compiled with ASan for our ASan built
executable which means we cannot use the same libc++ for libFuzzer when
building fuzz targets, instead we'll link a custom internal libc++ into
Fuchsia's build of libFuzzer like we already do on Linux.

Differential Revision: https://reviews.llvm.org/D47835

llvm-svn: 334144
2018-06-06 22:59:14 +00:00
Petr Hosek 3cf6db2d56 [CMake] Passthrough additional flags to custom libcxx CMake build
This is needed when we're cross-compiling compiler-rt.

Differential Revision: https://reviews.llvm.org/D47834

llvm-svn: 334139
2018-06-06 22:10:12 +00:00
Vitaly Buka 72fd23c9c5 [sanitizer] Replace deprecated mktemp with mkstemp
llvm-svn: 334138
2018-06-06 22:05:48 +00:00
Vitaly Buka 4ef9117b49 [sanitizer] Cleanup ReadFileToVector and ReadFileToBuffer
Summary:
Added unit-test.
Fixed behavior of max_len argument.
Call read syscall with all available buffer, not just a page.

Reviewers: eugenis

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46618

llvm-svn: 334130
2018-06-06 20:53:43 +00:00
Walter Lee 381072c417 [asan] Fix Myraid RTEMS port broken by r333985
Add nop CheckASLR() function.

llvm-svn: 334102
2018-06-06 15:39:47 +00:00
Ilya Biryukov b7c3745006 Fix compile error with libstdc++.
By adding a ctor to create fuzzer_allocator<T> from fuzzer_allocator<U>.
This mimics construcotrs of std::allocator<T>.

Without the constructors, some versions of libstdc++ can't compile
`vector<bool, fuzzer_allocator<bool>>`.

llvm-svn: 334077
2018-06-06 09:22:19 +00:00
Dean Michael Berris c561970b27 [XRay][compiler-rt] Refactor recursion guard for Basic and FDR Mode
Summary:
This change extracts the recursion guard implementation from FDR Mode
and updates it to do the following:

- Do the atomic operation correctly to be signal-handler safe.

- Make it usable in both FDR and Basic Modes.

Before this change, the recursion guard relied on an unsynchronised read
and write on a volatile thread-local. A signal handler could then run in
between the read and the write, and then be able to run instrumented
code as part of the signal handling. Using an atomic exchange instead
fixes that by doing a proper mutual exclusion even in the presence of
signal handling.

Reviewers: kpw, eizan, jfb

Reviewed By: eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47696

llvm-svn: 334064
2018-06-06 06:07:48 +00:00
Kostya Serebryany 1fd005f552 [libFuzzer] initial implementation of -data_flow_trace. It parses the data flow trace and prints the summary, but doesn't use the information in any other way yet
llvm-svn: 334058
2018-06-06 01:23:29 +00:00
Vitaly Buka b89704fa6f [lsan] Do not check for leaks in the forked process
Summary:
If calling process had threads then forked process will fail to detect
references from them.

Fixes https://github.com/google/sanitizers/issues/836

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47751

llvm-svn: 334036
2018-06-05 18:15:57 +00:00
Alex Shlyapnikov 236c3f9c4a [LSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47645

llvm-svn: 334034
2018-06-05 18:02:09 +00:00
Dean Michael Berris 4c1fdcfb97 [XRay][compiler-rt] Use DCHECK instead of assert(...) (NFC)
Use DCHECK instead of assert(...) in the XRay runtime.

llvm-svn: 334002
2018-06-05 10:27:20 +00:00
Dean Michael Berris 13b2fcc4e6 [XRay][compiler-rt] Use static instead of inline (NFC)
We don't actually need to support multiple definitions of the functions
in FDR mode, but rather want to make sure that the implementation-detail
functions are marked as 'static' instead. This allows the inliner to do
its magic better for these functions too, since inline functions must
have a unique address across translation units.

llvm-svn: 334001
2018-06-05 10:18:39 +00:00
Dean Michael Berris abbeb4cbf7 [XRay][compiler-rt] Remove __xray:: in some places (NFC)
This is a cosmetic change to remove unnecessary full-qualifications of
types/functions that are already in the __xray:: namespace.

llvm-svn: 334000
2018-06-05 10:12:58 +00:00
Dean Michael Berris bd37b57b4d [XRay][compiler-rt] Remove namespace __xray_fdr_internal (NFC)
We no longer need the __xray_fdr_internal namespace.

llvm-svn: 333998
2018-06-05 10:01:45 +00:00
Dean Michael Berris 7f88ea1fa2 [XRay][compiler-rt] Merge XRay FDR mode into a single file (NFC)
We planned to have FDR mode's internals unit-tested but it turns out
that we can just use end-to-end testing to verify the implementation.
We're going to move towards that approach more and more going forward,
so we're merging the implementation details of FDR mode into a single
.cc file.

We also avoid globbing in the XRay test helper macro, and instead list
down the files from the lib directory.

llvm-svn: 333986
2018-06-05 08:20:54 +00:00
Kamil Rytarowski 7d260775f3 Introduce CheckASLR() in sanitizers
Summary:
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD.

Introduce a generic CheckASLR() routine, that implements a check for the
current process. This flag depends on the global or per-process settings.

There is no simple way to disable ASLR in the build process from the
level of a sanitizer or during the runtime execution.

With ASLR enabled sanitizers that operate over the process virtual address
space can misbehave usually breaking with cryptic messages.

This check is dummy for !NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: cryptoad, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47442

llvm-svn: 333985
2018-06-05 07:29:23 +00:00
Dean Michael Berris 5eaaff6095 [XRay][compiler-rt] Remove __sanitizer:: from namespace __xray (NFC)
This is a non-functional change that removes the full qualification of
functions in __sanitizer:: being used in __xray.

llvm-svn: 333983
2018-06-05 06:12:42 +00:00
Dean Michael Berris 8ba925d2c3 [XRay][compiler-rt] Remove reliance on C++ ABI from BufferQueue
Summary:
This is part of the work to address http://llvm.org/PR32274.

We remove the calls to array-placement-new and array-delete. This allows
us to rely on the internal memory management provided by
sanitizer_common/sanitizer_internal_allocator.h.

Reviewers: eizan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47695

llvm-svn: 333982
2018-06-05 03:46:54 +00:00
David Carlier a6303a9866 [TSan] FreeBSD / intercept thr_exit
intercepting thr_exit to terminate threads under FreeBSD.
Unblock few unit tests hanging.

Reviewers: krytarowski, vitalybuka, emaste

Reviewed By: dvyukov		

Differential Revision: https://reviews.llvm.org/D47677

M    lib/tsan/rtl/tsan_interceptors.cc

llvm-svn: 333870
2018-06-04 06:02:38 +00:00
Dean Michael Berris d12cb59c7a [XRay][compiler-rt] Remove RTTI requirement from XRay Builds
XRay doesn't use RTTI and doesn't need it. We disable it explicitly in
the CMake config, similar to how the other sanitizers already do it.

Part of the work to address http://llvm.org/PR32274.

llvm-svn: 333867
2018-06-04 04:22:26 +00:00
Petr Hosek 8908aa4e30 [Fuzzer] Migrate Fuchsia port from launchpad to fdio_spawn
fdio_spawn is the new public API for launching processes in Fuchsia.

Differential Revision: https://reviews.llvm.org/D47649

llvm-svn: 333809
2018-06-02 01:17:10 +00:00
Kostya Serebryany d74d04a6c5 Add weak definitions of trace-cmp hooks to dfsan
Summary:
This allows to build and link the code with e.g.
-fsanitize=dataflow -fsanitize-coverage=trace-pc-guard,pc-table,func,trace-cmp
w/o providing (all) the definitions of trace-cmp hooks.

This is similar to dummy hooks provided by asan/ubsan/msan for the same purpose,
except that some of the hooks need to have the __dfsw_ prefix
since we need dfsan to replace them.

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47605

llvm-svn: 333796
2018-06-01 21:59:25 +00:00
Walter Lee e2f884d73f [asan, myriad] Implement aligned local pool allocation
Extend the local pool allocation support to posix_memalign.

Differential Revision: https://reviews.llvm.org/D47642

llvm-svn: 333788
2018-06-01 21:29:05 +00:00
Walter Lee bf03bfb0a3 [asan, myriad] Configure platform interceptors
Myriad only uses the platform interceptors for memory allocation
routines.  Configure them properly.

Also add a missing guard around aligned alloc interceptor.

Differential Revision: https://reviews.llvm.org/D47641

llvm-svn: 333784
2018-06-01 19:47:41 +00:00
David Carlier 45cdbdac06 [XRay] Fix supposely failing test for FreeBSD
One test was marked as XFAIL for FreeBSD however still running when launching make check-xray

Reviewers: krytarowski, vitalybuka

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D47622

llvm-svn: 333733
2018-06-01 09:15:44 +00:00
Yvan Roux 3b480d1858 [ASAN] Sanitize testsuite for ARM.
Address failures exhibited by ARMv8 bot in Thumb mode:

- Fix logic for fast unwinding support (i.e feature is not available for Thumb)
- Fix Unsupported and Requires rules to handle armv8 as well as soft and hard
  float targets
- Un-xfail passing tests

Differential Revision: https://reviews.llvm.org/D47575

llvm-svn: 333729
2018-06-01 06:23:36 +00:00
Reid Kleckner d44a6eab40 Fix GCC 4.8.5 build by avoiding braced default member initializer
Use internal_memset instead.  Should revive all Linux Chromium ToT bots.

llvm-svn: 333678
2018-05-31 18:26:36 +00:00
Sylvestre Ledru ac804c2238 Define SIZEOF_STRUCT_USTAT for 32bit sparc.
Patch landed on gcc upstream:
27453e962b
Patch by Matthias Klose

llvm-svn: 333644
2018-05-31 12:41:15 +00:00
Stephan Bergmann af3226752b Missing include
(For some reason, my Mac build complained about unknown pthread_once_t and
PTHREAD_ONCE_INIT, but not about pthread_once itself.)

llvm-svn: 333638
2018-05-31 09:26:31 +00:00
Dean Michael Berris d1fe506694 [XRay] Fixup: Remove unnecessary type alias
Follow-up to D45758.

llvm-svn: 333628
2018-05-31 05:25:47 +00:00
Dean Michael Berris 16c865b071 [XRay] Fixup: Explicitly call std::make_tuple(...)
Follow-up to D45758.

llvm-svn: 333627
2018-05-31 05:02:11 +00:00
Dean Michael Berris ca856e07de [XRay] Fixup: Address some warnings breaking build
Follow-up to D45758.

llvm-svn: 333625
2018-05-31 04:55:11 +00:00
Dean Michael Berris 1eb8c206cd [XRay][profiler] Part 3: Profile Collector Service
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements a centralised collector for `FunctionCallTrie`
instances, associated per thread. It maintains a global set of trie
instances which can be retrieved through the XRay API for processing
in-memory buffers (when registered). Future changes will include the
wiring to implement the actual profiling mode implementation.

This central service provides the following functionality:

*  Posting a `FunctionCallTrie` associated with a thread, to the central
   list of tries.

*  Serializing all the posted `FunctionCallTrie` instances into
   in-memory buffers.

*  Resetting the global state of the serialized buffers and tries.

Depends on D45757.

Reviewers: echristo, pelikan, kpw

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D45758

llvm-svn: 333624
2018-05-31 04:33:52 +00:00
Kostya Serebryany 980e45fe55 [libFuzzer] add collect_data_flow.py that allows to run the data-flow tracer several times on subsets of inputs bytes, to overcome DFSan out-of-label failures
llvm-svn: 333616
2018-05-31 01:27:07 +00:00
Reid Kleckner b54ac414d1 [asan] Remove unneeded VirtualQuery from exception handler
We don't use the result of the query, and all tests pass if I remove it.
During startup, ASan spends a fair amount of time in this handler, and
the query is much more expensive than the call to commit the memory.

llvm-svn: 333595
2018-05-30 21:21:18 +00:00
Walter Lee 3dc6cea0e3 [asan, myriad] Simplify main thread handling
On Myriad RTEMS, we don't need to treat the main thread differently.
The existing thread hooks will do the right thing, so get rid of all
the unneeded special logic.

Differential Revision: https://reviews.llvm.org/D47502

llvm-svn: 333504
2018-05-30 04:57:29 +00:00
Walter Lee 0a70e9db03 [asan, myriad] Reset shadow memory during exit
Reset shadow memory during exit.  Also update a cut-and-paste comment,
and do some minor refactoring of InitializeShadowMemory.

Differential Revision: https://reviews.llvm.org/D47501

llvm-svn: 333503
2018-05-30 04:57:04 +00:00
George Burgess IV d30bcab70e Fix a typo; NFC
llvm-svn: 333331
2018-05-26 02:28:43 +00:00
Vitaly Buka 4add8a1f4e [safestack] Lazy initialization of interceptors
Interceptors initialization may try to allocate memory and to call not
initialized allocator.

It's similar to r326025 for CFI.

llvm-svn: 333329
2018-05-26 01:18:32 +00:00
Petr Hosek 796bb7d4ce [Fuzzer] Update _zx_port_wait function use in Fuchsia port
Fuchsia's _zx_port_wait no longer takes the count argument. This
change also updates all symbol uses to use the underscored versions.

Differential Revision: https://reviews.llvm.org/D47403

llvm-svn: 333328
2018-05-26 01:02:34 +00:00
H.J. Lu 11c1c266cc Revert the last test commit
llvm-svn: 333310
2018-05-25 20:51:45 +00:00
H.J. Lu caab7c8cca This is a test commit to verify repository access
llvm-svn: 333309
2018-05-25 20:51:10 +00:00
David Carlier f6311c669f [MemorySanitizer] fix mmap test for oses not implementing MAP_NORESERVE flag
Reviewers: krytarowski, eugenis

Reviewed By: eugenis

Differential Revision: https://review.llvm.org/D47146

llvm-svn: 333306
2018-05-25 20:30:39 +00:00
Vitaly Buka 540d7e528d [libFuzzer] Avoid optimization of "abs(x) < 0"
llvm-svn: 333295
2018-05-25 18:27:12 +00:00
Vitaly Buka b161db099d Revert "[libFuzzer] [NFC] Generalize DSO tests to work even when files are moved."
Breaks libFuzzer tests.

This reverts commit r333243.

llvm-svn: 333257
2018-05-25 07:18:29 +00:00
George Karpenkov 698b0a674d [libFuzzer] [NFC] Generalize fdmask test.
The ordering of stderr/stdout writes might not be guaranteed,
and is irrelevant for this test.

Differential Revision: https://reviews.llvm.org/D47294

llvm-svn: 333245
2018-05-24 23:55:54 +00:00
George Karpenkov b4ccb3ab08 [libFuzzer] [NFC] Do not remove SanitizerLintCheck from libFuzzer tests
With %run commands being present now, we want to make sure that they
stay present.

Differential Revision: https://reviews.llvm.org/D47293

llvm-svn: 333244
2018-05-24 23:55:54 +00:00
George Karpenkov fe44eb4cf1 [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 333243
2018-05-24 23:55:53 +00:00
George Karpenkov 464d1415b2 [libFuzzer] Fix multi-arch build support for fuzzer tests.
The bug was not previously exposed as only one architecture is currently
supported for libFuzzer.

Differential Revision: https://reviews.llvm.org/D47287

llvm-svn: 333242
2018-05-24 23:55:53 +00:00
George Karpenkov 02c85f1489 [libFuzzer] Run libFuzzer unit tests only on host architecture.
LIT tests are used to test other cross-compiled architectures,
unit tests are only run on the host.
NFC now as currently only a single architecture is supported.

Differential Revision: https://reviews.llvm.org/D47286

llvm-svn: 333241
2018-05-24 23:55:52 +00:00
George Karpenkov 0dcb6902f1 [libFuzzer] XFAIL a few libFuzzer tests on iOS.
NFC for now.

Differential Revision: https://reviews.llvm.org/D47284

llvm-svn: 333240
2018-05-24 23:55:52 +00:00
George Karpenkov e7e3fc26ee [libFuzzer] Fix a typo in CMake configuration.
NFC now when libFuzzer supports only one architecture,
will stop being NFC after multiple architectures are supported.

Differential Revision: https://reviews.llvm.org/D47283

llvm-svn: 333239
2018-05-24 23:55:51 +00:00
Azharuddin Mohammed 06e0fd3bc1 Revert "[cmake] [ARM] Check if VFP is supported before including any VFP builtins"
This reverts commit 2a10f5da5acb1b51d0a2ecf13dca0bf1de859db2.

llvm-svn: 333232
2018-05-24 21:36:27 +00:00
Azharuddin Mohammed 80968a9aba [cmake] [ARM] Check if VFP is supported before including any VFP builtins
Summary:
rL325492 disables FPU features when using soft floating point
(-mfloat-abi=soft), which is used internally when building for armv7. This
causes errors with builtins that utililize VFP instructions. With this change
we first check if VFP is enabled (by checking if the preprocessor macro
__VFP_FP__ is defined) before including such builtins.

Reviewers: rengolin, samsonov, compnerd, smeenai, javed.absar, peter.smith

Reviewed By: peter.smith

Subscribers: peter.smith, mgorny, kristof.beyls, chrib, llvm-commits

Differential Revision: https://reviews.llvm.org/D47217

llvm-svn: 333216
2018-05-24 18:53:16 +00:00
Craig Topper 383fe5c866 sanitizer: Use pre-computed size of struct ustat for Linux
<sys/ustat.h> has been removed from glibc 2.28 by:

commit cf2478d53ad7071e84c724a986b56fe17f4f4ca7
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Sun Mar 18 11:28:59 2018 +0800

Deprecate ustat syscall interface
This patch uses pre-computed size of struct ustat for Linux to fix

https://bugs.llvm.org/show_bug.cgi?id=37418

Patch by H.J. Lu.

Differential Revision: https://reviews.llvm.org/D47281

llvm-svn: 333213
2018-05-24 17:59:47 +00:00
Igor Kudrin 13f8a77d79 [tsan] Fix a failure in test java_race_pc on AArch64 after r333071
llvm-svn: 333180
2018-05-24 12:26:20 +00:00
Kostya Serebryany 4d53b744ca [libFuzzer] DataFlow tracer now tags a subset of the input. A separate script merges traces from the subsets
llvm-svn: 333149
2018-05-24 01:43:48 +00:00
Kostya Serebryany 500ca8713c [libFuzzer] fix two off-by-ones (!!) in the data flow tracer
llvm-svn: 333142
2018-05-23 23:55:54 +00:00
Jake Ehrlich 767d92e3fb [fuchsia] Add line buffering in RawWrite
This change causes RawWrite to buffer upto 128 bytes or until
a line is reached. This helps group calls into more readable
lines.

llvm-svn: 333136
2018-05-23 22:27:12 +00:00
Kostya Serebryany 4925392897 [libFuzzer] change the output format for the DataFlow tracer
llvm-svn: 333122
2018-05-23 20:57:11 +00:00
Kostya Serebryany 28fe54fac8 [libFuzzer] add a stress test for the DataFlow tracer
llvm-svn: 333119
2018-05-23 20:23:33 +00:00
Max Moroz fe97441c2c [libFuzzer] Don't complain about lack of interesting inputs when -runs=0.
Summary:
The most common usecase for -runs=0 is for generating code coverage
over some corpus. Coverage reports based on sancov are about to be deprecated,
which means some external coverage solution will be used, e.g. Clang source
based code coverage, which does not use any sancov instrumentations and thus
libFuzzer would consider any input to be not interesting in that case.

Reviewers: kcc

Reviewed By: kcc

Subscribers: alex, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47271

llvm-svn: 333116
2018-05-23 19:42:30 +00:00
Kostya Kortchinsky 08c630457a [sanitizer] Define UINTPTR_MAX
Summary:
`sanitizer_internal_defs.h` didn't have this define, which will be useful in
an upcoming CL.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47270

llvm-svn: 333109
2018-05-23 18:13:21 +00:00
Kamil Rytarowski 5b1b594f64 Fix internal_mmap() on 32-bit NetBSD platforms
There is need to use internal_syscall64() instead of internal_syscall_ptr().
The offset argument of type off_t is always 64-bit.

llvm-svn: 333075
2018-05-23 10:37:00 +00:00
Igor Kudrin c0099f9be7 [sanitizer] Reland "Be more accurate when calculating the previous instruction address on ARM."
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 333071
2018-05-23 09:18:10 +00:00
Kostya Serebryany 7f5bf83f53 [libFuzzer] modify -print_corpus_stats to print whether the input reaches the focus function
llvm-svn: 333048
2018-05-23 01:42:53 +00:00
Petr Hosek aa6d13e501 [CMake] Support builtins as Clang default rtlib in compiler-rt
Use compiler-rt builtins when selected as default Clang rtlib and avoid
explicitly passing -rtlib= flag to avoid the "argument unused during
compilation" warning.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47115

llvm-svn: 333037
2018-05-22 22:58:48 +00:00
Evgeniy Stepanov 45f8986b80 [asan] Use dynamic allocator space address on Android/AArch64.
Summary:
We need one library to support all of 39, 42 and 48 bit VMAs, and
there is no common address that works for all of them.

Reviewers: kcc, alekseyshl, javed.absar

Subscribers: rengolin, srhines, kubamracek, kristof.beyls, llvm-commits, cryptoad

Differential Revision: https://reviews.llvm.org/D47160

llvm-svn: 333025
2018-05-22 20:44:45 +00:00
Petr Hosek 1ffaa48d4a [CMake] Support libc++ as Clang default stdlib in compiler-rt
Use libc++ when selected as default Clang stdlib and avoid checking
C++ compiler when using the in-tree version of libc++.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47100

llvm-svn: 333010
2018-05-22 18:33:27 +00:00
Petr Hosek 4ca5af0721 [CMake] Silence unused variable warning in compiler check
This is breaking the compiler check.

llvm-svn: 332936
2018-05-22 02:53:32 +00:00
Petr Hosek f6dda7cb44 [CMake] Use a different source depending on C++ support
When using system C++ library, assume we have a working C++ compiler and
try to compile a complete C++ program. When using in tree C++ library,
only check the C compiler since the C++ library likely won't have been
built yet at time of running the check.

Differential Revision: https://reviews.llvm.org/D47169

llvm-svn: 332924
2018-05-22 01:01:56 +00:00
Walter Lee ead3b3487b [asan] Make GetCurrentThread RTEMS-friendly
On RTEMS, system and user code all live in a single binary and address
space. There is no clean separation, and instrumented code may
execute before the ASan run-time is initialized (or after it has been
destroyed).

Currently, GetCurrentThread() may crash if it's called before ASan
run-time is initialized. Make it return nullptr instead.

Similarly, fix __asan_handle_no_return so that it gives up rather than
try something that may crash.

Differential Revision: https://reviews.llvm.org/D46459

llvm-svn: 332888
2018-05-21 20:43:36 +00:00
Kostya Serebryany 69c2b71a51 [libFuzzer] reinstate -dump_coverage, which is still in use (reverts r332036)
llvm-svn: 332876
2018-05-21 19:47:00 +00:00