Commit Graph

10585 Commits

Author SHA1 Message Date
Roman Lebedev f07c5a0e9c [NFC] Rename test/ubsan/TestCases/{ImplicitCast => ImplicitConversion}
Just to be consistent with the rest.
I should have done that in the commit itself, but the filepaths
is one thing i forgot to verify :S

llvm-svn: 338307
2018-07-30 21:11:34 +00:00
Roman Lebedev 0c234517fd [compiler-rt] integer-truncation-blacklist.c: XFAIL on android/ios
The Builder sanitizer-x86_64-linux-android is failing
starting with rL338287 / D48959.

It runs the tests via android_compile.py, so i'm not sure this
is actually *this* issue:
  https://code.google.com/p/address-sanitizer/issues/detail?id=316
but this seems oddly similar to the other XFAIL'ed cases...

Right now that seems to be the only failing builder,
so i *think* it makes sense to try to just blacklist it for now.

llvm-svn: 338296
2018-07-30 20:05:24 +00:00
Roman Lebedev eb4a9bc343 [compiler-rt][ubsan] Implicit Conversion Sanitizer - integer truncation - compiler-rt part
Summary:
This is a compiler-rt part.
The clang part is D48958.

See [[ https://bugs.llvm.org/show_bug.cgi?id=21530 | PR21530 ]], https://github.com/google/sanitizers/issues/940.

Reviewers: #sanitizers, samsonov, vsk, rsmith, pcc, eugenis, kcc, filcab

Reviewed By: #sanitizers, vsk, filcab

Subscribers: llvm-commits, eugenis, filcab, kubamracek, dberris, #sanitizers, regehr

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D48959

llvm-svn: 338287
2018-07-30 18:58:30 +00:00
Azharuddin Mohammed ec6bdf8d4f [cmake] [ARM] Exclude any VFP builtins if VFP is not supported
Summary:
rL325492 disables FPU features when using soft floating point
(-mfloat-abi=soft), which is used internally when building for arm. This causes
errors with builtins that utililize VFP instructions.

With this change we check if VFP is enabled (by checking if the preprocessor
macro __VFP_FP__ is defined), and exclude such builtins if it is not enabled.

Reviewers: rengolin, samsonov, compnerd, smeenai, javed.absar, peter.smith

Reviewed By: peter.smith

Subscribers: delcypher, peter.smith, mgorny, kristof.beyls, chrib, llvm-commits

Differential Revision: https://reviews.llvm.org/D47217

llvm-svn: 338284
2018-07-30 18:18:59 +00:00
Dean Michael Berris f1ceb0b376 [XRay][compiler-rt] FDR Mode: Use mmap instead of internal allocator
Summary:
This change moves FDR mode to use `internal_mmap(...)` from
sanitizer_common instead of the internal allocator interface. We're
doing this to sidestep the alignment issues we encounter with the
`InternalAlloc(...)` functions returning pointers that have some magic
bytes at the beginning.

XRay copies bytes into the buffer memory, and does not require the magic
bytes tracking the other sanitizers use when allocating/deallocating
buffers.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49972

llvm-svn: 338228
2018-07-30 05:56:42 +00:00
Fangrui Song 8c40e40a85 [asan] Fix typo
llvm-svn: 338225
2018-07-30 00:25:16 +00:00
Fangrui Song c0ca8089a2 Fix Asan-i386-calls-Test AddressSanitizer.ShadowGapTest on FreeBSD
0x22000000 happens to be on the left of a heap allocation and the error
message is different (heap-buffer-overflow).
FreeBSD NetBSD have larger SHADOW_OFFSET (0x40000000) but let's try not
using #ifdef here.

llvm-svn: 338208
2018-07-28 23:41:50 +00:00
Jonas Hahnfeld a7c9fe3762 [test] Use printf instead of C++ iostream, NFC.
This test fails with libc++ when built with MemorySanitizer. This
is because we link to an uninstrumented version of the library
so msan detects a nested error when calling std::cout << "...".
This can be easily avoided by using good old printf.

Differential Revision: https://reviews.llvm.org/D49867

llvm-svn: 338053
2018-07-26 18:23:40 +00:00
Dmitry Vyukov 04f97cf2f0 [tsan] Fix gcc pedantic warning
Fix gcc (7.2.0) pedantic warning
warning: extra ‘;’ [-Wpedantic]

Author: jasonl220 (Jason Lovett)
Review: https://reviews.llvm.org/D49817
llvm-svn: 338023
2018-07-26 13:02:54 +00:00
Fangrui Song cb1107ed14 [sanitizer] Include signal.h instead of sys/signal.h
llvm-svn: 338004
2018-07-26 04:50:33 +00:00
Max Moroz f55b8c6e91 [libFuzzer] Disable handle-unstable.test for ARM and delete duplicated test file.
Summary:
This change should fix the failures mentioned in
https://reviews.llvm.org/D49684#1175245

Reviewers: kevinwkt, morehouse, metzman

Reviewed By: kevinwkt, morehouse

Subscribers: kristof.beyls, delcypher, chrib, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49810

llvm-svn: 337949
2018-07-25 18:34:20 +00:00
Jeremy Morse 63c153edb2 [XRay tests] Don't filter test-critical calls
By default, xray filters events that takes less than 5uS from its log.
In this existing test, should printf complete very quickly this will
lead to test-critical function calls being filtered (i.e. print_parent_tid).
Given that we're not testing the filtering feature, disable it for this
test.

llvm-svn: 337929
2018-07-25 14:48:53 +00:00
David Carlier 62e06ff583 [Asan][Msan] Unit tests Disable some tests for FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D49784

llvm-svn: 337913
2018-07-25 12:38:23 +00:00
Petr Hosek 47e5fcba57 [profile] Support profiling runtime on Fuchsia
This ports the profiling runtime on Fuchsia and enables the
instrumentation. Unlike on other platforms, Fuchsia doesn't use
files to dump the instrumentation data since on Fuchsia, filesystem
may not be accessible to the instrumented process. We instead use
the data sink to pass the profiling data to the system the same
sanitizer runtimes do.

Differential Revision: https://reviews.llvm.org/D47208

llvm-svn: 337881
2018-07-25 03:01:35 +00:00
Matt Morehouse 5377b5b275 [sanitizer] Update symbolizer test.
llvm-svn: 337872
2018-07-24 23:50:42 +00:00
Max Moroz 8db0befc6d [libFuzzer] Handle unstable edges by disregarding unstable edges
Summary:
Added a new mode within flag -handle_unstable for new unstable handling algorithm that does the following:
    When an edge is shown as unstable, copy to UnstableCounters the value 0.
    During ApplyUnstableCounters we copy back the value 0 to ModuleInline8bitCounters if the edge was unstable.

This way we would be ignoring completely features that were collected through non-determinism.
Unstable hits would be counted as if it never hit.

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49684

llvm-svn: 337853
2018-07-24 21:02:44 +00:00
Kuba Mracek dc36389ea8 [tsan] Fix crash in objc_sync_enter/objc_sync_exit when using an Obj-C tagged pointer
Objective-C tagged pointers (either bottom-most or top-most bit is 1) are valid Obj-C objects but are not valid pointers. Make sure we don't crash on them when used in objc_sync_enter/objc_sync_exit. Instead, let's synchronize on a global object.

Differential Revision: https://reviews.llvm.org/D49707

llvm-svn: 337837
2018-07-24 16:19:06 +00:00
Igor Kudrin 356c2aeffe [profile] Fix finding the first and last directory separators on Windows.
Until now, our code preferred backslashes to slashes, whereas Windows
allows using both types of directory separators in one path string.

Differential Revision: https://reviews.llvm.org/D49664

llvm-svn: 337826
2018-07-24 13:06:19 +00:00
Igor Kudrin 63600c7487 [profile] Fix a possible memory leak in parseFilenamePattern().
Differential Revision: https://reviews.llvm.org/D49666

llvm-svn: 337823
2018-07-24 12:28:53 +00:00
Petr Hosek b8ab7e811b [sanitizer][fuzzer] Temporarily transition to ZX_TIME_INFINITE_OLD
This is a preparation for breaking change when all Zircon calls that
take time as an argument will start using signed valued. We will
transition back to ZX_TIME_INFITINE after all the changes to these
symbols are done and become part of the Fuchsia SDK.

Differential Revision: https://reviews.llvm.org/D49694

llvm-svn: 337802
2018-07-24 02:34:24 +00:00
Petr Hosek e2da642697 [sanitizer] Transition from _zx_vmar_... to _zx_vmar_..._old calls
This is a preparation for breaking changes to _zx_vmar_... calls.
We will transition back to _zx_vmar_... after all the changes to
these symbols are done and become part of the Fuchsia SDK.

Differential Revision: https://reviews.llvm.org/D49697

llvm-svn: 337801
2018-07-24 02:28:54 +00:00
Petr Hosek c14d513e0d Revert "[Fuzzer] Update path to libc++ headers"
This reverts commit r337775 since r337727 has been reverted in r337782.

llvm-svn: 337784
2018-07-24 00:34:55 +00:00
Petr Hosek 2cf5d81f04 [Fuzzer] Update path to libc++ headers
The path to headers which are installed into libc++ build directory
has changed in r337727 which broke the libFuzzer build.

Differential Revision: https://reviews.llvm.org/D49705

llvm-svn: 337775
2018-07-23 23:38:28 +00:00
Matthew Voss 1486d5c6e2 Reapply "[XRay] Remove scheduling dependency in fork_basic_logging.cc"
Summary:
Continuation of https://reviews.llvm.org/D49501

Second part of the test has an scheduling order when there shouldn't be.



Reviewers: dberris, ormris

Reviewed By: dberris, ormris

Subscribers: TWeaver

Differential Revision: https://reviews.llvm.org/D49559

llvm-svn: 337745
2018-07-23 21:22:02 +00:00
Max Moroz 1e954f78d1 [libFuzzer] Handle unstable edges by using minimum hit counts
Summary:
Created unstable_handle flag that takes 1 or 2, depending on the handling type.
Modified RunOne to accommodate the following heuristic:
    Use the first CollectFeatures to count how many features there are.
    If no new features, CollectFeatures like before.
    If there is new feature, we run CB 2 more times,
        Check which edges are unstable per input and we store the least amount of hit counts for each edge.
        Apply these hit counts back to inline8bitcounters so that CollectFeatures can work as intended.
Modified UnstableCounters to 8int_t and created a bitset UnstableSet to tell which edges are unstable.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, morehouse

Reviewed By: Dor1s, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49525

llvm-svn: 337696
2018-07-23 14:20:52 +00:00
Peter Collingbourne acf005676e Change the cap on the amount of padding for each vtable to 32-byte (previously it was 128-byte)
We tested different cap values with a recent commit of Chromium. Our results show that the 32-byte cap yields the smallest binary and all the caps yield similar performance.
Based on the results, we propose to change the cap value to 32-byte.

Patch by Zhaomo Yang!

Differential Revision: https://reviews.llvm.org/D49405

llvm-svn: 337622
2018-07-20 21:43:20 +00:00
H.J. Lu 0cb55919ec Mark REAL(swapcontext) with indirect_return attribute on x86
When shadow stack from Intel CET is enabled, the first instruction of all
indirect branch targets must be a special instruction, ENDBR.

lib/asan/asan_interceptors.cc has

...
  int res = REAL(swapcontext)(oucp, ucp);
...

REAL(swapcontext) is a function pointer to swapcontext in libc.  Since
swapcontext may return via indirect branch on x86 when shadow stack is
enabled, as in this case,

int res = REAL(swapcontext)(oucp, ucp);
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  This function may be
returned via an indirect branch.

Here compiler must insert ENDBR after call, like

call *bar(%rip)
endbr64

I opened an LLVM bug:

https://bugs.llvm.org/show_bug.cgi?id=38207

to add the indirect_return attribute so that it can be used to inform
compiler to insert ENDBR after REAL(swapcontext) call.  We mark
REAL(swapcontext) with the indirect_return attribute if it is available.

This fixed:

https://bugs.llvm.org/show_bug.cgi?id=38249

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D49608

llvm-svn: 337603
2018-07-20 19:24:11 +00:00
Dan Liew c358e51e9b On Darwin switch from the `VM_MEMORY_ANALYSIS_TOOL` VM tag to
`VM_MEMORY_SANITIZER`.

It turns out that `VM_MEMORY_ANALYSIS_TOOL` is already reserved for
use by other tools so switch to a tag reserved for use by the Sanitizers.

rdar://problem/41969783

Differential Revision: https://reviews.llvm.org/D49603

llvm-svn: 337579
2018-07-20 17:07:35 +00:00
Kostya Kortchinsky cccd21d42c [scudo] Simplify internal names (NFC)
Summary:
There is currently too much redundancy in the class/variable/* names in Scudo:
- we are in the namespace `__scudo`, so there is no point in having something
  named `ScudoX` to end up with a final name of `__scudo::ScudoX`;
- there are a lot of types/* that have `Allocator` in the name, given that
  Scudo is an allocator I figure this doubles up as well.

So change a bunch of the Scudo names to make them shorter, less redundant, and
overall simpler. They should still be pretty self explaining (or at least it
looks so to me).

The TSD part will be done in another CL (eg `__scudo::ScudoTSD`).

Reviewers: alekseyshl, eugenis

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49505

llvm-svn: 337557
2018-07-20 15:07:17 +00:00
Dmitry Vyukov 97cf5f7f40 esan: fix shadow setup
r337531 changed return type of MmapFixedNoReserve, but esan wasn't updated.
As the result esan shadow setup always fails.
We probably need to make MmapFixedNoAccess signature consistent
with MmapFixedNoReserve. But this is just to unbreak tests.
 

llvm-svn: 337550
2018-07-20 13:40:08 +00:00
David Carlier 12be7b7bf7 [Xray] fix c99 warning build about flexible array semantics
Reviewers: dberris

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D49590

llvm-svn: 337536
2018-07-20 09:22:22 +00:00
Dmitry Vyukov f52726aae9 sanitizers: consistently check result of MmapFixedNoReserve
MmapFixedNoReserve does not terminate process on failure.
Failure to check its result and die will always lead to harder
to debug crashes later in execution. This was observed in Go
processes due to some address space conflicts.

Consistently check result of MmapFixedNoReserve.
While we are here also add warn_unused_result attribute
to prevent such bugs in future and change return type to bool
as that's what all callers want.

Reviewed in https://reviews.llvm.org/D49367

llvm-svn: 337531
2018-07-20 08:33:41 +00:00
Joel E. Denny a47cb644c7 [libFuzzer] Use separate test directory for each config
Previously, check-all failed many tests for me.  It was running the
X86_64DefaultLinuxConfig, X86_64LibcxxLinuxConfig, and
X86_64StaticLibcxxLinuxConfig configs out of
llvm-build/projects/compiler-rt/test/fuzzer.  Now, it runs them out of
separate subdirectories there, and most tests pass.

Reviewed By: morehouse, george.karpenkov

Differential Revision: https://reviews.llvm.org/D49249

llvm-svn: 337521
2018-07-20 02:39:01 +00:00
Kostya Serebryany 44edc281d9 [libFuzzer] when -print_coverage=1 is given, print more stats (the number of seeds that hit every given function)
llvm-svn: 337501
2018-07-19 22:00:48 +00:00
Teresa Johnson 408f50dbb4 [profile] Fix typo in test
Noticed that this was causing a compiler warning in the test.

llvm-svn: 337493
2018-07-19 20:18:37 +00:00
Teresa Johnson 73053b221f [profile] Add interface to get profile filename
Summary:
Add __llvm_profile_get_filename interface to get the profile filename,
which can be used for identifying which profile file belongs to an app
when multiple binaries are instrumented and dumping profiles into the
same directory. The filename includes the path.

Reviewers: davidxl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49529

llvm-svn: 337482
2018-07-19 19:03:50 +00:00
Teresa Johnson d4143bc772 [compiler-rt] Only set lto_flags if lto_supported
Fix bot failure from r37465. Move the new lto_flags append under the
check for whether lto_supported. Otherwise TestingConfig may not have
that member.

llvm-svn: 337467
2018-07-19 16:12:15 +00:00
Teresa Johnson 293b0f43ac [compiler-rt] Add NewPM testing to CFI tests
Summary:
Executes both LTO and ThinLTO CFI tests an additional time using the new
pass manager. I only bothered to add with gold and not lld as testing
with one linker should be sufficient. I didn't add for APPLE or WIN32
since I don't have a way to test those.

Depends on D49429.

Reviewers: pcc

Subscribers: dberris, mgorny, mehdi_amini, delcypher, dexonsmith, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49432

llvm-svn: 337465
2018-07-19 15:32:48 +00:00
Dean Michael Berris 41cea19b4f [XRay][compiler-rt] Profiling: No files when empty
This change makes it so that the profiling mode implementation will only
write files when there are buffers to write. Before this change, we'd
always open a file even if there were no profiles collected when
flushing.

llvm-svn: 337443
2018-07-19 09:20:19 +00:00
Dean Michael Berris e0099f1c6d [XRay][compiler-rt] FDR Mode: Add extents metadata to buffer
When providing raw access to the FDR mode buffers, we used to not
include the extents metadata record. This oversight means that
processing the buffers in-memory will lose important information that
would have been written in files.

This change exposes the metadata record by serializing the data
similarly to how we would do it when flushing to files.

llvm-svn: 337441
2018-07-19 05:17:32 +00:00
David Carlier cfc1d1d46e [Xray] Fix allocator build, MAP_NORESERVE flag is not always supported
MAP_NORESERVE is not supported or a no-op on BSD.

Reviewers: dberris

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D49494

llvm-svn: 337440
2018-07-19 05:08:59 +00:00
Kostya Serebryany 7a6295f2e1 [libFuzzer] fix the bot (the no-assert build)
llvm-svn: 337437
2018-07-19 03:16:12 +00:00
Kostya Serebryany 79bd14772f [libFuzzer] quick hack to fix the bot
llvm-svn: 337436
2018-07-19 01:54:28 +00:00
Kostya Serebryany 6b87e0c18f [libFuzzer] first experimental attempt at DFT-based mutations (DFT=data-flow-trace)
llvm-svn: 337434
2018-07-19 01:23:32 +00:00
Matthew Voss ac1f9263d1 Remove scheduling dependency from XRay :: Posix/fork_basic_logging.cc
Summary:
We've been seeing intermittent failures on our internal bots and we suspect
this may be due to the OS scheduling the child process to run before the parent
process.

This version ensures that the parent and child can be run in either order.

Reviewers: Maknee, dberris

Reviewed By: dberris

Subscribers: delcypher, #sanitizers, Maknee, llvm-commits

Differential Revision: https://reviews.llvm.org/D49501

llvm-svn: 337432
2018-07-19 00:25:00 +00:00
Peter Collingbourne 4a653fa7f1 Rename __asan_gen_* symbols to ___asan_gen_*.
This prevents gold from printing a warning when trying to export
these symbols via the asan dynamic list after ThinLTO promotes them
from private symbols to external symbols with hidden visibility.

Differential Revision: https://reviews.llvm.org/D49498

llvm-svn: 337428
2018-07-18 22:23:14 +00:00
Petr Hosek 4915d3a1ec [Fuzzer] Improve crash unwinding on Fuchsia
Fuchsia doesn't have signals; instead it expects processes to have a
dedicated exception thread that binds to the process' exception port and
waits for exception packets to be delivered. On the other hand,
libFuzzer and sanitizer_common use expect to collect crash information
via libunwind from the same thread that caused the exception.

The long term fix is to improve support for remote unwinding in
libunbwind, plumb this through sanitizer_common and libFuzzer, and
handle the exception exclusively on the exception thread. In the
meantime, this revision has the exception thread "resurrect" the
crashing thread by:

* saving its general purpose register state onto the crashing thread's
  stack,
* setting the crashing thread's program counter to an assembly trampoline
  with the CFI information needed by libunwind, and
* resuming the crashed thread.

Patch By: aarongreen

Differential Revision: https://reviews.llvm.org/D48509

llvm-svn: 337418
2018-07-18 19:20:47 +00:00
Max Moroz 42b54e8117 [libFuzzer] Create single template for visiting Inline8bitCounters
Summary:
Created IterateInline8bitCounters, a single template for visiting  Inline8bitCounters (nested for loop)
Made InitializeUnstableCounters and UpdateUnstableCounters both send a lambda to IterateInline8bitCounters.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49453

llvm-svn: 337403
2018-07-18 17:03:27 +00:00
Dean Michael Berris 4719c52455 [XRay][compiler-rt] Segmented Array: Simplify and Optimise
Summary:
This is a follow-on to D49217 which simplifies and optimises the
implementation of the segmented array. In this patch we co-locate the
book-keeping for segments in the `__xray::Array<T>` with the data it's
managing. We take the chance in this patch to actually rename `Chunk` to
`Segment` to better align with the high-level description of the
segmented array.

With measurements using benchmarks landed in D48879, we've identified
that calls to `pthread_getspecific` started dominating the cycles, which
led us to revert the change made in D49217 to use C++ thread_local
initialisation instead (it reduces the cost by a huge margin, since we
save one PLT-based call to pthread functions in the hot path). In
particular, this is in `__xray::getThreadLocalData()`.

We also took the opportunity to remove the least-common-multiple based
calculation and instead pack as much data into segments of the array.
This greatly simplifies the API of the container which hides as much of
the implementation details as possible. For instance, we calculate the
number of elements we need for the each segment internally in the Array
instead of making it part of the type.

With the changes here, we're able to get a measurable improvement on the
performance of profiling mode on top of what D48879 already provides.

Depends on D48879.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49363

llvm-svn: 337343
2018-07-18 02:08:39 +00:00
Dean Michael Berris 9d6b7a5f2b [XRay][compiler-rt] Simplify Allocator Implementation
Summary:
This change simplifies the XRay Allocator implementation to self-manage
an mmap'ed memory segment instead of using the internal allocator
implementation in sanitizer_common.

We've found through benchmarks and profiling these benchmarks in D48879
that using the internal allocator in sanitizer_common introduces a
bottleneck on allocating memory through a central spinlock. This change
allows thread-local allocators to eliminate contention on the
centralized allocator.

To get the most benefit from this approach, we also use a managed
allocator for the chunk elements used by the segmented array
implementation. This gives us the chance to amortize the cost of
allocating memory when creating these internal segmented array data
structures.

We also took the opportunity to remove the preallocation argument from
the allocator API, simplifying the usage of the allocator throughout the
profiling implementation.

In this change we also tweak some of the flag values to reduce the
amount of maximum memory we use/need for each thread, when requesting
memory through mmap.

Depends on D48956.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49217

llvm-svn: 337342
2018-07-18 01:53:39 +00:00
Dean Michael Berris 1e3feb49e3 [XRay][compiler-rt] FDR Mode: Allow multiple runs
Summary:
Fix a bug in FDR mode which didn't allow for re-initialising the logging
in the same process. This change ensures that:

- When we flush the FDR mode logging, that the state of the logging
  implementation is `XRAY_LOG_UNINITIALIZED`.

- Fix up the thread-local initialisation to use aligned storage and
  `pthread_getspecific` as well as `pthread_setspecific` for the
  thread-specific data.

- Actually use the pointer provided to the thread-exit cleanup handling,
  instead of assuming that the thread has thread-local data associated
  with it, and reaching at thread-exit time.

In this change we also have an explicit test for two consecutive
sessions for FDR mode tracing, and ensuring both sessions succeed.

Reviewers: kpw, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D49359

llvm-svn: 337341
2018-07-18 01:31:30 +00:00
Max Moroz 061b4af998 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337324
2018-07-17 20:37:40 +00:00
Martin Storsjo 946a81f5b0 [builtins] Implement the __chkstk function for ARM for MinGW
This function is available for linking in from kernel32.dll, but
it's not allowed to link that function from there in Windows Store
apps.

Differential Revision: https://reviews.llvm.org/D49055

llvm-svn: 337313
2018-07-17 19:14:47 +00:00
Matt Morehouse 43a2296976 libFuzzer: prevent irrelevant strings from leaking into auto-dictionary
This is a fix for bug 37047.

https://bugs.llvm.org/show_bug.cgi?id=37047

Implemented by basically reversing the logic. Previously all strings
were considered, with some operations excluded. Now strings are excluded
by default, and only strings during the CB considered.

Patch By: pdknsk

Differential Revision: https://reviews.llvm.org/D48800

llvm-svn: 337296
2018-07-17 16:12:00 +00:00
Matt Morehouse ef521ffe24 [libFuzzer] Avoid STL in MSan test.
Summary:
STL can cause MSan false positives if lib[std]c++ isn't instrumented
with MSan.

Reviewers: kcc

Reviewed By: kcc

Subscribers: Dor1s, llvm-commits

Differential Revision: https://reviews.llvm.org/D49404

llvm-svn: 337224
2018-07-16 23:22:54 +00:00
Max Moroz 5697c59c7f Revert r337194 (https://reviews.llvm.org/D48891) due to compilation errors.
llvm-svn: 337206
2018-07-16 20:05:18 +00:00
Alex Lorenz 88feedd7c7 [cmake][libFuzzer] fixup r337193 to ensure msan/dfsan are not added
to test deps for libfuzzer when they're not supported by the platform

llvm-svn: 337203
2018-07-16 19:41:49 +00:00
Petr Hosek b05be5f4b4 [CMake] Use cxx-headers as a depedency for C++ headers
We no longer pass CLANG_DEFAULT_CXX_STDLIB to the runtimes build
as it was causing issues so we can no longer use this variable. We
instead use cxx-headers as a dependency whenever this is available
since both XRay and libFuzzer are built as static libraries so this
is sufficient.

Differential Revision: https://reviews.llvm.org/D49346

llvm-svn: 337199
2018-07-16 18:48:20 +00:00
Max Moroz 8a5083df53 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337194
2018-07-16 17:50:46 +00:00
Matt Morehouse 0876a889f7 [libFuzzer] Add msan and dfsan to test deps.
Required now that we have tests using MSan and DFSan.

llvm-svn: 337193
2018-07-16 17:37:15 +00:00
Max Moroz 08dad54924 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337187
2018-07-16 16:01:31 +00:00
Max Moroz 1d369a5d01 Revert r337175 (https://reviews.llvm.org/D49212) due to unintentional format changes.
llvm-svn: 337180
2018-07-16 15:15:34 +00:00
Chandler Carruth 4abbd13ab2 Fix warnings in a static assert added to CFI.
llvm-svn: 337178
2018-07-16 15:01:26 +00:00
Henry Zhu ebc68ff65f Test commit
llvm-svn: 337176
2018-07-16 14:54:29 +00:00
Max Moroz 2156d885e0 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337175
2018-07-16 14:54:23 +00:00
Marco Castelluccio 1f0b194b4a [gcov] Add a test showing differences in line counts when building with or without exceptions enabled.
Test for https://bugs.llvm.org/show_bug.cgi?id=38066.

llvm-svn: 337174
2018-07-16 14:40:33 +00:00
Filipe Cabecinhas 3bc3fb477a [cfi] Don't pass a uint16_t to memset. Make sure the 16-bit constant is appropriate for us.
Reviewers: eugenis, pcc, kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49252

llvm-svn: 337170
2018-07-16 13:41:54 +00:00
Marco Castelluccio 0a75de4bfe Add a test with __gcov_flush called before terminating the program.
Test for https://bugs.llvm.org/show_bug.cgi?id=38067.

llvm-svn: 337150
2018-07-16 09:13:46 +00:00
Dean Michael Berris 734e3928b3 [XRay][compiler-rt] Use `SANITIZER_CXX_ABI_LIBRARY` for XRay unit tests
Summary:
Fix a TODO in CMake config for XRay tests to use the detected C++ ABI
library in the tests.

Also make the tests depend on the llvm-xray target when built in-tree.

Reviewers: kpw, eizan

Reviewed By: eizan

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D49358

llvm-svn: 337142
2018-07-16 01:08:56 +00:00
Petr Hosek d533fae6ef [CMake] Change the flag to use compiler-rt builtins to boolean
This changes the name and the type to what it was prior to r333037
which matches the name of the flag used in other runtimes: libc++,
libc++abi and libunwind. We don't need the type to be a string since
there's only binary choice between libgcc and compiler-rt unlike in
the case of C++ library where there're multiple options.

Differential Revision: https://reviews.llvm.org/D49325

llvm-svn: 337116
2018-07-15 03:05:20 +00:00
Max Moroz 038771a25a [UBSan] Followup for silence_unsigned_overflow flag to handle negate overflows.
Summary:
That flag has been introduced in https://reviews.llvm.org/D48660 for
suppressing UIO error messages in an efficient way. The main motivation is to
be able to use UIO checks in builds used for fuzzing as it might provide an
interesting signal to a fuzzing engine such as libFuzzer.

See https://github.com/google/oss-fuzz/issues/910 for more information.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49324

llvm-svn: 337068
2018-07-13 22:49:06 +00:00
Vlad Tsyrklevich ddf998b608 Fix for Darwin buildbot failure due to r337037
Duplicate __get_unsafe_stack_bottom instead of using an alias for
platforms that don't suppport it like Darwin.

llvm-svn: 337044
2018-07-13 20:56:48 +00:00
Vlad Tsyrklevich be9a9fd3dd SafeStack: Add builtins to read unsafe stack top/bottom
Summary:
Introduce built-ins to read the unsafe stack top and bottom. The unsafe
stack top is required to implement garbage collection scanning for
Oilpan. Currently there is already a built-in 'get_unsafe_stack_start'
to read the bottom of the unsafe stack, but I chose to duplicate this
API because 'start' is ambiguous (e.g. Oilpan uses WTF::GetStackStart to
read the safe stack top.)

Reviewers: pcc

Reviewed By: pcc

Subscribers: llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49152

llvm-svn: 337037
2018-07-13 19:48:35 +00:00
Petr Hosek 2d46cfedf2 [CMake] Don't use CLANG_DEFAULT_* values
These are not necessarily correct, just because Clang is configured
to use libc++ or compiler-rt as defaults doesn't mean that the host
compiler uses these as defaults as well. A more correct solution
would be to test the host compiler to check what the default actually
is as attempted in D46857.

Part of PR-38025.

Differential Revision: https://reviews.llvm.org/D49272

llvm-svn: 337033
2018-07-13 19:16:30 +00:00
Kostya Kortchinsky 4a547fb62d [sanitizer] Use -Wl,-z,global on Android
Summary:
Use `-Wl,-z,global` for all Sanitizer shared libraries on
Android. We want them to be in the global group
(https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md#changes-to-library-search-order)
to avoid any alloc/dealloc mismatch between the libc allocator & said library.

`audioserver` was one of the binary that exhibited the problem with Scudo,
this seems to fix it.

[edited for accuracy]

Reviewers: eugenis, alekseyshl

Reviewed By: eugenis

Subscribers: kubamracek, srhines, mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49198

llvm-svn: 337010
2018-07-13 15:31:05 +00:00
Marco Castelluccio d47bde0d7a Add a test with multiple BBs on the same line
Test for https://bugs.llvm.org/show_bug.cgi?id=38065.

llvm-svn: 336996
2018-07-13 13:01:43 +00:00
Dan Liew 95d9d22e00 Remove `tsan/Darwin/gcd-after-null.mm` test.
Summary:
This test invokes undocumented behaviour that could change in
the future. Given this, it's probably best to just remove the
test.

rdar://problem/42022283

Reviewers: kubamracek

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49269

llvm-svn: 336977
2018-07-13 07:37:01 +00:00
Dean Michael Berris 10141261e1 [XRay][compiler-rt] Add PID field to llvm-xray tool and add PID metadata record entry in FDR mode
Summary:
llvm-xray changes:
- account-mode - process-id  {...} shows after thread-id
- convert-mode - process {...} shows after thread
- parses FDR and basic mode pid entries
- Checks version number for FDR log parsing.

Basic logging changes:
- Update header version from 2 -> 3

FDR logging changes:
- Update header version from 2 -> 3
- in writeBufferPreamble, there is an additional PID Metadata record (after thread id record and tsc record)

Test cases changes:
- fdr-mode.cc, fdr-single-thread.cc, fdr-thread-order.cc modified to catch process id output in the log.

Reviewers: dberris

Reviewed By: dberris

Subscribers: hiraditya, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49153

llvm-svn: 336974
2018-07-13 05:38:22 +00:00
Dean Michael Berris 5d92d3e5be [XRay][compiler-rt] Profiling Mode: Flush logs on exit
Summary:
This change adds support for writing out profiles at program exit.

Depends on D48653.

Reviewers: kpw, eizan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D48956

llvm-svn: 336969
2018-07-13 04:04:18 +00:00
George Karpenkov 01c762b9fd [fuzzer] [tests] Increase the number of iterations for three-bytes.test
The test is flaky otherwise on some of our macOS machines in the test fleet.

Differential Revision: https://reviews.llvm.org/D49277

llvm-svn: 336966
2018-07-13 01:21:50 +00:00
Dan Liew 0de57a676c [CMake] Fix a typo in the variable used to retrieve source file names
for the `RTHwasan_dynamic` target.

Differential Revision: https://reviews.llvm.org/D49178

llvm-svn: 336944
2018-07-12 20:55:02 +00:00
Marco Castelluccio 2bb551c6bf Simplify instrprof-dlopen-dlclose-gcov.test to avoid failures on Aarch64.
The test for a function with an if block in a single line (https://bugs.llvm.org/show_bug.cgi?id=38065) will be moved to a separate test.

llvm-svn: 336942
2018-07-12 20:28:09 +00:00
Matt Morehouse 4543816150 [SanitizerCoverage] Add associated metadata to 8-bit counters.
Summary:
This allows counters associated with unused functions to be
dead-stripped along with their functions.  This approach is the same one
we used for PC tables.

Fixes an issue where LLD removes an unused PC table but leaves the 8-bit
counter.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, hiraditya, kcc

Differential Revision: https://reviews.llvm.org/D49264

llvm-svn: 336941
2018-07-12 20:24:58 +00:00
Matt Morehouse 24d617cbee [libFuzzer] If LLD available, require it to build first.
Since we now have a test that requires LLD, make sure it is built before
that test runs.

llvm-svn: 336932
2018-07-12 18:52:10 +00:00
Matt Morehouse d507c96ff1 [libFuzzer] Use lld-available for gc-sections.test.
The lld feature is never available for libFuzzer tests, so
gc-sections.test never actually runs.

llvm-svn: 336926
2018-07-12 18:09:03 +00:00
Dan Liew 4625480acb [CMake] Remove unnecesary list of source files for Xray unit tests.
The list duplicates information already available in the parent
directory so use that instead. It is unclear to me why we need
to spell out the dependencies explicitly but fixing that should
be done in a separate patch.

Differential Revision: https://reviews.llvm.org/D49177

llvm-svn: 336905
2018-07-12 13:48:18 +00:00
Dan Liew c5d759a926 [CMake] Rename `SANITIZER_HEADERS` to `SANITIZER_IMPL_HEADERS` under `lib/sanitizer_common`.
The variable name `SANITIZER_HEADERS` is already used for the list of
public headers in `include/CMakeLists.txt`.  Although the previous
implementation worked it's probably best to avoid shadowing global
variables to avoid confusion.

Differential Revision: https://reviews.llvm.org/D49176

llvm-svn: 336904
2018-07-12 13:36:44 +00:00
Dean Michael Berris 4366b0122a [XRay][compiler-rt] Fixup: require x86_64 for profiling mode tests
This constrains the build environments we are testing/supporting for the runtime
tests until we can be sure xray works in more platforms.

llvm-svn: 336878
2018-07-12 01:54:29 +00:00
Joel E. Denny a09b9317f5 [FileCheck] Add -allow-deprecated-dag-overlap to another compiler-rt test
See https://reviews.llvm.org/D47106 for details.

llvm-svn: 336859
2018-07-11 22:07:31 +00:00
Joel E. Denny da660d6a70 [FileCheck] Add -allow-deprecated-dag-overlap to failing compiler-rt tests
See https://reviews.llvm.org/D47106 for details.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D47326

llvm-svn: 336845
2018-07-11 20:26:44 +00:00
Zaara Syeda f3fd38effd Remove ppc64 BE XFAILs now that gcov profiling works, after starting a clean
build this time.

llvm-svn: 336839
2018-07-11 19:17:43 +00:00
Marco Castelluccio e125955890 Link to the correct bug number about the Mac failure for instrprof-shared-gcov-flush.test.
llvm-svn: 336820
2018-07-11 15:44:15 +00:00
Zaara Syeda 47c2ddf6b3 Revert 336811, there are still some problems with the tests.
llvm-svn: 336819
2018-07-11 15:37:19 +00:00
Zaara Syeda 495adf52db Remove ppc64 BE XFAILs now that gcov profiling works.
llvm-svn: 336811
2018-07-11 14:55:19 +00:00
Marco Castelluccio e21cfa7e7f Fix reading 32 bit gcov tag values on little-endian machines
Summary:
The write buffer contains signed chars, which means the shift operations caused values such as the arc tag value (0x01a10000) to be read incorrectly (0xffa10000).

This fixes a regression from https://reviews.llvm.org/D49132.

Reviewers: uweigand, davidxl

Reviewed By: uweigand

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49161

llvm-svn: 336775
2018-07-11 09:30:25 +00:00
Dean Michael Berris 8299e4b8db [XRay] basic mode PID and TID always fetch
Summary: XRayRecords now includes a PID field. Basic handlers fetch pid and tid each time they are called instead of caching the value. Added a testcase that calls fork and checks if the child TID is different from the parent TID to verify that the processes' TID are different in the trace.

Reviewers: dberris, Maknee

Reviewed By: dberris, Maknee

Subscribers: kpw, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49025

llvm-svn: 336769
2018-07-11 07:14:27 +00:00
Stephen Hines 6812c28ef3 Add libcxxabi option back for sanitizer use.
Summary:
A prior refactoring accidentally dropped the case for using libc++abi as
the out-of-tree C++ runtime library for sanitizers. This patch restores
that functionality, which is used by Android, which can't depend on the
full libc++ for these libraries.

Reviewers: phosek, EricWF

Reviewed By: phosek

Subscribers: meikeb, kongyi, chh, mgorny, delcypher, llvm-commits, #sanitizers, pirama

Differential Revision: https://reviews.llvm.org/D49157

llvm-svn: 336749
2018-07-11 00:50:03 +00:00
Matt Morehouse 79d55d30c3 [libFuzzer] Disable dataflow.test on AArch64.
Summary:
After my recent change to allow MSan + libFuzzer, the
ExplodeDFSanLabelsTest.cpp test started to overflow the stack with
recursive function SetBytesForLabel() on an AArch64 bot.  Perhaps that
bot has a smaller stack size, or maybe AArch64 has larger stack frames
for this particular function.

Reviewers: kcc, javed.absar

Reviewed By: kcc

Subscribers: kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D49150

llvm-svn: 336725
2018-07-10 20:45:36 +00:00
Matt Morehouse 0e904e8806 Revert "[Fuzzer] Afl driver changing iterations handling"
This reverts rL334510 due to breakage of afl_driver's command line
interface.

Patch By: Jonathan Metzman

Differential Revision: https://reviews.llvm.org/D49141

llvm-svn: 336719
2018-07-10 19:58:42 +00:00
Vlad Tsyrklevich f90ad5b327 Limit ASan non-executable-pc test case to x86 bots
The test case fails on the big-endian PPC bot, probably because PowerPC
uses function descriptors. More over other architectures don't support
NX mappings. (This test case was not being exercised prior to r336633.)

llvm-svn: 336714
2018-07-10 19:34:46 +00:00