Commit Graph

8326 Commits

Author SHA1 Message Date
Max Moroz 8a5083df53 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337194
2018-07-16 17:50:46 +00:00
Max Moroz 08dad54924 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337187
2018-07-16 16:01:31 +00:00
Max Moroz 1d369a5d01 Revert r337175 (https://reviews.llvm.org/D49212) due to unintentional format changes.
llvm-svn: 337180
2018-07-16 15:15:34 +00:00
Chandler Carruth 4abbd13ab2 Fix warnings in a static assert added to CFI.
llvm-svn: 337178
2018-07-16 15:01:26 +00:00
Henry Zhu ebc68ff65f Test commit
llvm-svn: 337176
2018-07-16 14:54:29 +00:00
Max Moroz 2156d885e0 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337175
2018-07-16 14:54:23 +00:00
Filipe Cabecinhas 3bc3fb477a [cfi] Don't pass a uint16_t to memset. Make sure the 16-bit constant is appropriate for us.
Reviewers: eugenis, pcc, kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49252

llvm-svn: 337170
2018-07-16 13:41:54 +00:00
Dean Michael Berris 734e3928b3 [XRay][compiler-rt] Use `SANITIZER_CXX_ABI_LIBRARY` for XRay unit tests
Summary:
Fix a TODO in CMake config for XRay tests to use the detected C++ ABI
library in the tests.

Also make the tests depend on the llvm-xray target when built in-tree.

Reviewers: kpw, eizan

Reviewed By: eizan

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D49358

llvm-svn: 337142
2018-07-16 01:08:56 +00:00
Max Moroz 038771a25a [UBSan] Followup for silence_unsigned_overflow flag to handle negate overflows.
Summary:
That flag has been introduced in https://reviews.llvm.org/D48660 for
suppressing UIO error messages in an efficient way. The main motivation is to
be able to use UIO checks in builds used for fuzzing as it might provide an
interesting signal to a fuzzing engine such as libFuzzer.

See https://github.com/google/oss-fuzz/issues/910 for more information.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49324

llvm-svn: 337068
2018-07-13 22:49:06 +00:00
Vlad Tsyrklevich ddf998b608 Fix for Darwin buildbot failure due to r337037
Duplicate __get_unsafe_stack_bottom instead of using an alias for
platforms that don't suppport it like Darwin.

llvm-svn: 337044
2018-07-13 20:56:48 +00:00
Vlad Tsyrklevich be9a9fd3dd SafeStack: Add builtins to read unsafe stack top/bottom
Summary:
Introduce built-ins to read the unsafe stack top and bottom. The unsafe
stack top is required to implement garbage collection scanning for
Oilpan. Currently there is already a built-in 'get_unsafe_stack_start'
to read the bottom of the unsafe stack, but I chose to duplicate this
API because 'start' is ambiguous (e.g. Oilpan uses WTF::GetStackStart to
read the safe stack top.)

Reviewers: pcc

Reviewed By: pcc

Subscribers: llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49152

llvm-svn: 337037
2018-07-13 19:48:35 +00:00
Kostya Kortchinsky 4a547fb62d [sanitizer] Use -Wl,-z,global on Android
Summary:
Use `-Wl,-z,global` for all Sanitizer shared libraries on
Android. We want them to be in the global group
(https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md#changes-to-library-search-order)
to avoid any alloc/dealloc mismatch between the libc allocator & said library.

`audioserver` was one of the binary that exhibited the problem with Scudo,
this seems to fix it.

[edited for accuracy]

Reviewers: eugenis, alekseyshl

Reviewed By: eugenis

Subscribers: kubamracek, srhines, mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49198

llvm-svn: 337010
2018-07-13 15:31:05 +00:00
Dean Michael Berris 10141261e1 [XRay][compiler-rt] Add PID field to llvm-xray tool and add PID metadata record entry in FDR mode
Summary:
llvm-xray changes:
- account-mode - process-id  {...} shows after thread-id
- convert-mode - process {...} shows after thread
- parses FDR and basic mode pid entries
- Checks version number for FDR log parsing.

Basic logging changes:
- Update header version from 2 -> 3

FDR logging changes:
- Update header version from 2 -> 3
- in writeBufferPreamble, there is an additional PID Metadata record (after thread id record and tsc record)

Test cases changes:
- fdr-mode.cc, fdr-single-thread.cc, fdr-thread-order.cc modified to catch process id output in the log.

Reviewers: dberris

Reviewed By: dberris

Subscribers: hiraditya, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49153

llvm-svn: 336974
2018-07-13 05:38:22 +00:00
Dean Michael Berris 5d92d3e5be [XRay][compiler-rt] Profiling Mode: Flush logs on exit
Summary:
This change adds support for writing out profiles at program exit.

Depends on D48653.

Reviewers: kpw, eizan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D48956

llvm-svn: 336969
2018-07-13 04:04:18 +00:00
Dan Liew 0de57a676c [CMake] Fix a typo in the variable used to retrieve source file names
for the `RTHwasan_dynamic` target.

Differential Revision: https://reviews.llvm.org/D49178

llvm-svn: 336944
2018-07-12 20:55:02 +00:00
Dan Liew 4625480acb [CMake] Remove unnecesary list of source files for Xray unit tests.
The list duplicates information already available in the parent
directory so use that instead. It is unclear to me why we need
to spell out the dependencies explicitly but fixing that should
be done in a separate patch.

Differential Revision: https://reviews.llvm.org/D49177

llvm-svn: 336905
2018-07-12 13:48:18 +00:00
Dan Liew c5d759a926 [CMake] Rename `SANITIZER_HEADERS` to `SANITIZER_IMPL_HEADERS` under `lib/sanitizer_common`.
The variable name `SANITIZER_HEADERS` is already used for the list of
public headers in `include/CMakeLists.txt`.  Although the previous
implementation worked it's probably best to avoid shadowing global
variables to avoid confusion.

Differential Revision: https://reviews.llvm.org/D49176

llvm-svn: 336904
2018-07-12 13:36:44 +00:00
Marco Castelluccio e21cfa7e7f Fix reading 32 bit gcov tag values on little-endian machines
Summary:
The write buffer contains signed chars, which means the shift operations caused values such as the arc tag value (0x01a10000) to be read incorrectly (0xffa10000).

This fixes a regression from https://reviews.llvm.org/D49132.

Reviewers: uweigand, davidxl

Reviewed By: uweigand

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49161

llvm-svn: 336775
2018-07-11 09:30:25 +00:00
Dean Michael Berris 8299e4b8db [XRay] basic mode PID and TID always fetch
Summary: XRayRecords now includes a PID field. Basic handlers fetch pid and tid each time they are called instead of caching the value. Added a testcase that calls fork and checks if the child TID is different from the parent TID to verify that the processes' TID are different in the trace.

Reviewers: dberris, Maknee

Reviewed By: dberris, Maknee

Subscribers: kpw, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49025

llvm-svn: 336769
2018-07-11 07:14:27 +00:00
Matt Morehouse 0e904e8806 Revert "[Fuzzer] Afl driver changing iterations handling"
This reverts rL334510 due to breakage of afl_driver's command line
interface.

Patch By: Jonathan Metzman

Differential Revision: https://reviews.llvm.org/D49141

llvm-svn: 336719
2018-07-10 19:58:42 +00:00
Ulrich Weigand dd4d2ac607 [gcov] Fix fallout from r336693
Fix building GCDAProfiling.c with pre-C99 compilers.
This caused a build bot failure.

llvm-svn: 336706
2018-07-10 17:08:05 +00:00
Dan Liew 9f6302e950 Try to fix broken build due to r336663.
It turns out that the `${XRAY_HEADERS}` CMake variable was already
in use and was used for public headers. It seems that
`lib/xray/tests/CMakeLists.txt` was depending on this.

To fix rename the new `${XRAY_HEADERS}` to `${XRAY_IMPL_HEADERS}`.

llvm-svn: 336699
2018-07-10 16:22:05 +00:00
Ulrich Weigand e508cd2a04 [gcov] Fix gcov profiling on big-endian machines
Two fixes required to handle big-endian systems:

- 64-bit counter values are stored in a mixed-endian format in the
gcov files: a 32-bit low-part followed by a 32-bit high part. Note that
this is already implemented correctly on the LLVM side, see
GCOVBuffer::readInt64.

- The tag values (e.g. arcs tag, object summary tag, ...) are aways
written as the same sequence of bytes independent of byte order. But
when *reading* them back in, the code reads them as 32-bit values in
host byte order. For the comparisons to work correctly, this should
instead always read them as little-endian values.

Fixes PR 38121.

Reviewed By: marco-c

Differential Revision: https://reviews.llvm.org/D49132

llvm-svn: 336693
2018-07-10 16:08:27 +00:00
Marco Castelluccio 2827420aab Reapply "Make __gcov_flush flush counters for all shared libraries"
This reapplies r336365, after marking tests as failing on various
configurations.

llvm-svn: 336678
2018-07-10 14:12:03 +00:00
Dan Liew b1f95697c1 [CMake] Add compiler-rt header files to the list of sources for targets
when building with an IDE so that header files show up in the UI.
This massively improves the development workflow in IDEs.

To implement this a new function `compiler_rt_process_sources(...)` has
been added that adds header files to the list of sources when the
generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no
changes are made to the list of source files.

The function can be passed a list of headers via the
`ADDITIONAL_HEADERS` argument. For each runtime library a list of
explicit header files has been added and passed via
`ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of
headers was already present but it was stale and has been updated
to reflect the current state of the source tree.

The original version of this patch used file globbing (`*.{h,inc,def}`)
to find the headers but the approach was changed due to this being a
CMake anti-pattern (if the list of headers changes CMake won't
automatically re-generate if globbing is used).

The LLVM repo contains a similar function named `llvm_process_sources()`
but we don't use it here for several reasons:

* It depends on the `LLVM_ENABLE_OPTION` cache variable which is
  not set in standalone compiler-rt builds.
* We would have to `include(LLVMProcessSources)` which I'd like to
  avoid because it would include a bunch of stuff we don't need.

Differential Revision: https://reviews.llvm.org/D48422

llvm-svn: 336663
2018-07-10 13:00:17 +00:00
Dean Michael Berris b251f62428 [XRay][compiler-rt] Fixup build breakage
Changes:

- Remove static assertion on size of a structure, fails on systems where
  pointers aren't 8 bytes.

- Use size_t instead of deducing type of arguments to
  `nearest_boundary`.

Follow-up to D48653.

llvm-svn: 336648
2018-07-10 08:58:12 +00:00
Dean Michael Berris 0dd4f9f22f [XRay][compiler-rt] xray::Array Freelist and Iterator Updates
Summary:
We found a bug while working on a benchmark for the profiling mode which
manifests as a segmentation fault in the profiling handler's
implementation. This change adds unit tests which replicate the
issues in isolation.

We've tracked this down as a bug in the implementation of the Freelist
in the `xray::Array` type. This happens when we trim the array by a
number of elements, where we've been incorrectly assigning pointers for
the links in the freelist of chunk nodes. We've taken the chance to add
more debug-only assertions to the code path and allow us to verify these
assumptions in debug builds.

In the process, we also took the opportunity to use iterators to
implement both `front()` and `back()` which exposes a bug in the
iterator decrement operation.  In particular, when we decrement past a
chunk size boundary, we end up moving too far back and reaching the
`SentinelChunk` prematurely.

This change unblocks us to allow for contributing the non-crashing
version of the benchmarks in the test-suite as well.

Reviewers: kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D48653

llvm-svn: 336644
2018-07-10 08:25:44 +00:00
Matt Morehouse a34c65e845 [libFuzzer] Make -fsanitize=memory,fuzzer work.
This patch allows libFuzzer to fuzz applications instrumented with MSan
without recompiling libFuzzer with MSan instrumentation.

Fixes https://github.com/google/sanitizers/issues/958.

Differential Revision: https://reviews.llvm.org/D48891

llvm-svn: 336619
2018-07-09 23:51:08 +00:00
Matt Morehouse a5bb6d53f2 Revert "[libFuzzer] Mutation tracking and logging implemented"
This reverts r336597 due to bot breakage.

llvm-svn: 336616
2018-07-09 22:31:26 +00:00
Matt Morehouse d153d46884 [libFuzzer] Mutation tracking and logging implemented
Code now exists to track number of mutations that are used in fuzzing in
total and ones that produce new coverage. The stats are currently being
dumped to the command line.

Patch By: Kode Williams

Differntial Revision: https://reviews.llvm.org/D48054

llvm-svn: 336597
2018-07-09 20:17:52 +00:00
Alex Shlyapnikov 63af91574f [ASan] Minor ASan error reporting cleanup
Summary:
- use proper Error() decorator for error messages
- refactor ASan thread id and name reporting

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49044

llvm-svn: 336573
2018-07-09 17:54:55 +00:00
Michael Zolotukhin 9664ca9dce Revert "Make __gcov_flush flush counters for all shared libraries"
This reverts r336365: the added tests are failing on various
configurations (e.g. on green-dragon).

llvm-svn: 336474
2018-07-07 00:07:00 +00:00
Kostya Serebryany 2bd02db943 libFuzzer: always print line-break for NEW_FUNC/PC output
Summary: This is a minor cosmetic change. When function/path exceed ~1000 characters, the output is truncated before the line-break. I noticed this for NEW_FUNC.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48799

llvm-svn: 336461
2018-07-06 19:47:00 +00:00
Matt Morehouse 373d6ed7cf [MSan] Add functions to enable/disable interceptor checks.
Summary:
The motivation for this change is to make libFuzzer+MSan possible
without instrumenting libFuzzer.

See https://github.com/google/sanitizers/issues/958.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D48890

llvm-svn: 336447
2018-07-06 17:10:51 +00:00
Kostya Kortchinsky 2efb847b6f [scudo] Add some logs for Android
Summary:
Namely, set the abort message, and allow to write the message to syslog if the
option is enabled.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48902

llvm-svn: 336445
2018-07-06 16:50:12 +00:00
Marco Castelluccio 32d18beb8c Make __gcov_flush flush counters for all shared libraries
Summary:
This will make the behavior of __gcov_flush match the GCC behavior.

I would like to rename __gcov_flush to __llvm_gcov_flush (in case of programs linking to libraries built with different compilers), but I guess we can't for compatibility reasons.

Reviewers: davidxl

Reviewed By: davidxl

Subscribers: samsonov, vitalybuka, pcc, kcc, junbuml, glider, fhahn, eugenis, dvyukov, davidxl, srhines, chh, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48538

llvm-svn: 336365
2018-07-05 15:52:59 +00:00
George Karpenkov aeeac6d41c [libFuzzer] [NFC] Inline static variable to avoid the linker warning.
Differential Revision: https://reviews.llvm.org/D48650

llvm-svn: 336238
2018-07-04 00:37:45 +00:00
Kostya Serebryany 51ddb88300 [libFuzzer] add one more value profile metric, under a flag (experimental)
llvm-svn: 336234
2018-07-03 22:33:09 +00:00
Kostya Serebryany dcac0a3b5e [libFuzzer] remove stale code, as suggested in https://reviews.llvm.org/D48800
llvm-svn: 336230
2018-07-03 21:22:44 +00:00
Kostya Kortchinsky e64a81475c [scudo] Get rid of builtin-declaration-mismatch warnings
Summary:
The C interceptors were using `SIZE_T` defined in the interception library as
a `__sanitizer::uptr`. On some 32-bit platforms, this lead to the following
warning:
```
warning: declaration of ‘void* malloc(SIZE_T)’ conflicts with built-in declaration ‘void* malloc(unsigned int)’ [-Wbuiltin-declaration-mismatch]
INTERCEPTOR_ATTRIBUTE void *malloc(SIZE_T size) {
```
`__sanitizer::uptr` is indeed defined as an `unsigned long` on those.

So just include `stddef.h` and use `size_t` instead.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48885

llvm-svn: 336221
2018-07-03 19:03:46 +00:00
Kostya Kortchinsky 4bc7f3d4d6 [scudo] Enable Scudo memory hooks for Fuchsia.
Summary:
    It would be useful for Flutter apps, especially, to be able to use
    malloc hooks to debug memory leaks on Fuchsia. They're not able to do
    this right now, so it'd be a nice bonus to throw in with the Scudo
    switchover.


Reviewers: cryptoad, alekseyshl

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D48618

llvm-svn: 336139
2018-07-02 19:48:01 +00:00
Fangrui Song 5f27e0c021 [asan] Fix deadlock issue on FreeBSD, caused by use of .preinit_array in rL325240
Summary:
Without this patch,
clang -fsanitize=address -xc =(printf 'int main(){}') -o a; ./a => deadlock in __asan_init>AsanInitInternal>AsanTSDInit>...>__getcontextx_size>_rtld_bind>rlock_acquire(rtld_bind_lock, &lockstate)

libexec/rtld-elf/rtld.c
  wlock_acquire(rtld_bind_lock, &lockstate);
  if (obj_main->crt_no_init)
    preinit_main(); // unresolved PLT functions cannot be called here

lib/libthr/thread/thr_rtld.c
  uc_len = __getcontextx_size(); // unresolved PLT function in libthr.so.3

check-xray tests currently rely on .preinit_array so we special case in
xray_init.cc

Subscribers: srhines, kubamracek, krytarowski, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48806

llvm-svn: 336067
2018-07-01 17:52:41 +00:00
Fangrui Song da891dc6b4 [asan] Use MADV_NOCORE for use_madv_dontdump on FreeBSD.
Currently in FreeBSD 12.0-CURRENT with trunk clang+compiler-rt, faulty -fsanitize=address executable hangs at 'urdlck' state.

Ka Ho Ng has verified that by backporting this to llvm 6.0.1, with use_madv_dontdump=1, shadow memory is not dumped.

ASAN_OPTIONS=abort_on_error=1:disable_coredump=0:use_madv_dontdump=1 ./a

Reviewers: dimitry, kcc, dvyukov, emaste, khng300

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48257

llvm-svn: 336046
2018-06-30 08:27:48 +00:00
Chih-Hung Hsieh 7222e8e30b [profile] Add llvm_gcov_flush to be called outside a shared library
__gcov_flush is hidden.
For applications to dump profiling data of selected .so files,
they can use dlsym to find and call llvm_gcov_flush in each .so file.

Differential Revision: https://reviews.llvm.org/D45454

llvm-svn: 336019
2018-06-29 21:45:55 +00:00
Alex Shlyapnikov fb1644835b [TSan] More detailed error message on failed sahdow memory madvise
Summary:
Report errno value on failed shadow memory madvise attempt and add a
hint message with the possible workaround.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48668

llvm-svn: 335928
2018-06-28 21:38:34 +00:00
Petr Hosek 61574fdb06 [sanitizer] zx_cprng_draw no longer returns any value
Remove the return value check.

Differential Revision: https://reviews.llvm.org/D48671

llvm-svn: 335790
2018-06-27 21:25:21 +00:00
Matt Morehouse 520748f01e [UBSan] Add silence_unsigned_overflow flag.
Summary:
Setting UBSAN_OPTIONS=silence_unsigned_overflow=1 will silence all UIO
reports.  This feature, combined with
-fsanitize-recover=unsigned-integer-overflow, is useful for providing
fuzzing signal without the excessive log output.

Helps with https://github.com/google/oss-fuzz/issues/910.

Reviewers: kcc, vsk

Reviewed By: vsk

Subscribers: vsk, kubamracek, Dor1s, llvm-commits

Differential Revision: https://reviews.llvm.org/D48660

llvm-svn: 335762
2018-06-27 18:24:46 +00:00
Petr Hosek 3209b28aa9 [sanitizer] zx_cprng_draw no longer takes the output argument
The zx_cprng_draw system call no longer takes the output argument.

Differential Revision: https://reviews.llvm.org/D48657

llvm-svn: 335755
2018-06-27 16:49:37 +00:00
Dan Liew bb78eef6b6 [CMake] Tidy up the organisation of compiler-rt when configured as a standalone
build with an IDE (e.g. Xcode) as the generator.

Previously the global `USE_FOLDERS` property wasn't set in standalone
builds leading to existing settings of FOLDER not being respected.

In addition to this there were several targets that appeared at the top
level that were not interesting and clustered up the view. These have
been changed to be displayed in "Compiler-RT Misc".

Now when an Xcode project is generated from a standalone compiler-rt
build the project navigator is much less cluttered. The interesting
libraries should appear in "Compiler-RT Libraries" in the IDE.

Differential Revision: https://reviews.llvm.org/D48378

llvm-svn: 335728
2018-06-27 12:56:34 +00:00
Vlad Tsyrklevich e745cf9bf3 CFI: Print DSO names for failed cross-DSO icalls
Reviewers: pcc

Reviewed By: pcc

Subscribers: kubamracek, delcypher, llvm-commits, kcc, #sanitizers

Differential Revision: https://reviews.llvm.org/D48583

llvm-svn: 335644
2018-06-26 18:51:04 +00:00
Matt Morehouse 0948bc2086 [libFuzzer] Do not turn unittest warnings into errors.
Some warnings originating from googletest were causing bots to fail
while bulding unit tests.  The sanitizers address this issue by not
using -Werror.  We adopt this approach for libFuzzer.

llvm-svn: 335640
2018-06-26 18:37:37 +00:00
Peter Collingbourne e44acadf6a Implement CFI for indirect calls via a member function pointer.
Similarly to CFI on virtual and indirect calls, this implementation
tries to use program type information to make the checks as precise
as possible.  The basic way that it works is as follows, where `C`
is the name of the class being defined or the target of a call and
the function type is assumed to be `void()`.

For virtual calls:
- Attach type metadata to the addresses of function pointers in vtables
  (not the functions themselves) of type `void (B::*)()` for each `B`
  that is a recursive dynamic base class of `C`, including `C` itself.
  This type metadata has an annotation that the type is for virtual
  calls (to distinguish it from the non-virtual case).
- At the call site, check that the computed address of the function
  pointer in the vtable has type `void (C::*)()`.

For non-virtual calls:
- Attach type metadata to each non-virtual member function whose address
  can be taken with a member function pointer. The type of a function
  in class `C` of type `void()` is each of the types `void (B::*)()`
  where `B` is a most-base class of `C`. A most-base class of `C`
  is defined as a recursive base class of `C`, including `C` itself,
  that does not have any bases.
- At the call site, check that the function pointer has one of the types
  `void (B::*)()` where `B` is a most-base class of `C`.

Differential Revision: https://reviews.llvm.org/D47567

llvm-svn: 335569
2018-06-26 02:15:47 +00:00
Alex Shlyapnikov d42edb28d0 [HWASan] Initalize shadow earler.
Summary:
Initialize shadow memory before calling more libc functions to allow
for HWASan-instrumented libc.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48551

llvm-svn: 335502
2018-06-25 17:27:13 +00:00
Matt Morehouse 988f2da613 [libFuzzer] Use Vector rather than std::vector.
llvm-svn: 335487
2018-06-25 15:59:24 +00:00
Vedant Kumar c835306ac5 [ubsan] Fix __ubsan_on_report interface definition
Speculative fix for the interface definition of __ubsan_on_report for
the Windows bots:

  http://lab.llvm.org:8011/builders/sanitizer-windows/builds/30528
  lib\ubsan\ubsan_interface.inc(55): error C2065: '__ubsan_on_report':
  undeclared identifier

INTERCEPT_SANITIZER_WEAK_FUNCTION was the wrong macro to use to begin
with because __ubsan_on_report isn't weak. Reading through that macro,
it's still not clear to me why there is an undefined reference, though,
because it appears to define a dummy __ubsan_on_report shim.

llvm-svn: 335383
2018-06-22 20:15:33 +00:00
Vedant Kumar 059d20360a [ubsan] Add support for reporting diagnostics to a monitor process
Add support to the ubsan runtime for reporting diagnostics to a monitor
process (e.g a debugger).

The Xcode IDE uses this by setting a breakpoint on __ubsan_on_report and
collecting diagnostic information via __ubsan_get_current_report_data,
which it then surfaces to users in the editor UI.

Testing for this functionality already exists in upstream lldb, here:
lldb/packages/Python/lldbsuite/test/functionalities/ubsan

Apart from that, this is `ninja check-{a,ub}san` clean.

Differential Revision: https://reviews.llvm.org/D48446

llvm-svn: 335371
2018-06-22 17:21:17 +00:00
Dmitry Vyukov ac5fa6605c tsan: fix deficiency in MutexReadOrWriteUnlock
MutexUnlock uses ReleaseStore on s->clock, which is the right thing to do.
However MutexReadOrWriteUnlock for writers uses Release on s->clock.
Make MutexReadOrWriteUnlock also use ReleaseStore for consistency and performance.
Unfortunately, I don't think any test can detect this as this only potentially
affects performance.

llvm-svn: 335322
2018-06-22 08:27:52 +00:00
Kostya Kortchinsky 307c2eb94f [scudo] Add a minimal runtime for -fsanitize-minimal-runtime compatibility
Summary:
This patch follows D48373.

The point is to be able to use Scudo with `-fsanitize-minimal-runtime`. For that
we need a runtime that doesn't embed the UBSan one. This results in binaries
that can be compiled with `-fsanitize=scudo,integer -fsanitize-minimal-runtime`.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48377

llvm-svn: 335296
2018-06-21 21:48:04 +00:00
Kuba Mracek 14c786a960 [tsan] Use DARWIN_osx_LINK_FLAGS when building unit tests to match ASan behavior.
llvm-svn: 335265
2018-06-21 18:00:51 +00:00
David Carlier 7d796ececd [TSan] fix build and couple of unit tests on FreeBSD
Similarly to Msan adding -pie provokes linkage issue, was spotted with pie_test.cc
Set to XFAIL for couple of unit tests.

Reviewers: vitalybuka, krytarowski, dim

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48317

llvm-svn: 335166
2018-06-20 20:17:44 +00:00
David Carlier 8979d44d31 [Lsan] intercept thr_exit on FreeBSD
Intercepts thr_exit call on FreeBSD.
Disable pthread key workflow.
The pthread key create approach does not function under FreeBSD as the libpthread is not initialised enough at this stage.

Reviewers: vitalybuka, krytarowski, dim

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48268

llvm-svn: 335164
2018-06-20 20:13:25 +00:00
Alex Shlyapnikov 7e067ab1af [Sanitizers] Remove OOM/BadRequest allocator error handling policies.
Summary:
Remove the generic error nadling policies and handle each allocator error
explicitly. Although more verbose, it allows for more comprehensive, precise
and actionable allocator related failure reports.

This finishes up the series of changes of the particular sanitizer
allocators, improves the internal allocator error reporting and removes
now unused policies.

Reviewers: vitalybuka, cryptoad

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48328

llvm-svn: 335147
2018-06-20 17:10:33 +00:00
Fangrui Song 6fa3bf9588 [XRay] Fix error message. NFC
Reviewers: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48313

llvm-svn: 335055
2018-06-19 15:39:19 +00:00
Kostya Kortchinsky 63c33b1c9b [scudo] Move noinline functions definitions out of line
Summary:
Mark `isRssLimitExceeded` as `NOINLINE`, and move it's definition as well as
the one of `performSanityChecks` out of the class definition, as requested.

Reviewers: filcab, alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48228

llvm-svn: 335054
2018-06-19 15:36:30 +00:00
Francis Visoiu Mistrih 972a462a7e Fix bots after r334981
llvm-svn: 335013
2018-06-19 02:56:24 +00:00
Reid Kleckner b8a5ee2b00 [asan] Avoid deadlock when initializing the symbolizer CHECK fails
llvm-svn: 335007
2018-06-19 00:36:47 +00:00
Chris Bieneman feb4537b44 Fixing os_version_check.c to be actual C source
The initial implementaiton was using the C++ typeof keyword. This
causes the compiler to spew warnings unnecissarilly. This patch removes
the uses of typeof and replaces them with explicit typedefs of the
function types.

llvm-svn: 334981
2018-06-18 20:56:19 +00:00
Alex Shlyapnikov c75d47b52d [TSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404 and other sanitizer allocators.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, no stack, no details, not too
helpful nor informative. To improve the situation, detailed and structured
common errors were defined and reported under the appropriate conditions.

Common tests were generalized a bit to cover a slightly different TSan
stack reporting format, extended to verify errno value and returned
pointer value check is now explicit to facilitate debugging.

Reviewers: dvyukov

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48087

llvm-svn: 334975
2018-06-18 20:03:31 +00:00
Francis Visoiu Mistrih 1197f01bb9 [sanitizer] Guard call to internal_lseek when SANITIZER_MAC is true
r334881 breaks macOS bots because internal_lseek is not defined (neither
used on macOS):
http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/46240/consoleFull.

See discussion from r334881: https://reviews.llvm.org/rL334881

llvm-svn: 334944
2018-06-18 13:53:51 +00:00
Petr Hosek 068246886a [Fuzzer] Set an explicit libc++ dependency when needed
On targets that don't link internal libc++ (Fuchsia and Linux) but
use libc++ as their C++ library and libFuzzer is being built using
the just built compiler together with libc++ as part of runtimes, we
need an explicit dependency from libFuzzer object library to libc++ to
make sure the headers are available by the time we start building
libFuzzer.

Differential Revision: https://reviews.llvm.org/D48261

llvm-svn: 334928
2018-06-18 08:01:57 +00:00
Vitaly Buka 7fd0d388e0 [sanitizer] Suppress unused function warning
llvm-svn: 334923
2018-06-18 05:55:44 +00:00
Vitaly Buka 87ea4487f7 [sanitizer] Fix tsan GO build
llvm-svn: 334914
2018-06-17 17:58:08 +00:00
Vitaly Buka 4647c8b93d [sanitizer] Fix s390 build after r334900
llvm-svn: 334913
2018-06-17 17:40:38 +00:00
Vitaly Buka 755f0e8174 [sanitizer] Use confstr to check libc version in InitTlsSize
Reviewers: Lekensteyn, jakubjelinek

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D48265

llvm-svn: 334912
2018-06-17 17:31:22 +00:00
Vitaly Buka 4d6b33c510 [fuzzer] Python 3 print fixes
llvm-svn: 334902
2018-06-17 09:11:56 +00:00
Vitaly Buka b213a345e1 [fuzzer] Fix collect_data_flow.py for python 3
llvm-svn: 334901
2018-06-17 08:41:56 +00:00
Vitaly Buka b957f99d57 [sanitizer] Use const char* in internal_simple_strtoll
llvm-svn: 334900
2018-06-17 08:41:45 +00:00
Fangrui Song 87bcd4abef [sanitizer_common] Fix windows build caused by r334881
llvm-svn: 334884
2018-06-16 05:05:19 +00:00
Fangrui Song d40fc6019b [sanitizer_common] Use O_TRUNC for WrOnly access mode.
Summary: Otherwise if the file existed and was larger than the write size before the OpenFile call, the file will not be truncated and contain garbage in trailing bytes.

Reviewers: glider, kcc, vitalybuka

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48250

llvm-svn: 334881
2018-06-16 03:32:59 +00:00
Petr Hosek 5e6bc090ed [Fuzzer] Don't hardcode target architecture for Fuzzer tests
Don't hardcode the architecture for Fuzzer tests which breaks when
compiler-rt is being compiled for architectures other than x86_64.

Differential Revision: https://reviews.llvm.org/D48207

llvm-svn: 334852
2018-06-15 18:21:02 +00:00
Kostya Kortchinsky 4adf24502e [scudo] Add verbose failures in place of CHECK(0)
Summary:
The current `FailureHandler` mechanism was fairly opaque with regard to the
failure reason due to using `CHECK(0)`. Scudo is a bit different from the other
Sanitizers as it prefers to avoid spurious processing in its failure path. So
we just `dieWithMessage` using a somewhat explicit string.

Adapted the tests for the new strings.

While this takes care of the `OnBadRequest` & `OnOOM` failures, the next step
is probably to migrate the other Scudo failures in the same failes (header
corruption, invalid state and so on).

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: filcab, mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48199

llvm-svn: 334843
2018-06-15 16:45:19 +00:00
Eli Friedman 16c4456022 [compiler-rt] [builtins] Don't build __atomic_* by default.
The locks need to be implemented in a shared library to work correctly,
so they shouldn't be part of libclang_rt.builtins.a, except in
specialized scenarios where the user can prove it will only be linked
once.

Differential Revision: https://reviews.llvm.org/D47606

llvm-svn: 334779
2018-06-14 23:22:53 +00:00
George Karpenkov d1e0365ce3 [libFuzzer] [NFC] Support multi-arch and multi-OS building and testing
Differential Revision: https://reviews.llvm.org/D47296

llvm-svn: 334768
2018-06-14 20:46:07 +00:00
Alex Shlyapnikov c9b41354ff [MSan] Linker-initialize static fallback_mutex in msan_allocator.cc
Summary:
static fallback_mutex in msan_allocator.cc does not need the ctor
call and can be linker initialized.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: morehouse, eugenis

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48145

llvm-svn: 334749
2018-06-14 18:30:51 +00:00
Alex Shlyapnikov 241b758690 [ASan] Linker-initialize static ScopedInErrorReport::current_error_.
Summary:
Static ScopedInErrorReport::current_error_ can be linker initialized to
shave one global ctor call on application startup and be __asan_init-safe.

Global constructors in ASan runtime are bad because __asan_init runs
from preinit_array, before any such constructors.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: eugenis, morehouse

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48141

llvm-svn: 334748
2018-06-14 18:22:23 +00:00
Peter Wu 4ff2d87f30 [ASAN] fix typos and disable long-object-path test for win32
Glob patterns seem unsupported for commands executed by the emulated
shell (LIT_USE_INTERNAL_SHELL=1). Disable the test while that is being
addressed (a workaround such as "cd a-*" also does not work).

llvm-svn: 334719
2018-06-14 15:12:46 +00:00
Kostya Kortchinsky 1ee741bdb6 [scudo] Make Secondary linker-initialized compliant
Summary:
As a follow up to D48142 for Scudo, switch the `SpinMutex` to its static
counterpart, and ensure zero-initialization by memset'ing the whole class.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48148

llvm-svn: 334716
2018-06-14 14:33:28 +00:00
Peter Wu 1bbab1e558 [ASAN] fix startup crash in dlsym for long paths since glibc 2.27
Summary:
Error messages for dlsym used to be stored on the stack, but since
commit 2449ae7b ("ld.so: Introduce struct dl_exception") in glibc 2.27
these are now stored on the heap (and thus use the dlsym alloc pool).

Messages look like "undefined symbol: __isoc99_printf\0/path/to/a.out".
With many missing library functions and long object paths, the pool is
quickly exhausted. Implement a simple mechanism to return freed memory
to the pool (clear it in case it is used for calloc).

Fixes https://github.com/google/sanitizers/issues/957

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D47995

llvm-svn: 334703
2018-06-14 10:42:54 +00:00
David Carlier 706ea47bdc [Sanitizer] fix compilation warning
In most of systems, this field is a signed type but in some it is an unsigned.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D48118

llvm-svn: 334686
2018-06-14 04:51:26 +00:00
Peter Wu 0569660af3 [ASAN] Fix fputs interception for 32-bit macOS
On 32-bit macOS, "_fputs$UNIX2003" is called instead of "fputs" and the
"fgets_fputs" test fails. Apparently previous versions still passed the
test due to the internal implementation calling "strlen", but that does
not seem to be the case with macOS 10.13.3. Fixes r334450.

llvm-svn: 334670
2018-06-13 23:19:21 +00:00
Alex Shlyapnikov 3fc66075c7 [Sanitizers] Make sanitizer allocator linker-initialize compliant.
Summary:
These four SpinMutex ctors was the only code executed in the ctor for
the static __asan::Allocator instance (same for the other sanitizers
allocators), which is supposed to be fully linker-initialized.

Also, when the global ctor for this allocator instance is executed,
this instance might already be initialized by __asan_init called from
.preinit_array.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: morehouse, eugenis, cryptoad

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48142

llvm-svn: 334660
2018-06-13 21:45:01 +00:00
Dimitry Andric a00c372cd3 Disable MSan tests of prlimit on FreeBSD
Like NetBSD, FreeBSD does not have prlimit(2), so do not attempt to use
it during the MSan tests.

llvm-svn: 334659
2018-06-13 21:37:49 +00:00
Alex Shlyapnikov 4bf9b5612a [TSan] Fix madvise(MADV_NOHUGEPAGE) for meta shadow memory
Summary:
Move madvise(MADV_NOHUGEPAGE) for the meta shadow memory after the meta
shadow memory is mapped (currently it silently fails with ENOMEM).

Add a diagnostic message to detect similar problems in the future.

Reviewers: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48097

llvm-svn: 334624
2018-06-13 17:18:41 +00:00
Petr Hosek 7252e2a8bd [XRay] Set an explicit dependency on libc++ when needed
When XRay is being built as part of the just built compiler together
with libc++ as part of the runtimes build, we need an explicit
dependency from XRay to libc++ to make sure that the library is
available by the time we start building XRay.

Differential Revision: https://reviews.llvm.org/D48113

llvm-svn: 334575
2018-06-13 07:08:28 +00:00
Petr Hosek 22f8825dca Reland "Passthrough additional flags to custom libcxx CMake build"
This is needed when we're cross-compiling compiler-rt.

Differential Revision: https://reviews.llvm.org/D47834

llvm-svn: 334570
2018-06-13 05:32:22 +00:00
Matt Morehouse e3300d2ff3 Revert r334458, r334220, r334212, r334139.
Reverts changes to AddCompilerRT.cmake due to breakage of
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/.

llvm-svn: 334528
2018-06-12 19:43:40 +00:00
Walter Lee 0a58f9c953 [asan, myriad] Support environment variables
Provide a buffer that the test harness can write into to provide
values for the environment variables.  Format is a null-separated list
of VAR=value pairs; this is sufficent for our purpose.

Differential Revision: https://reviews.llvm.org/D47960

llvm-svn: 334522
2018-06-12 18:34:12 +00:00
David Carlier 301855fb0d [Fuzzer] Afl driver changing iterations handling
Handling differently the iterations with the type limit and eventually an error message.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D47880

llvm-svn: 334510
2018-06-12 15:47:58 +00:00
Kostya Kortchinsky 76969eaf3d [scudo] Add C++17 aligned new/delete operators support
Summary:
This CL adds support for aligned new/delete operators (C++17). Currently we
do not support alignment inconsistency detection on deallocation, as this
requires a header change, but the APIs are introduced and are functional.

Add a smoke test for the aligned version of the operators.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48031

llvm-svn: 334505
2018-06-12 14:42:40 +00:00
Dean Michael Berris aea8ece102 [XRay][profiler] Part 5: Profiler File Writing
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements the profile writing mechanism, to allow profiles
collected through the profiler mode to be persisted to files.

Follow-on patches would allow us to load these profiles and start
converting/analysing them through the `llvm-xray` tool.

Depends on D44620.

Reviewers: echristo, kpw, pelikan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45998

llvm-svn: 334472
2018-06-12 04:06:25 +00:00
Dean Michael Berris cfd7eec3d8 [XRay][profiler] Part 4: Profiler Mode Wiring
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements the wiring required to enable us to actually
select the `xray-profiling` mode, and install the handlers to start
measuring the time and frequency of the function calls in call stacks.
The current way to get the profile information is by working with the
XRay API to `__xray_process_buffers(...)`.

In subsequent changes we'll implement profile saving to files, similar
to how the FDR and basic modes operate, as well as means for converting
this format into those that can be loaded/visualised as flame graphs. We
will also be extending the accounting tool in LLVM to support
stack-based function call accounting.

We also continue with the implementation to support building small
histograms of latencies for the `FunctionCallTrie::Node` type, to allow
us to actually approximate the distribution of latencies per function.

Depends on D45758 and D46998.

Reviewers: eizan, kpw, pelikan

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D44620

llvm-svn: 334469
2018-06-12 03:29:39 +00:00
Ryan Prichard 1e216a59f6 [builtins] Delay emutls deallocation for one round
Summary:
With Android/Bionic, delay deallocation to round 2 of 4. It must run after
C++ thread_local destructors have been called, but before the final 2
rounds, because emutls calls free, and jemalloc then needs another 2
rounds to free its thread-specific data.

Fixes https://github.com/android-ndk/ndk/issues/687

Reviewers: cmtice, srhines, jyknight, chh, echristo

Reviewed By: srhines, chh, echristo

Subscribers: echristo, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46978

llvm-svn: 334463
2018-06-12 01:32:26 +00:00
Ryan Prichard 31e4a550a3 [builtins] emutls cleanup: determine header size using sizeof
Summary: Also add a few post-#else/#endif comments

Reviewers: echristo, srhines

Reviewed By: echristo

Subscribers: chh, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47861

llvm-svn: 334462
2018-06-12 01:32:25 +00:00
Peter Wu 1c05c95739 [sanitizer] Add fgets, fputs and puts into sanitizer_common
Summary:
Add fgets, fputs and puts to sanitizer_common. This adds ASAN coverage
for these functions, extends MSAN support from fgets to fputs/puts and
extends TSAN support from puts to fputs.

Fixes: https://github.com/google/sanitizers/issues/952

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46545

llvm-svn: 334450
2018-06-11 22:58:04 +00:00
Alex Shlyapnikov 789494026e [Sanitizers] Move pvalloc overflow tests to common.
Summary:
Now all sanitizers with improved allocator error reporting are covered
by these common tests.

Also, add pvalloc-specific checks to LSan.

HWASan is not covered by sanitizer_common, hence its own pvalloc
and other allocator tests.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47970

llvm-svn: 334424
2018-06-11 17:33:53 +00:00
Kostya Kortchinsky 4410e2c43f [scudo] Improve the scalability of the shared TSD model
Summary:
The shared TSD model in its current form doesn't scale. Here is an example of
rpc2-benchmark (with default parameters, which is threading heavy) on a 72-core
machines (defaulting to a `CompactSizeClassMap` and no Quarantine):
- with tcmalloc: 337K reqs/sec, peak RSS of 338MB;
- with scudo (exclusive): 321K reqs/sec, peak RSS of 637MB;
- with scudo (shared): 241K reqs/sec, peak RSS of 324MB.

This isn't great, since the exclusive model uses a lot of memory, while the
shared model doesn't even come close to be competitive.

This is mostly due to the fact that we are consistently scanning the TSD pool
starting at index 0 for an available TSD, which can result in a lot of failed
lock attempts, and touching some memory that needs not be touched.

This CL attempts to make things better in most situations:
- first, use a thread local variable on Linux (intead of pthread APIs) to store
  the current TSD in the shared model;
- move the locking boolean out of the TSD: this allows the compiler to use a
  register and potentially optimize out a branch instead of reading it from the
  TSD everytime (we also save a tiny bit of memory per TSD);
- 64-bit atomic operations on 32-bit ARM platforms happen to be expensive: so
  store the `Precedence` in a `uptr` instead of a `u64`. We lose some
  nanoseconds of precision and we'll wrap around at some point, but the benefit
  is worth it;
- change a `CHECK` to a `DCHECK`: this should never happen, but if something is
  ever terribly wrong, we'll crash on a near null AV if the TSD happens to be
  null;
- based on an idea by dvyukov@, we are implementing a bound random scan for
  an available TSD. This requires computing the coprimes for the number of TSDs,
  and attempting to lock up to 4 TSDs in an random order before falling back to
  the current one. This is obviously slightly more expansive when we have just
  2 TSDs (barely noticeable) but is otherwise beneficial. The `Precedence` still
  basically corresponds to the moment of the first contention on a TSD. To seed
  on random choice, we use the precedence of the current TSD since it is very
  likely to be non-zero (since we are in the slow path after a failed `tryLock`)

With those modifications, the benchmark yields to:
- with scudo (shared): 330K reqs/sec, peak RSS of 327MB.

So the shared model for this specific situation not only becomes competitive but
outperforms the exclusive model. I experimented with some values greater than 4
for the number of TSDs to attempt to lock and it yielded a decrease in QPS. Just
sticking with the current TSD is also a tad slower. Numbers on platforms with
less cores (eg: Android) remain similar.

Reviewers: alekseyshl, dvyukov, javed.absar

Reviewed By: alekseyshl, dvyukov

Subscribers: srhines, kristof.beyls, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47289

llvm-svn: 334410
2018-06-11 14:50:31 +00:00
Peter Wu 1db004ebdd [ASAN] Fix crash on i?86-linux (32-bit) against glibc 2.27 and later
Summary:
Running sanitized 32-bit x86 programs on glibc 2.27 crashes at startup, with:

    ERROR: AddressSanitizer: SEGV on unknown address 0xf7a8a250 (pc 0xf7f807f4 bp 0xff969fc8 sp 0xff969f7c T16777215)
    The signal is caused by a WRITE memory access.
    #0 0xf7f807f3 in _dl_get_tls_static_info (/lib/ld-linux.so.2+0x127f3)
    #1 0xf7a92599  (/lib/libasan.so.5+0x112599)
    #2 0xf7a80737  (/lib/libasan.so.5+0x100737)
    #3 0xf7f7e14f in _dl_init (/lib/ld-linux.so.2+0x1014f)
    #4 0xf7f6eb49  (/lib/ld-linux.so.2+0xb49)

The problem is that glibc changed the calling convention for the GLIBC_PRIVATE
symbol that sanitizer uses (even when it should not, GLIBC_PRIVATE is exactly
for symbols that can change at any time, be removed etc.), see
https://sourceware.org/ml/libc-alpha/2017-08/msg00497.html

Fixes https://github.com/google/sanitizers/issues/954

Patch By: Jakub Jelinek

Reviewed By: vitalybuka, Lekensteyn

Differential Revison: https://reviews.llvm.org/D44623

llvm-svn: 334363
2018-06-10 11:17:47 +00:00
Alex Shlyapnikov 643600566d [MSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47793

llvm-svn: 334338
2018-06-08 23:31:42 +00:00
Walter Lee 5a780ee9a9 [asan, myriad] Use local pool for new/delete when ASan run-time is not up
This can happen on Myriad RTEMS so needs to be handled.

Differential Revision: https://reviews.llvm.org/D47916

llvm-svn: 334329
2018-06-08 21:49:38 +00:00
Alex Shlyapnikov dcf0097962 [Sanitizers] Check alignment != 0 for aligned_alloc and posix_memalign
Summary:
Move the corresponding tests to the common folder (as all of the
sanitizer allocators will support this feature soon) and add the checks
specific to aligned_alloc to ASan and LSan allocators.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47924

llvm-svn: 334316
2018-06-08 20:40:35 +00:00
David Carlier 6dedf2b5d8 [Xray] logging forgotten header
pthread.h missing for pthread_key* functions.

Reviewers: dberris

Reviewed By: dberris

Differential Revison: https://reviews.llvm.org/D47933

llvm-svn: 334272
2018-06-08 08:42:37 +00:00
Dean Michael Berris b81372b56b [XRay][compiler-rt] Cleanup some internal XRay utilities
This change uses 'const' for the retryingWriteAll(...) API and removes
unnecessary 'static' local variables in getting the temporary filename.

llvm-svn: 334267
2018-06-08 07:48:03 +00:00
Dean Michael Berris af4d583ea8 [XRay][compiler-rt] Remove reliance on C++ ABI features
Summary:
This fixes http://llvm.org/PR32274.

This change adds a test to ensure that we're able to link XRay modes and
the runtime to binaries that don't need to depend on the C++ standard
library or a C++ ABI library. In particular, we ensure that this will work
with C programs compiled+linked with XRay.

To make the test pass, we need to change a few things in the XRay
runtime implementations to remove the reliance on C++ ABI features. In
particular, we change the thread-safe function-local-static
initialisation to use pthread_* instead of the C++ features that ensure
non-trivial thread-local/function-local-static initialisation.

Depends on D47696.

Reviewers: dblaikie, jfb, kpw, eizan

Reviewed By: kpw

Subscribers: echristo, eizan, kpw, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46998

llvm-svn: 334262
2018-06-08 04:00:07 +00:00
Alex Shlyapnikov af12729185 [HWASan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47798

llvm-svn: 334248
2018-06-07 23:33:33 +00:00
Kostya Serebryany 111d0b2aec [libFuzzer] When printing NEW_FUNC, use 1-base indexing.
Summary: Otherwise we print things like [0/1] which is visually confusing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47837

llvm-svn: 334234
2018-06-07 21:15:24 +00:00
Douglas Yung 20fa3d2504 Silence a -Wconstant-logical-operand warning.
llvm-svn: 334214
2018-06-07 18:51:47 +00:00
Petr Hosek 157a5c2f09 [Fuzzer] Update the header path for fdio/spawn.h on Fuchsia
The path now includes lib/ prefix.

Differential Revision: https://reviews.llvm.org/D47866

llvm-svn: 334213
2018-06-07 18:41:35 +00:00
Kamil Rytarowski b9c78e41ba Disable recursive interceptors in signal(3)/MSan
Summary:
signal(3) on NetBSD calls internally sigaction(2).

Without disabling the recursive interceptor, there are
false positives about uninitialized memory reads inside libc.

This change fixes execution of such programs as sh(1) and
vmstat(1) in the NetBSD userland.

Sponsored by <The NetBSD Foundation>

Reviewers: eugenis, vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47650

llvm-svn: 334168
2018-06-07 07:55:20 +00:00
Kostya Serebryany 67af99235f [libFuzzer] make the corpus elements aware of their data flow traces
llvm-svn: 334158
2018-06-07 01:40:20 +00:00
Vitaly Buka 661d81ef61 [sanitizer] Don't use internal_unlink on Windows
llvm-svn: 334152
2018-06-07 00:26:06 +00:00
Kostya Serebryany 4d9fd7a266 [libFuzzer] remove an experimental flag -use_feature_frequency
llvm-svn: 334146
2018-06-06 23:24:41 +00:00
Petr Hosek 3a32b41024 [Fuzzer] Use private libc++ even for Fuchsia
On Fuchsia, we use libc++ compiled with ASan for our ASan built
executable which means we cannot use the same libc++ for libFuzzer when
building fuzz targets, instead we'll link a custom internal libc++ into
Fuchsia's build of libFuzzer like we already do on Linux.

Differential Revision: https://reviews.llvm.org/D47835

llvm-svn: 334144
2018-06-06 22:59:14 +00:00
Petr Hosek 3cf6db2d56 [CMake] Passthrough additional flags to custom libcxx CMake build
This is needed when we're cross-compiling compiler-rt.

Differential Revision: https://reviews.llvm.org/D47834

llvm-svn: 334139
2018-06-06 22:10:12 +00:00
Vitaly Buka 72fd23c9c5 [sanitizer] Replace deprecated mktemp with mkstemp
llvm-svn: 334138
2018-06-06 22:05:48 +00:00
Vitaly Buka 4ef9117b49 [sanitizer] Cleanup ReadFileToVector and ReadFileToBuffer
Summary:
Added unit-test.
Fixed behavior of max_len argument.
Call read syscall with all available buffer, not just a page.

Reviewers: eugenis

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46618

llvm-svn: 334130
2018-06-06 20:53:43 +00:00
Walter Lee 381072c417 [asan] Fix Myraid RTEMS port broken by r333985
Add nop CheckASLR() function.

llvm-svn: 334102
2018-06-06 15:39:47 +00:00
Ilya Biryukov b7c3745006 Fix compile error with libstdc++.
By adding a ctor to create fuzzer_allocator<T> from fuzzer_allocator<U>.
This mimics construcotrs of std::allocator<T>.

Without the constructors, some versions of libstdc++ can't compile
`vector<bool, fuzzer_allocator<bool>>`.

llvm-svn: 334077
2018-06-06 09:22:19 +00:00
Dean Michael Berris c561970b27 [XRay][compiler-rt] Refactor recursion guard for Basic and FDR Mode
Summary:
This change extracts the recursion guard implementation from FDR Mode
and updates it to do the following:

- Do the atomic operation correctly to be signal-handler safe.

- Make it usable in both FDR and Basic Modes.

Before this change, the recursion guard relied on an unsynchronised read
and write on a volatile thread-local. A signal handler could then run in
between the read and the write, and then be able to run instrumented
code as part of the signal handling. Using an atomic exchange instead
fixes that by doing a proper mutual exclusion even in the presence of
signal handling.

Reviewers: kpw, eizan, jfb

Reviewed By: eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47696

llvm-svn: 334064
2018-06-06 06:07:48 +00:00
Kostya Serebryany 1fd005f552 [libFuzzer] initial implementation of -data_flow_trace. It parses the data flow trace and prints the summary, but doesn't use the information in any other way yet
llvm-svn: 334058
2018-06-06 01:23:29 +00:00
Vitaly Buka b89704fa6f [lsan] Do not check for leaks in the forked process
Summary:
If calling process had threads then forked process will fail to detect
references from them.

Fixes https://github.com/google/sanitizers/issues/836

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47751

llvm-svn: 334036
2018-06-05 18:15:57 +00:00
Alex Shlyapnikov 236c3f9c4a [LSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47645

llvm-svn: 334034
2018-06-05 18:02:09 +00:00
Dean Michael Berris 4c1fdcfb97 [XRay][compiler-rt] Use DCHECK instead of assert(...) (NFC)
Use DCHECK instead of assert(...) in the XRay runtime.

llvm-svn: 334002
2018-06-05 10:27:20 +00:00
Dean Michael Berris 13b2fcc4e6 [XRay][compiler-rt] Use static instead of inline (NFC)
We don't actually need to support multiple definitions of the functions
in FDR mode, but rather want to make sure that the implementation-detail
functions are marked as 'static' instead. This allows the inliner to do
its magic better for these functions too, since inline functions must
have a unique address across translation units.

llvm-svn: 334001
2018-06-05 10:18:39 +00:00
Dean Michael Berris abbeb4cbf7 [XRay][compiler-rt] Remove __xray:: in some places (NFC)
This is a cosmetic change to remove unnecessary full-qualifications of
types/functions that are already in the __xray:: namespace.

llvm-svn: 334000
2018-06-05 10:12:58 +00:00
Dean Michael Berris bd37b57b4d [XRay][compiler-rt] Remove namespace __xray_fdr_internal (NFC)
We no longer need the __xray_fdr_internal namespace.

llvm-svn: 333998
2018-06-05 10:01:45 +00:00
Dean Michael Berris 7f88ea1fa2 [XRay][compiler-rt] Merge XRay FDR mode into a single file (NFC)
We planned to have FDR mode's internals unit-tested but it turns out
that we can just use end-to-end testing to verify the implementation.
We're going to move towards that approach more and more going forward,
so we're merging the implementation details of FDR mode into a single
.cc file.

We also avoid globbing in the XRay test helper macro, and instead list
down the files from the lib directory.

llvm-svn: 333986
2018-06-05 08:20:54 +00:00
Kamil Rytarowski 7d260775f3 Introduce CheckASLR() in sanitizers
Summary:
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD.

Introduce a generic CheckASLR() routine, that implements a check for the
current process. This flag depends on the global or per-process settings.

There is no simple way to disable ASLR in the build process from the
level of a sanitizer or during the runtime execution.

With ASLR enabled sanitizers that operate over the process virtual address
space can misbehave usually breaking with cryptic messages.

This check is dummy for !NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: cryptoad, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47442

llvm-svn: 333985
2018-06-05 07:29:23 +00:00
Dean Michael Berris 5eaaff6095 [XRay][compiler-rt] Remove __sanitizer:: from namespace __xray (NFC)
This is a non-functional change that removes the full qualification of
functions in __sanitizer:: being used in __xray.

llvm-svn: 333983
2018-06-05 06:12:42 +00:00
Dean Michael Berris 8ba925d2c3 [XRay][compiler-rt] Remove reliance on C++ ABI from BufferQueue
Summary:
This is part of the work to address http://llvm.org/PR32274.

We remove the calls to array-placement-new and array-delete. This allows
us to rely on the internal memory management provided by
sanitizer_common/sanitizer_internal_allocator.h.

Reviewers: eizan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47695

llvm-svn: 333982
2018-06-05 03:46:54 +00:00
David Carlier a6303a9866 [TSan] FreeBSD / intercept thr_exit
intercepting thr_exit to terminate threads under FreeBSD.
Unblock few unit tests hanging.

Reviewers: krytarowski, vitalybuka, emaste

Reviewed By: dvyukov		

Differential Revision: https://reviews.llvm.org/D47677

M    lib/tsan/rtl/tsan_interceptors.cc

llvm-svn: 333870
2018-06-04 06:02:38 +00:00
Dean Michael Berris d12cb59c7a [XRay][compiler-rt] Remove RTTI requirement from XRay Builds
XRay doesn't use RTTI and doesn't need it. We disable it explicitly in
the CMake config, similar to how the other sanitizers already do it.

Part of the work to address http://llvm.org/PR32274.

llvm-svn: 333867
2018-06-04 04:22:26 +00:00
Petr Hosek 8908aa4e30 [Fuzzer] Migrate Fuchsia port from launchpad to fdio_spawn
fdio_spawn is the new public API for launching processes in Fuchsia.

Differential Revision: https://reviews.llvm.org/D47649

llvm-svn: 333809
2018-06-02 01:17:10 +00:00
Kostya Serebryany d74d04a6c5 Add weak definitions of trace-cmp hooks to dfsan
Summary:
This allows to build and link the code with e.g.
-fsanitize=dataflow -fsanitize-coverage=trace-pc-guard,pc-table,func,trace-cmp
w/o providing (all) the definitions of trace-cmp hooks.

This is similar to dummy hooks provided by asan/ubsan/msan for the same purpose,
except that some of the hooks need to have the __dfsw_ prefix
since we need dfsan to replace them.

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47605

llvm-svn: 333796
2018-06-01 21:59:25 +00:00
Walter Lee e2f884d73f [asan, myriad] Implement aligned local pool allocation
Extend the local pool allocation support to posix_memalign.

Differential Revision: https://reviews.llvm.org/D47642

llvm-svn: 333788
2018-06-01 21:29:05 +00:00
Walter Lee bf03bfb0a3 [asan, myriad] Configure platform interceptors
Myriad only uses the platform interceptors for memory allocation
routines.  Configure them properly.

Also add a missing guard around aligned alloc interceptor.

Differential Revision: https://reviews.llvm.org/D47641

llvm-svn: 333784
2018-06-01 19:47:41 +00:00
Reid Kleckner d44a6eab40 Fix GCC 4.8.5 build by avoiding braced default member initializer
Use internal_memset instead.  Should revive all Linux Chromium ToT bots.

llvm-svn: 333678
2018-05-31 18:26:36 +00:00
Sylvestre Ledru ac804c2238 Define SIZEOF_STRUCT_USTAT for 32bit sparc.
Patch landed on gcc upstream:
27453e962b
Patch by Matthias Klose

llvm-svn: 333644
2018-05-31 12:41:15 +00:00
Stephan Bergmann af3226752b Missing include
(For some reason, my Mac build complained about unknown pthread_once_t and
PTHREAD_ONCE_INIT, but not about pthread_once itself.)

llvm-svn: 333638
2018-05-31 09:26:31 +00:00
Dean Michael Berris d1fe506694 [XRay] Fixup: Remove unnecessary type alias
Follow-up to D45758.

llvm-svn: 333628
2018-05-31 05:25:47 +00:00
Dean Michael Berris 16c865b071 [XRay] Fixup: Explicitly call std::make_tuple(...)
Follow-up to D45758.

llvm-svn: 333627
2018-05-31 05:02:11 +00:00
Dean Michael Berris ca856e07de [XRay] Fixup: Address some warnings breaking build
Follow-up to D45758.

llvm-svn: 333625
2018-05-31 04:55:11 +00:00
Dean Michael Berris 1eb8c206cd [XRay][profiler] Part 3: Profile Collector Service
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements a centralised collector for `FunctionCallTrie`
instances, associated per thread. It maintains a global set of trie
instances which can be retrieved through the XRay API for processing
in-memory buffers (when registered). Future changes will include the
wiring to implement the actual profiling mode implementation.

This central service provides the following functionality:

*  Posting a `FunctionCallTrie` associated with a thread, to the central
   list of tries.

*  Serializing all the posted `FunctionCallTrie` instances into
   in-memory buffers.

*  Resetting the global state of the serialized buffers and tries.

Depends on D45757.

Reviewers: echristo, pelikan, kpw

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D45758

llvm-svn: 333624
2018-05-31 04:33:52 +00:00
Kostya Serebryany 980e45fe55 [libFuzzer] add collect_data_flow.py that allows to run the data-flow tracer several times on subsets of inputs bytes, to overcome DFSan out-of-label failures
llvm-svn: 333616
2018-05-31 01:27:07 +00:00
Reid Kleckner b54ac414d1 [asan] Remove unneeded VirtualQuery from exception handler
We don't use the result of the query, and all tests pass if I remove it.
During startup, ASan spends a fair amount of time in this handler, and
the query is much more expensive than the call to commit the memory.

llvm-svn: 333595
2018-05-30 21:21:18 +00:00
Walter Lee 3dc6cea0e3 [asan, myriad] Simplify main thread handling
On Myriad RTEMS, we don't need to treat the main thread differently.
The existing thread hooks will do the right thing, so get rid of all
the unneeded special logic.

Differential Revision: https://reviews.llvm.org/D47502

llvm-svn: 333504
2018-05-30 04:57:29 +00:00
Walter Lee 0a70e9db03 [asan, myriad] Reset shadow memory during exit
Reset shadow memory during exit.  Also update a cut-and-paste comment,
and do some minor refactoring of InitializeShadowMemory.

Differential Revision: https://reviews.llvm.org/D47501

llvm-svn: 333503
2018-05-30 04:57:04 +00:00
Vitaly Buka 4add8a1f4e [safestack] Lazy initialization of interceptors
Interceptors initialization may try to allocate memory and to call not
initialized allocator.

It's similar to r326025 for CFI.

llvm-svn: 333329
2018-05-26 01:18:32 +00:00
Petr Hosek 796bb7d4ce [Fuzzer] Update _zx_port_wait function use in Fuchsia port
Fuchsia's _zx_port_wait no longer takes the count argument. This
change also updates all symbol uses to use the underscored versions.

Differential Revision: https://reviews.llvm.org/D47403

llvm-svn: 333328
2018-05-26 01:02:34 +00:00
H.J. Lu 11c1c266cc Revert the last test commit
llvm-svn: 333310
2018-05-25 20:51:45 +00:00
H.J. Lu caab7c8cca This is a test commit to verify repository access
llvm-svn: 333309
2018-05-25 20:51:10 +00:00
George Karpenkov 02c85f1489 [libFuzzer] Run libFuzzer unit tests only on host architecture.
LIT tests are used to test other cross-compiled architectures,
unit tests are only run on the host.
NFC now as currently only a single architecture is supported.

Differential Revision: https://reviews.llvm.org/D47286

llvm-svn: 333241
2018-05-24 23:55:52 +00:00
Azharuddin Mohammed 06e0fd3bc1 Revert "[cmake] [ARM] Check if VFP is supported before including any VFP builtins"
This reverts commit 2a10f5da5acb1b51d0a2ecf13dca0bf1de859db2.

llvm-svn: 333232
2018-05-24 21:36:27 +00:00
Azharuddin Mohammed 80968a9aba [cmake] [ARM] Check if VFP is supported before including any VFP builtins
Summary:
rL325492 disables FPU features when using soft floating point
(-mfloat-abi=soft), which is used internally when building for armv7. This
causes errors with builtins that utililize VFP instructions. With this change
we first check if VFP is enabled (by checking if the preprocessor macro
__VFP_FP__ is defined) before including such builtins.

Reviewers: rengolin, samsonov, compnerd, smeenai, javed.absar, peter.smith

Reviewed By: peter.smith

Subscribers: peter.smith, mgorny, kristof.beyls, chrib, llvm-commits

Differential Revision: https://reviews.llvm.org/D47217

llvm-svn: 333216
2018-05-24 18:53:16 +00:00
Craig Topper 383fe5c866 sanitizer: Use pre-computed size of struct ustat for Linux
<sys/ustat.h> has been removed from glibc 2.28 by:

commit cf2478d53ad7071e84c724a986b56fe17f4f4ca7
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Sun Mar 18 11:28:59 2018 +0800

Deprecate ustat syscall interface
This patch uses pre-computed size of struct ustat for Linux to fix

https://bugs.llvm.org/show_bug.cgi?id=37418

Patch by H.J. Lu.

Differential Revision: https://reviews.llvm.org/D47281

llvm-svn: 333213
2018-05-24 17:59:47 +00:00
Kostya Serebryany 4d53b744ca [libFuzzer] DataFlow tracer now tags a subset of the input. A separate script merges traces from the subsets
llvm-svn: 333149
2018-05-24 01:43:48 +00:00
Kostya Serebryany 500ca8713c [libFuzzer] fix two off-by-ones (!!) in the data flow tracer
llvm-svn: 333142
2018-05-23 23:55:54 +00:00
Jake Ehrlich 767d92e3fb [fuchsia] Add line buffering in RawWrite
This change causes RawWrite to buffer upto 128 bytes or until
a line is reached. This helps group calls into more readable
lines.

llvm-svn: 333136
2018-05-23 22:27:12 +00:00
Kostya Serebryany 4925392897 [libFuzzer] change the output format for the DataFlow tracer
llvm-svn: 333122
2018-05-23 20:57:11 +00:00
Max Moroz fe97441c2c [libFuzzer] Don't complain about lack of interesting inputs when -runs=0.
Summary:
The most common usecase for -runs=0 is for generating code coverage
over some corpus. Coverage reports based on sancov are about to be deprecated,
which means some external coverage solution will be used, e.g. Clang source
based code coverage, which does not use any sancov instrumentations and thus
libFuzzer would consider any input to be not interesting in that case.

Reviewers: kcc

Reviewed By: kcc

Subscribers: alex, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47271

llvm-svn: 333116
2018-05-23 19:42:30 +00:00
Kostya Kortchinsky 08c630457a [sanitizer] Define UINTPTR_MAX
Summary:
`sanitizer_internal_defs.h` didn't have this define, which will be useful in
an upcoming CL.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47270

llvm-svn: 333109
2018-05-23 18:13:21 +00:00
Kamil Rytarowski 5b1b594f64 Fix internal_mmap() on 32-bit NetBSD platforms
There is need to use internal_syscall64() instead of internal_syscall_ptr().
The offset argument of type off_t is always 64-bit.

llvm-svn: 333075
2018-05-23 10:37:00 +00:00
Igor Kudrin c0099f9be7 [sanitizer] Reland "Be more accurate when calculating the previous instruction address on ARM."
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 333071
2018-05-23 09:18:10 +00:00
Kostya Serebryany 7f5bf83f53 [libFuzzer] modify -print_corpus_stats to print whether the input reaches the focus function
llvm-svn: 333048
2018-05-23 01:42:53 +00:00
Evgeniy Stepanov 45f8986b80 [asan] Use dynamic allocator space address on Android/AArch64.
Summary:
We need one library to support all of 39, 42 and 48 bit VMAs, and
there is no common address that works for all of them.

Reviewers: kcc, alekseyshl, javed.absar

Subscribers: rengolin, srhines, kubamracek, kristof.beyls, llvm-commits, cryptoad

Differential Revision: https://reviews.llvm.org/D47160

llvm-svn: 333025
2018-05-22 20:44:45 +00:00
Walter Lee ead3b3487b [asan] Make GetCurrentThread RTEMS-friendly
On RTEMS, system and user code all live in a single binary and address
space. There is no clean separation, and instrumented code may
execute before the ASan run-time is initialized (or after it has been
destroyed).

Currently, GetCurrentThread() may crash if it's called before ASan
run-time is initialized. Make it return nullptr instead.

Similarly, fix __asan_handle_no_return so that it gives up rather than
try something that may crash.

Differential Revision: https://reviews.llvm.org/D46459

llvm-svn: 332888
2018-05-21 20:43:36 +00:00
Kostya Serebryany 69c2b71a51 [libFuzzer] reinstate -dump_coverage, which is still in use (reverts r332036)
llvm-svn: 332876
2018-05-21 19:47:00 +00:00
Kamil Rytarowski 434606c8e4 Align ClearShadowForThreadStackAndTLS for NetBSD/i386
Summary:
The static TLS vector for the main thread on NetBSD/i386 can be
unaligned in terms of the shadow granularity. Align the start of it with
Round Down and end of it with Round Up operations for the shadow
granularity shift.

Example static TLS vector ranges on NetBSD/i386:
tls_begin_=0xfbee7244 tls_end_=0xfbee726c.

ClearShadowForThreadStackAndTLS() is called from the Main Thread
bootstrap functions.

This change restores the NetBSD x86 32-bit (i386) support.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46585

llvm-svn: 332792
2018-05-19 01:20:00 +00:00
Kamil Rytarowski 51daee08cd Port msan_test.cc to NetBSD
Summary:
The changes allows building this file on NetBSD, mostly by
disabling the unsupported functions and adding OS-specific
system includes.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46712

llvm-svn: 332791
2018-05-19 01:10:25 +00:00
Evgeniy Stepanov 28f330fd6f [msan] Don't check divisor shadow in fdiv.
Summary:
Floating point division by zero or even undef does not have undefined
behavior and may occur due to optimizations.

Fixes https://bugs.llvm.org/show_bug.cgi?id=37523.

Reviewers: kcc

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D47085

llvm-svn: 332761
2018-05-18 20:19:53 +00:00
Walter Lee 0edca4f505 [asan] Explicitly declare memintrinsics interceptors to have C linkage
This is needed on RTEMS.  Also update a comment.

Differential Revision: https://reviews.llvm.org/D47079

llvm-svn: 332746
2018-05-18 18:01:08 +00:00
Walter Lee 927f4ec1ff [asan] Remove an unsigned compare >= 0
This is not needed after we've forked the Myriad version.  Not to
mention it produces a compiler warning.

Differential Revision: https://reviews.llvm.org/D47054

llvm-svn: 332744
2018-05-18 17:52:12 +00:00
Walter Lee e35f57f023 [asan] Add target-specific files for Myriad RTEMS port
Differential Revision: https://reviews.llvm.org/D46468

llvm-svn: 332691
2018-05-18 04:10:12 +00:00
Walter Lee ce26498e3f [asan] Add support for Myriad RTEMS memory map
The Myriad RTEMS memory system has a few unique aspects that
require support in the ASan run-time.

- A limited amount of memory (currently 512M).

- No virtual memory, no memory protection.

- DRAM starts at address 0x80000000.  Other parts of memory may be
  used for MMIO, etc.

- The second highest address bit is the "cache" bit, and 0x80000000
  and 0x84000000 alias to the same memory.

To support the above, we make the following changes:

- Use a ShadowScale of 5, to reduce shadow memory overhead.

- Adjust some existing macros to remove assumption that the lowest
  memory address is 0.

- add a RawAddr macro that on Myriad strips the cache bit from the
  input address, before using the address for shadow memory (for other
  archs this does nothing).

- We must check that an address is in DRAM range before using it to
  index into shadow memory.

Differential Revision: https://reviews.llvm.org/D46456

llvm-svn: 332690
2018-05-18 04:09:45 +00:00
Walter Lee b134dbb3c3 [sanitizer] Trivial portion of the port to Myriad RTEMS
This commit contains the trivial portion of the port of ASan to
Myriad RTEMS.
- Whitelist platform in sanitizer_platform.h, ubsan_platform.h
- Turn off general interception
- Use memset for FastPoisonShadow
- Define interception wrappers
- Set errno symbol correctly
- Enable ASAN_LOW_MEMORY
- Enable preinit array
- Disable slow unwinding
- Use fuchsia offline symbolizer
- Disable common code for: InitializeShadowMemory, CreateMainThread,
  AsanThread::ThreadStart, StartReportDeadlySignal,
  MaybeReportNonExecRegion.

Differential Revision: https://reviews.llvm.org/D46454

llvm-svn: 332681
2018-05-18 00:43:54 +00:00
Walter Lee c5368c230e [asan] Add a magic shadow value for shadw gap
This gives us something to insert into the shadow gap for systems that
don't have memory protection turned on there (i.e. on Myriad).

Differential Revision: https://reviews.llvm.org/D46457

llvm-svn: 332557
2018-05-16 23:36:01 +00:00
Kostya Serebryany e9c6f06cce [libFuzzer] add an experimental flag -focus_function: libFuzzer will try to focus on inputs that trigger that function
llvm-svn: 332554
2018-05-16 23:26:37 +00:00
Walter Lee 3bf63c9201 [asan] Restore check removed by r332033
Needed by fiber handling code, and possibly other code paths.

Differential Revision: https://reviews.llvm.org/D46981

llvm-svn: 332553
2018-05-16 23:23:56 +00:00
Kostya Kortchinsky e5c9e9f0bd [scudo] Quarantine optimization
Summary:
It turns out that the previous code construct was not optimizing the allocation
and deallocation of batches. The class id was read as a class member (even
though a precomputed one) and nothing else was optimized. By changing the
construct this way, the compiler actually optimizes most of the allocation and
deallocation away to only work with a single class id, which not only saves some
CPU but also some code footprint.

Reviewers: alekseyshl, dvyukov

Reviewed By: dvyukov

Subscribers: dvyukov, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46961

llvm-svn: 332502
2018-05-16 18:12:31 +00:00
Kostya Kortchinsky 561228b2fa [sanitizer] Minor 32-bit primary improvements
Summary:
For the 32-bit TransferBatch:
- `SetFromArray` callers have bounds `count`, so relax the `CHECK` to `DCHECK`;
- same for `Add`;
- mark `CopyToArray` as `const`;
For the 32-bit Primary:
- `{Dea,A}llocateBatch` are only called from places that check `class_id`,
  relax the `CHECK` to `DCHECK`;
- same for `AllocateRegion`;
- remove `GetRegionBeginBySizeClass` that is not used;
- use a local variable for the random shuffle state, so that the compiler can
  use a register instead of reading and writing to the `SizeClassInfo` at every
  iteration;
For the 32-bit local cache:
- pass the count to drain instead of doing a `Min` everytime which is at times
  superfluous.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46657

llvm-svn: 332478
2018-05-16 15:13:26 +00:00
Vitaly Buka 75e01fa00b [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock
Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46793

llvm-svn: 332320
2018-05-15 01:39:25 +00:00
Vitaly Buka 3f1fd7988c [asan] Workaround to avoid hangs in Chromium tests
Summary:
For some reasons on Chromium when we start leak checking we get own pid as 1.
After that we see threads with PPID:0 assuming that thread is dead in infinite
loop.

To resolve particularly this case and possible issues like this, when IsAlive check failed to detect thread status, we need to limit the number of SuspendAllThreads
iterations.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46852

llvm-svn: 332319
2018-05-15 01:39:13 +00:00
Kostya Serebryany 2f7edaeb39 [libFuzzer] deprecate equivalence_server
llvm-svn: 332316
2018-05-15 01:15:47 +00:00
Dean Michael Berris 980d93d0e0 [XRay][profiler] Part 2: XRay Function Call Trie
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements a central data structure for capturing statistics
about XRay instrumented function call stacks. The `FunctionCallTrie`
type does the following things:

*  It keeps track of a shadow function call stack of XRay instrumented
   functions as they are entered (function enter event) and as they are
   exited (function exit event).

*  When a function is entered, the shadow stack contains information
   about the entry TSC, and updates the trie (or prefix tree)
   representing the current function call stack. If we haven't
   encountered this function call before, this creates a unique node for
   the function in this position on the stack. We update the list of
   callees of the parent function as well to reflect this newly found
   path.

*  When a function is exited, we compute statistics (TSC deltas,
   function call count frequency) for the associated function(s) up the
   stack as we unwind to find the matching entry event.

This builds upon the XRay `Allocator` and `Array` types in Part 1 of
this series of patches.

Depends on D45756.

Reviewers: echristo, pelikan, kpw

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D45757

llvm-svn: 332313
2018-05-15 00:42:36 +00:00
Vitaly Buka cdb89b72f5 Revert "[sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock"
Tsan tests fail.

This reverts commit r332268

llvm-svn: 332276
2018-05-14 18:53:13 +00:00
Vitaly Buka e5bd326822 [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock
Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46793

llvm-svn: 332268
2018-05-14 18:03:34 +00:00
Dean Michael Berris 238aa1366e [XRay][compiler-rt] Relocate a DCHECK to the correct location.
Fixes a bad DCHECK where the condition being checked is still valid (for
iterators pointing to sentinels).

Follow-up to D45756.

llvm-svn: 332212
2018-05-14 04:21:12 +00:00
Dean Michael Berris 034b6c1a66 [XRay][compiler-rt] Fixup: Use proper string comparison for DCHECK
Fixes the sanitizer build.

Follow-up to D46574.

llvm-svn: 332211
2018-05-14 04:14:39 +00:00
Dean Michael Berris 845fe8a708 [XRay][compiler-rt] Fixup: Avoid C++11 narrowing in non-x86_64
This should fix non-x86_64 builds where size_t != atomic_uint64_t::Type.

Follow-up to D46574.

llvm-svn: 332209
2018-05-14 03:55:12 +00:00
Dean Michael Berris a2a0e1f353 [XRay][compiler-rt] Support in-memory processing of FDR mode logs
Summary:
This change allows for handling the in-memory data associated with the
FDR mode implementation through the new `__xray_log_process_buffers`
API. With this change, we can now allow users to process the data
in-memory of the process instead of through writing files.

This for example allows users to stream the data of the FDR logging
implementation through network sockets, or through other mechanisms
instead of saving them to local files.

We introduce an FDR-specific flag, for "no_file_flush" which lets the
flushing logic skip opening/writing to files.

This option can be defaulted to `true` when building the compiler-rt
XRay runtime through the `XRAY_FDR_OPTIONS` preprocessor macro.

Reviewers: kpw, echristo, pelikan, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46574

llvm-svn: 332208
2018-05-14 03:35:01 +00:00
Vitaly Buka afce413098 [sanitizer] Replace #if SANITIZER_NETBSD with #if SANITIZER_INTERCEPT___LIBC_MUTEX
llvm-svn: 332190
2018-05-13 05:55:45 +00:00
Vitaly Buka 5e9dd3174d [sanitizer] Fix typo in macro name
llvm-svn: 332171
2018-05-12 03:23:55 +00:00
Walter Lee d8f7b0b5d5 [sanitizer] Allow Fuchsia symbolizer to be reused by Myriad RTEMS
Like Fuchsia, Myriad RTEMS uses an off-line symbolizer -- we just need
a custom backtrace frame string.  Move this definition to
sanitizer_fuchsia.h; the corresponding RTEMS one will be added when we
add sanitizer_rtems.h.

Differential Revision: https://reviews.llvm.org/D46462

llvm-svn: 332157
2018-05-11 23:52:19 +00:00
David Major 868b31bf21 [winasan] Update GetInstructionSize for Win10 1803
In Windows version 1803, the first instruction of ntdll!strchr is:
8a01 mov al,byte ptr [rcx]

This is the only needed change for this version as far as I can tell.

Differential Revision: https://reviews.llvm.org/D46458

llvm-svn: 332095
2018-05-11 14:31:14 +00:00
David Major 49fcf17d5f [ASan] Fix range check in AddrIsInHighShadow
This appears to be a copy/paste artifact from `AddrIsInHighMem`. It was caught by Firefox's jit-tests on Win64.

Differential Revision: https://reviews.llvm.org/D46291

llvm-svn: 332092
2018-05-11 14:23:39 +00:00
Kostya Serebryany 6a6e690d24 [libFuzzer] refactor the implementation of -print_coverage
llvm-svn: 332073
2018-05-11 01:17:52 +00:00
Kamil Rytarowski 65e9f1f2c9 Enable SANITIZER_INTERCEPTOR_HOOKS for NetBSD
Summary:
This feature is required for proper libFuzzer support.

Adding SI_NETBSD to the list of OSes, fixes breakage in several
libFuzzer tests. It has been debugged with aid from kcc@.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46729

llvm-svn: 332051
2018-05-10 22:09:24 +00:00
Kuba Mracek 2903a9bb02 [tsan] Add debugging API to retrieve the "external tag" from reports
Differential Revision: https://reviews.llvm.org/D46661

llvm-svn: 332048
2018-05-10 21:46:00 +00:00
Walter Lee ca804caed0 [asan] Enable memtrinsics interception for RTEMS
Replace decltype(memcpy) with decltype(__asan_memcpy) because memcpy
has not been defined in any headers on RTEMS.  Similarly for memmove
and memset.

Differential Revision: https://reviews.llvm.org/D46625

llvm-svn: 332047
2018-05-10 21:45:13 +00:00
Walter Lee 603c7ae232 [sanitizer] Port fast stack unwinder to sparcv8
Differential Revision: https://reviews.llvm.org/D46469

llvm-svn: 332046
2018-05-10 21:40:16 +00:00
Kostya Serebryany d80e821646 [libFuzzer] remove the dump_coverage flag, it hasn't been working with the inline sanitizer coverage anyway
llvm-svn: 332036
2018-05-10 20:24:39 +00:00
Kostya Serebryany 59bf1e8b36 [libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal is weaker than the SanitizerCoverage
llvm-svn: 332034
2018-05-10 20:12:15 +00:00
Walter Lee 3bde962255 [asan] Initialize fake stack during thread init
If detect-stack-use-after-return is on, initialize fake stack during
AsanThread::Init(), rather than lazily.  This is required on Myriad.
From kcc: "There used to be a reason why this was done lazily, but I
don't remember if we still have that reason."  Tested on x86.

Differential Revision: https://reviews.llvm.org/D46626

llvm-svn: 332033
2018-05-10 20:09:03 +00:00
Kostya Serebryany f489e2bfef [libFuzzer] Experimental data flow tracer for fuzz targets.
Summary:
Experimental data flow tracer for fuzz targets.
Allows to tell which bytes of the input affect which functions of the fuzz target.

We previously attempted to use DFSan directly in the libFuzzer process,
and that didn't work nicely.
Now we will try to collect the data flow information for the seed corpus
in a separate process (using this tracer), and then use it in the regular libFuzzer runs.

Reviewers: morehouse, pcc, Dor1s

Reviewed By: morehouse, Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46666

llvm-svn: 332029
2018-05-10 19:59:01 +00:00
Kamil Rytarowski b8238db4c4 Register NetBSD/i386 in asan_mapping.h
Summary:
Introduce kNetBSD_ShadowOffset32 and document
NetBSD/i386 (hosted on amd64 kernel) process virtual
address space ranges.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46708

llvm-svn: 332027
2018-05-10 19:51:40 +00:00
Kamil Rytarowski aee7b0f209 wrong usages of sem_open in the libFuzzer
Summary:
Fixed two non-standard usages of sem_open in the libFuzzer library and
one NetBSD-related modification with test script.


  - The return value to indicate error should be SEM_FAILED instead of
    (void *)-1 (please refer to "RETURN VALUE" section in this [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | page ]]). Actually, SEM_FAILED != (void *)-1 holds in NetBSD.

  - The SharedMemoryRegion::SemName function should return name
    starting with slash. Because the behaviour of name which does not
    start with slash is unspecified as the [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | "DESCRIPTION" section ]] specified:

> If name does not begin with the <slash> character, the effect is implementation-defined.

  - The length of name is limited to 14 in NetBSD, it is suggested to
    reduce the length of equivalence server name in the test script.

Patch by: Yang Zheng

Reviewers: vitalybuka, krytarowski, kcc

Reviewed By: kcc

Subscribers: kcc, #sanitizers, llvm-commits, joerg

Differential Revision: https://reviews.llvm.org/D46622

llvm-svn: 332003
2018-05-10 17:31:06 +00:00
Vitaly Buka 327f5f3a92 [sanitizer] Attempt to fix strace_test.cc on ppc64le
llvm-svn: 331964
2018-05-10 08:16:23 +00:00
Vitaly Buka ebbc754717 [sanitizer] Fix typo in comment
llvm-svn: 331956
2018-05-10 04:21:41 +00:00
Vitaly Buka 5dcc94c177 [sanitizer] Renamed local variable
llvm-svn: 331955
2018-05-10 04:16:44 +00:00
Vitaly Buka edad575a35 [sanitizer] Fix compilation after invalid rebase
llvm-svn: 331954
2018-05-10 04:10:09 +00:00
Vitaly Buka e0c6eadef7 [sanitizer] Don't miss threads by ThreadSuspender
Summary:
Enumerating /proc/<pid>/task/ dir Linux may stop if thread is dead. In this case
we miss some alive threads and can report false memory leaks.
To solve this issue we repeat enumeration if the last thread is dead.
Do detect dead threads same way as proc_task_readdir we use
/proc/<pid>/task/<tid>/status.

Similarly it also ends enumeration of if proc_fill_cache fails, but in this case
Linux sets inode to 1 (Bad block).

And just in case re-list threads if we had to call internal_getdents more than
twice or result takes more than half of the buffer.

Reviewers: eugenis, dvyukov, glider

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46517

llvm-svn: 331953
2018-05-10 04:02:59 +00:00
Vitaly Buka c779388c7b [sanitizer] Use all available rounded up capacity
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46617

llvm-svn: 331940
2018-05-09 23:31:05 +00:00
Vitaly Buka ed43f18b55 [lsan] Report unsuspended threads
Summary:
Leak checker needs to suspend all process threads. If we have some running
thread in registry but not suspended we can have false leak report. So we will
report this case here for future debugging.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46663

llvm-svn: 331936
2018-05-09 23:02:14 +00:00
Vitaly Buka cf50425b77 [sanitizer] Fix argument type and remove unneeded vector resize
llvm-svn: 331927
2018-05-09 22:03:52 +00:00
Petr Hosek 1608ca6d64 [CMake] Build shared version of runtimes for Fuchsia
Fuchsia is no longer treated as UNIX which means we need to explicitly
enable building of shared versions of runtimes.

Differential Revision: https://reviews.llvm.org/D46609

llvm-svn: 331922
2018-05-09 21:24:06 +00:00
Vitaly Buka 34794a9669 [sanitizer] Use tid_t in ThreadLister
llvm-svn: 331921
2018-05-09 21:21:26 +00:00
Vitaly Buka d3e55bf7fc [sanitizer] Cleanup sorting functions
llvm-svn: 331915
2018-05-09 20:42:11 +00:00
Kostya Serebryany 73634e4037 [dfsan] add one more sanitizer-coverage hook to the whitelist
llvm-svn: 331908
2018-05-09 18:35:09 +00:00
Kostya Kortchinsky cb7912cc0f [sanitizer] Correct 64-bit atomic_store on 32-bit "other" platforms
Summary:
I think there might be something to optimize in `atomic_store`.
Currently, if everything goes well (and we have a different new value), we
always iterate 3 times.
For example, `with a = 0`, `oldval = a`, `newval = 42`, we get:
```
oldval = 0, newval = 42, curval = 0  
oldval = 0, newval = 42, curval = 42 
oldval = 42, newval = 42, curval = 42
```
and then it breaks.

Unless I am not seeing something, I don't see a point to the third iteration.
If the current value is the one we want, we should just break.
This means that 2 iterations (with a different newval) should be sufficient to
achieve what we want.

Reviewers: dvyukov, alekseyshl

Reviewed By: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46597

llvm-svn: 331890
2018-05-09 16:20:52 +00:00
Sid Manning ed3065f7a1 Add basic compiler-rt builtins support for hexagon.
Differential Revision: https://reviews.llvm.org/D46364

llvm-svn: 331881
2018-05-09 14:44:54 +00:00
Vitaly Buka fb663789d3 [sanitizer] Remove unneeded blank lines
llvm-svn: 331831
2018-05-09 00:44:26 +00:00
Vitaly Buka 0590a5bd4b [sanitizer] Update .clang-format in compiler-rt
Historically style is Google, but we never used AllowShortIfStatementsOnASingleLine.

llvm-svn: 331829
2018-05-09 00:41:53 +00:00
Matt Morehouse 14cf71a3a5 [libFuzzer] Guard symbolization with try-lock.
Summary:
When out-of-memory or timeout occurs, threads can be stopped during
symbolization, thereby causing a deadlock when the OOM/TO handlers
attempt symbolization.  We avoid this deadlock by skipping symbolization
if another thread is symbolizing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46605

llvm-svn: 331825
2018-05-08 23:45:05 +00:00
Vitaly Buka e21fbe5af2 [sanitizer] Read file to InternalMmapVectorNoCtor
llvm-svn: 331791
2018-05-08 18:35:11 +00:00
Vitaly Buka 6b13684d15 [sanitizer] Close fd on ReadFromFile error
llvm-svn: 331789
2018-05-08 18:32:53 +00:00
Vitaly Buka 7381f26b45 [sanitizer] Fix InternalMmapVectorNoCtor reserve and resize
Remap on reserve of more than the current size.
Don't remap on downsize.

llvm-svn: 331784
2018-05-08 17:59:44 +00:00
Walter Lee 47bc8f1239 [asan] Fix bug introduced by r331647
unmap_shadow_on_exit was inadvertently flipped for non-RTEMS.

llvm-svn: 331737
2018-05-08 05:58:57 +00:00
Vitaly Buka 4e76d7fcb6 [sanitizer] Add InternalMmapVector::swap
llvm-svn: 331736
2018-05-08 04:57:08 +00:00
Vitaly Buka 4199dbd633 [sanitizer] Fix error checking in ThreadLister
llvm-svn: 331735
2018-05-08 04:56:57 +00:00
Igor Kudrin 47db4e6e5b Revert "[sanitizer] Be more accurate when calculating the previous instruction address on ARM."
This reverts commit r331626 because it causes build bot failures:
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/5069

Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 331734
2018-05-08 04:39:00 +00:00
Dean Michael Berris afdaaba272 [XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC)
This change adds some assembler comments to facilitate analysis with
llvm-mca. In particular, we're interested in identifying and later
optimising (reducing) the cost of the key functions in the XRay
implementation using both static analysis (with llvm-mca, etc.) and
dynamic analysis (perf profiling, etc.) of microbenchmarks.

llvm-svn: 331711
2018-05-08 01:57:51 +00:00
Kostya Serebryany 356e584d91 [dfsan] update the abilist for the sanitizer coverage; also remove the fuzz target from this list
llvm-svn: 331710
2018-05-08 01:52:40 +00:00
Vitaly Buka e2953dcbf7 [sanitizer] Simplify ThreadLister interface
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46516

llvm-svn: 331701
2018-05-07 23:29:48 +00:00
Kostya Kortchinsky 440d76c559 [sanitizer] s/TestOnlyInit/Init for the allocator ByteMap (NFC)
Summary:
The `TestOnlyInit` function of `{Flat,TwoLevel}ByteMap` seems to be a misnomer
since the function is used outside of tests as well, namely in
`SizeClassAllocator32::Init`. Rename it to `Init` and update the callers.

Reviewers: alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46408

llvm-svn: 331662
2018-05-07 19:02:19 +00:00
Walter Lee 7b24aea150 [asan] Port asan_malloc_linux.cc to RTEMS
We reuse the allocation interceptors as is.  RTEMS doesn't support
dlsyms.  However, it needs to handle memory allocation requests before
the ASan run-time has been initialized.  We use the dlsym alloc pool
for this purpose, and we increase its size to 4k to support this
usage.

Differential Revision: https://reviews.llvm.org/D46465

llvm-svn: 331649
2018-05-07 16:39:09 +00:00
Walter Lee 5db43c66b7 [sanitizer] On RTEMS, avoid recursion when reporting Mmap failure
Differential Revision: https://reviews.llvm.org/D46463

llvm-svn: 331648
2018-05-07 16:38:45 +00:00
Walter Lee 483eb8640c [asan] Set flags appropriately for RTEMS
Disable both unmap_shadow_on_exit and protect_shadow_gap.

Differential Revision: https://reviews.llvm.org/D46461

llvm-svn: 331647
2018-05-07 16:38:20 +00:00
Walter Lee 29306b9f62 [sanitizer] Add definitions for Myriad RTEMS platform
Introduce two definitions to be used by the Myriad RTEMS port of the
ASan run-time: SANITIZER_MYRIAD2 for the platform and SANITIZER_RTEMS
for the OS.  We expect to use SANITIZER_MYRIAD2 to guard the portion
of the port corresponding to Myriad's unique memory map, and
SANITIZER_RTEMS for most of the rest.

Differential Revision: https://reviews.llvm.org/D46453

llvm-svn: 331646
2018-05-07 16:37:55 +00:00
Igor Kudrin 61de8a5aef [sanitizer] Be more accurate when calculating the previous instruction address on ARM.
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 331626
2018-05-07 10:07:22 +00:00
Vitaly Buka fac095caba [sanitizer] Add operator== and operator!= for InternalMmapVectorNoCtor
llvm-svn: 331619
2018-05-07 06:14:12 +00:00
Vitaly Buka 2a20955169 [sanitizer] Replace InternalScopedBuffer with InternalMmapVector
llvm-svn: 331618
2018-05-07 05:56:36 +00:00
Vitaly Buka 44f55509d7 [sanitizer] Remove reserving constructor from InternalMmapVector
llvm-svn: 331617
2018-05-07 05:56:24 +00:00
Vitaly Buka db4241a046 [sanitizer] Implement InternalScopedBuffer with InternalMmapVector
llvm-svn: 331616
2018-05-07 05:56:12 +00:00
Vitaly Buka 20f49662f6 [sanitizer] Make InternalScopedBuffer::size() behavior similar to vector.
llvm-svn: 331612
2018-05-07 01:08:13 +00:00
Kamil Rytarowski bbe9e2a13c Remove dead sanitizer_procmaps_freebsd.cc
This file has been obsoleted by sanitizer_procmaps_bsd.cc.

llvm-svn: 331581
2018-05-05 03:57:07 +00:00
David Major af79e30ad5 Comment fix.
Test commit!

llvm-svn: 331523
2018-05-04 14:41:49 +00:00
Dean Michael Berris 47e6b8c2c7 [XRay][compiler-rt] Support string-based config for Basic mode.
Summary:
This addresses http://llvm.org/PR36790.

This change allows the XRay Basic Mode implementation to use the
string-based initialization routine provided through
`__xray_log_init_mode(...)`. In the process, we've also deprecated some
flags defined for the `XRAY_OPTIONS` environment variable.

We then introduce another environment variable that can control the XRay
Basic Mode implementation through `XRAY_BASIC_OPTIONS`.

We also rename files from `xray_inmemory_log` to `xray_basic_logging` to
be more in line with the mode implementation.

Depends on D46174.

Reviewers: echristo, kpw, pelikan, eizan

Reviewed By: kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46246

llvm-svn: 331507
2018-05-04 06:27:53 +00:00
Dean Michael Berris 5c389ace7d [XRay][compiler-rt] Support string-based config for FDR mode
Summary:
In this chage we add support for the string-based configuration
mechanism for configuring FDR mode.

We deprecate most of the `xray_fdr_log_*` flags that are set with the
`XRAY_OPTIONS` environment variable. Instead we make the FDR
implementation take defaults from the `XRAY_FDR_OPTIONS` environment
variable, and use the flags defined in `xray_fdr_flags.{h,cc,inc}` for
the options we support.

This change addresses http://llvm.org/PR36790.

Depends on D46173.

Reviewers: eizan, pelikan, kpw, echristo

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D46174

llvm-svn: 331506
2018-05-04 06:13:35 +00:00
Dean Michael Berris f0a6927932 [XRay][compiler-rt+docs] Introduce __xray_log_init_mode(...).
Summary:
This addresses http://llvm.org/PR36790.

The change Deprecates a number of functions and types in
`include/xray/xray_log_interface.h` to recommend using string-based
configuration of XRay through the __xray_log_init_mode(...) function. In
particular, this deprecates the following:

- `__xray_set_log_impl(...)` -- users should instead use the
`__xray_log_register_mode(...)` and `__xray_log_select_mode(...)` APIs.

- `__xray_log_init(...)` -- users should instead use the
`__xray_log_init_mode(...)` function, which also requires using the
`__xray_log_register_mode(...)` and `__xray_log_select_mode(...)`
functionality.

- `__xray::FDRLoggingOptions` -- in following patches, we'll be
migrating the FDR logging implementations (and tests) to use the
string-based configuration. In later stages we'll remove the
`__xray::FDRLoggingOptions` type, and ask users to migrate to using the
string-based configuration mechanism instead.

- `__xray::BasicLoggingOptions` -- same as `__xray::FDRLoggingOptions`,
we'll be removing this type later and instead rely exclusively on the
string-based configuration API.

We also update the documentation to reflect the new advice and remove
some of the deprecated notes.

Reviewers: eizan, kpw, echristo, pelikan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46173

llvm-svn: 331503
2018-05-04 06:01:12 +00:00
Kostya Kortchinsky 3a9245481f [sanitizer] Remove unused 32-bit allocator TransferBatch parameter
Summary:
NFC. Remove an unused parameter in
`SizeClassAllocator32::TransferBatch::SetFromArray`, and thus get rid of the
compilation warning.

Reviewers: alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46397

llvm-svn: 331477
2018-05-03 19:14:07 +00:00
Petr Hosek 716d9949f6 [sanitizer] Fix Fuchsia ReadBinaryName not to crash when uninitialized
If the sanitizer runtime is loaded in a binary that doesn't really
support it, then __sanitizer_startup_hook will never have been
called to initialize StoredArgv. This case can't be supported, but
its failure mode shouldn't be to crash in sanitizer_common internals.

Patch By: mcgrathr

Differential Revision: https://reviews.llvm.org/D46344

llvm-svn: 331382
2018-05-02 18:08:47 +00:00
Matt Morehouse 7764a04af0 [libFuzzer] Don't short-circuit from CrashCallback.
Short-circuiting causes tests to fail on Mac since libFuzzer crashes
rather than exiting with an error code when an unexpected signal
happens.

llvm-svn: 331324
2018-05-02 02:55:28 +00:00
Matt Morehouse 52fd169035 [libFuzzer] Report at most one crash per input.
Summary:
Fixes https://github.com/google/sanitizers/issues/788/, a deadlock
caused by multiple crashes happening at the same time.  Before printing
a crash report, we now test and set an atomic flag.  If the flag was
already set, the crash handler returns immediately.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46277

llvm-svn: 331310
2018-05-01 21:01:53 +00:00
Roman Lebedev e8e95b5b01 [compiler-rt][X86][AMD][Bulldozer] Fix Bulldozer Model 2 detection.
Summary:
The compiler-rt side of D46314

I have discovered an issue by accident.
```
$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          21
Model:               2
Model name:          AMD FX(tm)-8350 Eight-Core Processor
Stepping:            0
CPU MHz:             3584.018
CPU max MHz:         4000.0000
CPU min MHz:         1400.0000
BogoMIPS:            8027.22
Virtualization:      AMD-V
L1d cache:           16K
L1i cache:           64K
L2 cache:            2048K
L3 cache:            8192K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb cpb hw_pstate vmmcall bmi1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold
```
So this is model-2 bulldozer AMD CPU.

GCC agrees:
```
$ echo | gcc -E - -march=native -###
<...>
 /usr/lib/gcc/x86_64-linux-gnu/7/cc1 -E -quiet -imultiarch x86_64-linux-gnu - "-march=bdver2" -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -mno-movbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mlwp -mfma -mfma4 -mxop -mbmi -mno-sgx -mno-bmi2 -mtbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mf16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mxsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps -mno-avx5124vnniw -mno-clwb -mno-mwaitx -mno-clzero -mno-pku -mno-rdpid --param "l1-cache-size=16" --param "l1-cache-line-size=64" --param "l2-cache-size=2048" "-mtune=bdver2"
<...>
```

But clang does not: (look for `bdver1`)
```
$ echo | clang -E - -march=native -###
clang version 7.0.0- (trunk)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
 "/usr/lib/llvm-7/bin/clang" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-E" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "-" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-fuse-init-array" "-target-cpu" "bdver1" "-target-feature" "+sse2" "-target-feature" "+cx16" "-target-feature" "+sahf" "-target-feature" "+tbm" "-target-feature" "-avx512ifma" "-target-feature" "-sha" "-target-feature" "-gfni" "-target-feature" "+fma4" "-target-feature" "-vpclmulqdq" "-target-feature" "+prfchw" "-target-feature" "-bmi2" "-target-feature" "-cldemote" "-target-feature" "-fsgsbase" "-target-feature" "-xsavec" "-target-feature" "+popcnt" "-target-feature" "+aes" "-target-feature" "-avx512bitalg" "-target-feature" "-xsaves" "-target-feature" "-avx512er" "-target-feature" "-avx512vnni" "-target-feature" "-avx512vpopcntdq" "-target-feature" "-clwb" "-target-feature" "-avx512f" "-target-feature" "-clzero" "-target-feature" "-pku" "-target-feature" "+mmx" "-target-feature" "+lwp" "-target-feature" "-rdpid" "-target-feature" "+xop" "-target-feature" "-rdseed" "-target-feature" "-waitpkg" "-target-feature" "-ibt" "-target-feature" "+sse4a" "-target-feature" "-avx512bw" "-target-feature" "-clflushopt" "-target-feature" "+xsave" "-target-feature" "-avx512vbmi2" "-target-feature" "-avx512vl" "-target-feature" "-avx512cd" "-target-feature" "+avx" "-target-feature" "-vaes" "-target-feature" "-rtm" "-target-feature" "+fma" "-target-feature" "+bmi" "-target-feature" "-rdrnd" "-target-feature" "-mwaitx" "-target-feature" "+sse4.1" "-target-feature" "+sse4.2" "-target-feature" "-avx2" "-target-feature" "-wbnoinvd" "-target-feature" "+sse" "-target-feature" "+lzcnt" "-target-feature" "+pclmul" "-target-feature" "-prefetchwt1" "-target-feature" "+f16c" "-target-feature" "+ssse3" "-target-feature" "-sgx" "-target-feature" "-shstk" "-target-feature" "+cmov" "-target-feature" "-avx512vbmi" "-target-feature" "-movbe" "-target-feature" "-xsaveopt" "-target-feature" "-avx512dq" "-target-feature" "-adx" "-target-feature" "-avx512pf" "-target-feature" "+sse3" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/usr/lib/llvm-7/lib/clang/7.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/lib/llvm-7/lib/clang/7.0.0/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/build/llvm-build-Clang-release" "-ferror-limit" "19" "-fmessage-length" "271" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "-" "-x" "c" "-"
```

So clang, unlike gcc, considers this to be `bdver1`.

After some digging, i've come across `getAMDProcessorTypeAndSubtype()` in `Host.cpp`.
I have added the following debug printf after the call to that function in `sys::getHostCPUName()`:
```
errs() << "Family " << Family << " Model " << Model << " Type " << Type "\n";
```
Which produced:
```
Family 21 Model 2 Type 5
```
Which matches the `lscpu` output.

As it was pointed in the review by @craig.topper:
>>! In D46314#1084123, @craig.topper wrote:
> I dont' think this is right. Here is what I found on wikipedia. https://en.wikipedia.org/wiki/List_of_AMD_CPU_microarchitectures.
>
> AMD Bulldozer Family 15h - the successor of 10h/K10. Bulldozer is designed for processors in the 10 to 220W category, implementing XOP, FMA4 and CVT16 instruction sets. Orochi was the first design which implemented it. For Bulldozer, CPUID model numbers are 00h and 01h.
> AMD Piledriver Family 15h (2nd-gen) - successor to Bulldozer. CPUID model numbers are 02h (earliest "Vishera" Piledrivers) and 10h-1Fh.
> AMD Steamroller Family 15h (3rd-gen) - third-generation Bulldozer derived core. CPUID model numbers are 30h-3Fh.
> AMD Excavator Family 15h (4th-gen) - fourth-generation Bulldozer derived core. CPUID model numbers are 60h-6Fh, later updated revisions have model numbers 70h-7Fh.
>
>
> So there's a weird exception where model 2 should go with 0x10-0x1f.

Though It does not help that the code can't be tested at the moment.
With this logical change, the `bdver2` is properly detected.
```
$ echo | /build/llvm-build-Clang-release/bin/clang -E - -march=native -###
clang version 7.0.0 (trunk 331249) (llvm/trunk 331256)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /build/llvm-build-Clang-release/bin
 "/build/llvm-build-Clang-release/bin/clang-7" "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-E" "-disable-free" "-main-file-name" "-" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-fuse-init-array" "-target-cpu" "bdver2" "-target-feature" "+sse2" "-target-feature" "+cx16" "-target-feature" "+sahf" "-target-feature" "+tbm" "-target-feature" "-avx512ifma" "-target-feature" "-sha" "-target-feature" "-gfni" "-target-feature" "+fma4" "-target-feature" "-vpclmulqdq" "-target-feature" "+prfchw" "-target-feature" "-bmi2" "-target-feature" "-cldemote" "-target-feature" "-fsgsbase" "-target-feature" "-xsavec" "-target-feature" "+popcnt" "-target-feature" "+aes" "-target-feature" "-avx512bitalg" "-target-feature" "-movdiri" "-target-feature" "-xsaves" "-target-feature" "-avx512er" "-target-feature" "-avx512vnni" "-target-feature" "-avx512vpopcntdq" "-target-feature" "-clwb" "-target-feature" "-avx512f" "-target-feature" "-clzero" "-target-feature" "-pku" "-target-feature" "+mmx" "-target-feature" "+lwp" "-target-feature" "-rdpid" "-target-feature" "+xop" "-target-feature" "-rdseed" "-target-feature" "-waitpkg" "-target-feature" "-movdir64b" "-target-feature" "-ibt" "-target-feature" "+sse4a" "-target-feature" "-avx512bw" "-target-feature" "-clflushopt" "-target-feature" "+xsave" "-target-feature" "-avx512vbmi2" "-target-feature" "-avx512vl" "-target-feature" "-avx512cd" "-target-feature" "+avx" "-target-feature" "-vaes" "-target-feature" "-rtm" "-target-feature" "+fma" "-target-feature" "+bmi" "-target-feature" "-rdrnd" "-target-feature" "-mwaitx" "-target-feature" "+sse4.1" "-target-feature" "+sse4.2" "-target-feature" "-avx2" "-target-feature" "-wbnoinvd" "-target-feature" "+sse" "-target-feature" "+lzcnt" "-target-feature" "+pclmul" "-target-feature" "-prefetchwt1" "-target-feature" "+f16c" "-target-feature" "+ssse3" "-target-feature" "-sgx" "-target-feature" "-shstk" "-target-feature" "+cmov" "-target-feature" "-avx512vbmi" "-target-feature" "-movbe" "-target-feature" "-xsaveopt" "-target-feature" "-avx512dq" "-target-feature" "-adx" "-target-feature" "-avx512pf" "-target-feature" "+sse3" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/build/llvm-build-Clang-release/lib/clang/7.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/build/llvm-build-Clang-release/lib/clang/7.0.0/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/build/llvm-build-Clang-release" "-ferror-limit" "19" "-fmessage-length" "271" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "-" "-x" "c" "-"
```

Reviewers: craig.topper, asbirlea, rnk, GGanesh, andreadb

Reviewed By: craig.topper

Subscribers: sdardis, dberris, aprantl, arichardson, JDevlieghere, #sanitizers, llvm-commits, cfe-commits, craig.topper

Differential Revision: https://reviews.llvm.org/D46323

llvm-svn: 331295
2018-05-01 18:40:15 +00:00
Dmitry Vyukov 6c33998ce1 tsan: disable trace switching after multithreaded fork
The problem is reported in:
https://github.com/google/sanitizers/issues/945

We already disable as much as possible after multithreaded fork,
trace switching is last place that can hang due to basic
operations (memory accesses, function calls).
Disable it too.

llvm-svn: 331163
2018-04-30 07:28:45 +00:00
Dean Michael Berris 26e81209ef [XRay][profiler] Part 1: XRay Allocator and Array Implementations
Summary:
This change is part of the larger XRay Profiling Mode effort.

Here we implement an arena allocator, for fixed sized buffers used in a
segmented array implementation. This change adds the segmented array
data structure, which relies on the allocator to provide and maintain
the storage for the segmented array.

Key features of the `Allocator` type:

*  It uses cache-aligned blocks, intended to host the actual data. These
   blocks are cache-line-size multiples of contiguous bytes.

*  The `Allocator` has a maximum memory budget, set at construction
   time. This allows us to cap the amount of data each specific
   `Allocator` instance is responsible for.

*  Upon destruction, the `Allocator` will clean up the storage it's
   used, handing it back to the internal allocator used in
   sanitizer_common.

Key features of the `Array` type:

*  Each segmented array is always backed by an `Allocator`, which is
   either user-provided or uses a global allocator.

*  When an `Array` grows, it grows by appending a segment that's
   fixed-sized. The size of each segment is computed by the number of
   elements of type `T` that can fit into cache line multiples.

*  An `Array` does not return memory to the `Allocator`, but it can keep
   track of the current number of "live" objects it stores.

*  When an `Array` is destroyed, it will not return memory to the
   `Allocator`. Users should clean up the `Allocator` independently of
   the `Array`.

*  The `Array` type keeps a freelist of the chunks it's used before, so
   that trimming and growing will re-use previously allocated chunks.

These basic data structures are used by the XRay Profiling Mode
implementation to implement efficient and cache-aware storage for data
that's typically read-and-write heavy for tracking latency information.
We're relying on the cache line characteristics of the architecture to
provide us good data isolation and cache friendliness, when we're
performing operations like searching for elements and/or updating data
hosted in these cache lines.

Reviewers: echristo, pelikan, kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D45756

llvm-svn: 331141
2018-04-29 13:46:30 +00:00
Dmitry Vyukov 21dc68fe7b tsan: improve "destroy of a locked mutex" reports
1. Allow to suppress by current stack.
We generally allow to suppress by all main stacks.
Current is probably the stack one wants to use to
suppress such reports.

2. Fix last lock stack restoration.
We trimmed shadow value by storing it in u32.
This magically worked for the test that provoked
the report on the main thread. But this breaks
for locks in any other threads.

llvm-svn: 331023
2018-04-27 08:59:35 +00:00
Reid Kleckner 824eb0e6a0 [asan] Align __asan_global_start so that it works with LLD
Otherwise LLD will not align the .ASAN$GA section start, and
&__asan_globals + 1 will not be the start of the next real ASan global
metadata in .ASAN$GL.

We discovered this issue when attempting to use LLD on Windows in
Chromium: https://crbug.com/837090

llvm-svn: 330990
2018-04-26 20:46:50 +00:00
Kostya Kortchinsky d8803d3d92 [scudo] Adding an interface function to print allocator stats
Summary:
This adds `__scudo_print_stats` as an interface function to display the Primary
and Secondary allocator statistics for Scudo.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46016

llvm-svn: 330857
2018-04-25 18:52:29 +00:00
Petar Jovanovic 1dd3584ec0 [mips] Implement GetWriteFlag() for mips
The read/write flag is set by manually decoding the instruction that caused
the exception. It is implemented this way because the cause register which
contains the needed flag was removed from the signal context structure which
the user handler receives from the kernel.

Patch by Milos Stojanovic.

Differential Revision: https://reviews.llvm.org/D45768

llvm-svn: 330840
2018-04-25 16:21:00 +00:00
Richard Smith c1aaf8ebb7 Remove 'noexcept's that do not match between header and source file.
This code is ill-formed, but under -fno-exceptions compilers generally accept it (at least, prior to C++17). This allows this code to be built by Clang in C++17 mode.

llvm-svn: 330765
2018-04-24 20:33:37 +00:00
Kostya Kortchinsky efe5afbc3d [sanitizer] More dead code removal
Summary:
The following functions are only used in tests: `SetEnv`,
`SanitizerSetThreadName`, `SanitizerGetThreadName`. I don't think they are
going to be used in the future, and I propose to get rid of them, and associated
tests and include.

Reviewers: alekseyshl, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: dvyukov, vitalybuka, kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45838

llvm-svn: 330724
2018-04-24 14:58:10 +00:00
Rainer Orth 482cef6c3d The OpenBSD UBsan port introduced two typos that broke the Solaris sanitizer build.
The following patch restores it, will shortly commit as obvious.

Differential Revision: https://reviews.llvm.org/D46006

llvm-svn: 330712
2018-04-24 13:00:24 +00:00
Dan Liew dfd5a90a99 [LibFuzzer] Tweak `MutationDispatcher::Mutate_CopyPart` mutation.
It doesn't make sense to non-deterministically choose between
`CopyPart(..)` and `InsertPart(..)` when it is known that
`InsertPart(..)` will fail.

This upstream's a change from JFS solver's fork of LibFuzzer.

Differential Revision: https://reviews.llvm.org/D45693

llvm-svn: 330687
2018-04-24 06:31:09 +00:00
Alex Shlyapnikov cf30f6425a [HWASan] Use dynamic shadow memory on Android only.
There're issues with IFUNC support on other platforms.

Differential Revision: https://reviews.llvm.org/D45847

llvm-svn: 330666
2018-04-24 00:17:02 +00:00
Matt Morehouse f66221c6ec [libFuzzer] Add feature to not use AFL's deferred forkserver.
A small but substantial minority of libFuzzer-based fuzzers run code that
does not play well with fork in global constructors or LLVMFuzzerInitialize.
This patch allows these fuzzers to use afl_driver by allowing them to
opt-out of using AFL's deferred forkserver which deferres calling fork until
after this code.

Patch By: metzman

Differential Revision: https://reviews.llvm.org/D45744

llvm-svn: 330652
2018-04-23 21:36:21 +00:00
Martin Liska d6af13e737 Change kAllocatorSpace for powerpc64.
Fixes issue: https://github.com/google/sanitizers/issues/933

Differential Revision: https://reviews.llvm.org/D45950

llvm-svn: 330650
2018-04-23 21:13:40 +00:00
Alex Shlyapnikov 88b7be0d13 [Sanitizer] Internal Printf string width + left-justify.
Summary:
Example:

  Printf("%-5s", "123");

should yield:

  '123  '

In case Printf's requested string field width is larger than the string
argument length, the resulting string should be padded up to the requested
width.

For the simplicity sake, implementing left-justified (right padding) only.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45906

llvm-svn: 330643
2018-04-23 20:30:50 +00:00
Alex Shlyapnikov d03fb0e3e0 [HWASan] Add files missing in r330624
llvm-svn: 330628
2018-04-23 19:05:12 +00:00
Alex Shlyapnikov 79866cad6f [HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt).
Retire the fixed shadow memory mapping to avoid conflicts with default
process memory mapping (currently manifests on Android).

Tests on AArch64 show <1% performance loss and code size increase,
making it possible to use dynamic shadow memory by default.

Keep the fixed shadow memory mapping around to be able to run
performance comparison tests later.

Re-commiting D45847 with fixed shadow for x86-64.

llvm-svn: 330624
2018-04-23 18:19:23 +00:00
Reid Kleckner 061fcd586d Revert r330474 - "[HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt)."
This commit causes internal errors with ld.bfd 2.24. My guess is that
the ifunc usage in this commit is causing problems. This is the default
system linker on Trusty Tahr, which is from 2014. I claim it's still in
our support window. Maybe we will decide to drop support for it, but
let's get the bots green while we do the investigation and have that
discussion.

Discovered here: https://crbug.com/835864

llvm-svn: 330619
2018-04-23 17:26:33 +00:00
Reid Kleckner 963aba3452 Fix clang-cl warnings in compiler-rt
The profile library was missing some includes and was erroneously using
ftruncate. WinASan was using `= {0}` to initialize structs, which
creates -Wmissing-field-initializers and -Wmissing-braces warnings with
clang. Use `= {}` instead, since this is C++.

llvm-svn: 330616
2018-04-23 17:05:47 +00:00
Kostya Kortchinsky 5a8bdc81d6 [scudo] Read ARM feature bits using Fuchsia APIs.
Summary:
    Fuchsia uses zx_system_get_features in lieu of getauxval.
    
    Use this call when checking for CRC32 support.

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: delcypher, llvm-commits, #sanitizers, kristof.beyls, chrib

Differential Revision: https://reviews.llvm.org/D45896

llvm-svn: 330598
2018-04-23 14:53:13 +00:00
Alex Shlyapnikov 8368bd82c3 Revert "[Sanitizer] Internal Printf string precision argument + padding."
This reverts commit r330458.

There are existing code using string precision as 'max len', need more
work.

llvm-svn: 330476
2018-04-20 20:24:02 +00:00
Alex Shlyapnikov c3ec1643f1 [HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt).
Summary:
Retire the fixed shadow memory mapping to avoid conflicts with default
process memory mapping (currently manifests on Android).

Tests on AArch64 show <1% performance loss and code size increase,
making it possible to use dynamic shadow memory by default.

For the simplicity and unifirmity sake, use dynamic shadow memory mapping
with base address accessed via ifunc resolver on all supported platforms.

Keep the fixed shadow memory mapping around to be able to run
performance comparison tests later.

Complementing D45840.

Reviewers: eugenis

Subscribers: srhines, kubamracek, dberris, mgorny, kristof.beyls, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45847

llvm-svn: 330474
2018-04-20 20:03:57 +00:00
Alex Shlyapnikov 5bb2345985 [Sanitizer] Internal Printf string precision argument + padding.
Summary:
Example:
  Printf("%.*s", 5, "123");
should yield:
  '123  '

In case Printf's requested string precision is larger than the string
argument, the resulting string should be padded up to the requested
precision.

For the simplicity sake, implementing right padding only.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45844

llvm-svn: 330458
2018-04-20 18:03:10 +00:00
Hans Wennborg 2473183c15 Revert r330376 "[sanitizer] Generalize atomic_uint8_t, atomic_uint16_t, ... into a template. NFC."
This broke the Windows build, see e.g. http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10130

> Differential Revision: https://reviews.llvm.org/D44246

llvm-svn: 330395
2018-04-20 07:34:59 +00:00
Dan Liew b7a3bc99c9 [LibFuzzer] Report when custom counters are available.
This upstreams a feature from the JFS solver's fork of LibFuzzer.

Differential Revision: https://reviews.llvm.org/D45675

llvm-svn: 330391
2018-04-20 06:46:19 +00:00
Dan Liew 25d0c65ff3 [LibFuzzer] Try to unbreak the `FuzzerMutate.ShuffleBytes1` unit test.
This test is failing on my Linux box. Just increasing the number of
iterations works around this. The divergence is likely due to
our reliance on `std::shuffle()` which is not guaranteed to have
the same behaviour across platforms.

This is a strong argument for us to implement our own shuffle
function to avoid divergence in behaviour across platforms.

Differential Revision: https://reviews.llvm.org/D45767

llvm-svn: 330390
2018-04-20 06:46:14 +00:00
Dan Liew c8879c989e [LibFuzzer] Unbreak the `trace-malloc-unbalanced.test` when using Python 3.
The `unbalanced_allocs.py` script uses Python 2 print statement
and `iteritems()`. Running `2to3` over the script fixes these.

Differential Revision: https://reviews.llvm.org/D45765

llvm-svn: 330389
2018-04-20 06:46:09 +00:00
Petr Hosek dd097445f0 [Fuzzer] Add a missing header in Fuchsia implementation
This is needed for fd_set.

Differential Revision: https://reviews.llvm.org/D45852

llvm-svn: 330383
2018-04-20 00:41:06 +00:00
Kuba Mracek 1707fa3374 [sanitizer] Generalize atomic_uint8_t, atomic_uint16_t, ... into a template. NFC.
Differential Revision: https://reviews.llvm.org/D44246

llvm-svn: 330376
2018-04-19 23:00:43 +00:00
Kostya Kortchinsky 46eab8d9fe [sanitizer] Minor refactor of some ReservedAddressRange functions
Summary:
Some of the functions had spurious conditional statements and checks, and some
intermediary variables that I feel made the code more complicated than it needs
to be. Also, when unmapping the whole range, the range size would be 0, but
the base was set to the address of the end of the range, which sounds prone to
error. I think nulling out the base in this scenario is a better way to go.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45775

llvm-svn: 330355
2018-04-19 18:38:15 +00:00
Petr Hosek 7a31c7ad23 [Fuzzer] Make InterruptHandler non-blocking for Fuchsia
The initial naive approach to simulate SIGINT on Fuchsia was to getchar
and look for ETX. This caused the InterruptHandler thread to lock stdin,
preventing musl's exit() from being able to close the stdio descriptors
and complete. This change uses select() instead.

Patch By: aarongreen

Differential Revision: https://reviews.llvm.org/D45636

llvm-svn: 330328
2018-04-19 14:01:46 +00:00
Dmitry Vyukov 4132b849e5 tsan: fix compiler warnings
vmaSize is uptr, so we need to print it with %zd.

llvm-svn: 330312
2018-04-19 07:42:08 +00:00
Alex Shlyapnikov 02f7841e4a [HWASan] Add "N" suffix to generic __hwasan_load/store.
Summary:
"N" suffix is added by the instrumentation and interface functions
are expected to be exported from the library as __hwasan_loadN* and
__hwasan_storeN*.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45739

llvm-svn: 330297
2018-04-18 22:05:18 +00:00
Kostya Kortchinsky eaeb64a814 [sanitizer] Minor refactor of ThreadDescriptorSize
Summary:
While I was sifting through dead code findings, I stumbled on this function.

First, for `__s390__` it always returned 0 for the 1st call, which doesn't seem
right. 2nd call & beyond would return the correct value though.
Then it duplicated the `atomic_store` multiple times, sometimes with a `if`,
sometimes without. Finally it used a capitalized variable name starting with `k`
which indicates a constant, and it is not.

So:
- rename the static global variable;
- change the atomic functions to their relaxed version;
- move the store to the end, and make sure we return `val` all the time.

Reviewers: alekseyshl, eugenis, koriakin

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45725

llvm-svn: 330268
2018-04-18 15:30:08 +00:00
Keith Wyss 557890126a [XRay][compiler-rt] Add noop patch functions for unsupported arches.
Summary:
Typed event patching is implemented for x86-64, but functions must
be defined for other arches.

Reviewers: dberris, pelikan

Subscribers: nemanjai, javed.absar, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45749

llvm-svn: 330231
2018-04-17 23:19:23 +00:00
Keith Wyss adb092e0ae Implement trampoline and handler for typed xray event tracing.
Summary:
Compiler-rt support first before defining the __xray_typedevent() lowering in
llvm. I'm looking for some early feedback before I touch much more code.

Reviewers: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43668

llvm-svn: 330218
2018-04-17 21:28:53 +00:00
Kostya Kortchinsky 54764ca235 [sanitizer] Remove low-hanging-fruit dead code
Summary:
Going through the dead code findings, the code removed in this CL appears to be
pretty straightforward to remove, and seems to be some leftover from previous
refactors.

Reviewers: alekseyshl, eugenis

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45704

llvm-svn: 330190
2018-04-17 14:54:50 +00:00
Kostya Kortchinsky 25eae5c879 [sanitizer] Fix OSX build failure post D45457
Summary:
It looks like OSX's UBSan needs a "NoHooks" version of
`RTSanitizerCommonSymbolizer` to work build properly.

Subscribers: kubamracek, mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45696

llvm-svn: 330146
2018-04-16 18:17:22 +00:00