Commit Graph

10308 Commits

Author SHA1 Message Date
Craig Topper 383fe5c866 sanitizer: Use pre-computed size of struct ustat for Linux
<sys/ustat.h> has been removed from glibc 2.28 by:

commit cf2478d53ad7071e84c724a986b56fe17f4f4ca7
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Sun Mar 18 11:28:59 2018 +0800

Deprecate ustat syscall interface
This patch uses pre-computed size of struct ustat for Linux to fix

https://bugs.llvm.org/show_bug.cgi?id=37418

Patch by H.J. Lu.

Differential Revision: https://reviews.llvm.org/D47281

llvm-svn: 333213
2018-05-24 17:59:47 +00:00
Igor Kudrin 13f8a77d79 [tsan] Fix a failure in test java_race_pc on AArch64 after r333071
llvm-svn: 333180
2018-05-24 12:26:20 +00:00
Kostya Serebryany 4d53b744ca [libFuzzer] DataFlow tracer now tags a subset of the input. A separate script merges traces from the subsets
llvm-svn: 333149
2018-05-24 01:43:48 +00:00
Kostya Serebryany 500ca8713c [libFuzzer] fix two off-by-ones (!!) in the data flow tracer
llvm-svn: 333142
2018-05-23 23:55:54 +00:00
Jake Ehrlich 767d92e3fb [fuchsia] Add line buffering in RawWrite
This change causes RawWrite to buffer upto 128 bytes or until
a line is reached. This helps group calls into more readable
lines.

llvm-svn: 333136
2018-05-23 22:27:12 +00:00
Kostya Serebryany 4925392897 [libFuzzer] change the output format for the DataFlow tracer
llvm-svn: 333122
2018-05-23 20:57:11 +00:00
Kostya Serebryany 28fe54fac8 [libFuzzer] add a stress test for the DataFlow tracer
llvm-svn: 333119
2018-05-23 20:23:33 +00:00
Max Moroz fe97441c2c [libFuzzer] Don't complain about lack of interesting inputs when -runs=0.
Summary:
The most common usecase for -runs=0 is for generating code coverage
over some corpus. Coverage reports based on sancov are about to be deprecated,
which means some external coverage solution will be used, e.g. Clang source
based code coverage, which does not use any sancov instrumentations and thus
libFuzzer would consider any input to be not interesting in that case.

Reviewers: kcc

Reviewed By: kcc

Subscribers: alex, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47271

llvm-svn: 333116
2018-05-23 19:42:30 +00:00
Kostya Kortchinsky 08c630457a [sanitizer] Define UINTPTR_MAX
Summary:
`sanitizer_internal_defs.h` didn't have this define, which will be useful in
an upcoming CL.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47270

llvm-svn: 333109
2018-05-23 18:13:21 +00:00
Kamil Rytarowski 5b1b594f64 Fix internal_mmap() on 32-bit NetBSD platforms
There is need to use internal_syscall64() instead of internal_syscall_ptr().
The offset argument of type off_t is always 64-bit.

llvm-svn: 333075
2018-05-23 10:37:00 +00:00
Igor Kudrin c0099f9be7 [sanitizer] Reland "Be more accurate when calculating the previous instruction address on ARM."
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 333071
2018-05-23 09:18:10 +00:00
Kostya Serebryany 7f5bf83f53 [libFuzzer] modify -print_corpus_stats to print whether the input reaches the focus function
llvm-svn: 333048
2018-05-23 01:42:53 +00:00
Petr Hosek aa6d13e501 [CMake] Support builtins as Clang default rtlib in compiler-rt
Use compiler-rt builtins when selected as default Clang rtlib and avoid
explicitly passing -rtlib= flag to avoid the "argument unused during
compilation" warning.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47115

llvm-svn: 333037
2018-05-22 22:58:48 +00:00
Evgeniy Stepanov 45f8986b80 [asan] Use dynamic allocator space address on Android/AArch64.
Summary:
We need one library to support all of 39, 42 and 48 bit VMAs, and
there is no common address that works for all of them.

Reviewers: kcc, alekseyshl, javed.absar

Subscribers: rengolin, srhines, kubamracek, kristof.beyls, llvm-commits, cryptoad

Differential Revision: https://reviews.llvm.org/D47160

llvm-svn: 333025
2018-05-22 20:44:45 +00:00
Petr Hosek 1ffaa48d4a [CMake] Support libc++ as Clang default stdlib in compiler-rt
Use libc++ when selected as default Clang stdlib and avoid checking
C++ compiler when using the in-tree version of libc++.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47100

llvm-svn: 333010
2018-05-22 18:33:27 +00:00
Petr Hosek 4ca5af0721 [CMake] Silence unused variable warning in compiler check
This is breaking the compiler check.

llvm-svn: 332936
2018-05-22 02:53:32 +00:00
Petr Hosek f6dda7cb44 [CMake] Use a different source depending on C++ support
When using system C++ library, assume we have a working C++ compiler and
try to compile a complete C++ program. When using in tree C++ library,
only check the C compiler since the C++ library likely won't have been
built yet at time of running the check.

Differential Revision: https://reviews.llvm.org/D47169

llvm-svn: 332924
2018-05-22 01:01:56 +00:00
Walter Lee ead3b3487b [asan] Make GetCurrentThread RTEMS-friendly
On RTEMS, system and user code all live in a single binary and address
space. There is no clean separation, and instrumented code may
execute before the ASan run-time is initialized (or after it has been
destroyed).

Currently, GetCurrentThread() may crash if it's called before ASan
run-time is initialized. Make it return nullptr instead.

Similarly, fix __asan_handle_no_return so that it gives up rather than
try something that may crash.

Differential Revision: https://reviews.llvm.org/D46459

llvm-svn: 332888
2018-05-21 20:43:36 +00:00
Kostya Serebryany 69c2b71a51 [libFuzzer] reinstate -dump_coverage, which is still in use (reverts r332036)
llvm-svn: 332876
2018-05-21 19:47:00 +00:00
Petr Hosek 702d073642 Revert "[CMake] Reland "Make simple source used for CMake checks a C file""
This reverts commit rCRT332679 which doesn't quite work and will be
replaced by D47100 and D47115 which is a cleaner solution.

llvm-svn: 332871
2018-05-21 19:32:43 +00:00
Kamil Rytarowski 434606c8e4 Align ClearShadowForThreadStackAndTLS for NetBSD/i386
Summary:
The static TLS vector for the main thread on NetBSD/i386 can be
unaligned in terms of the shadow granularity. Align the start of it with
Round Down and end of it with Round Up operations for the shadow
granularity shift.

Example static TLS vector ranges on NetBSD/i386:
tls_begin_=0xfbee7244 tls_end_=0xfbee726c.

ClearShadowForThreadStackAndTLS() is called from the Main Thread
bootstrap functions.

This change restores the NetBSD x86 32-bit (i386) support.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46585

llvm-svn: 332792
2018-05-19 01:20:00 +00:00
Kamil Rytarowski 51daee08cd Port msan_test.cc to NetBSD
Summary:
The changes allows building this file on NetBSD, mostly by
disabling the unsupported functions and adding OS-specific
system includes.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46712

llvm-svn: 332791
2018-05-19 01:10:25 +00:00
Kamil Rytarowski c7fe74576f Do not link with -ldl in NetBSD/ASan tests
Summary:
The dlopen(3) features on NetBSD are in libc.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46713

llvm-svn: 332790
2018-05-19 01:02:51 +00:00
Evgeniy Stepanov 28f330fd6f [msan] Don't check divisor shadow in fdiv.
Summary:
Floating point division by zero or even undef does not have undefined
behavior and may occur due to optimizations.

Fixes https://bugs.llvm.org/show_bug.cgi?id=37523.

Reviewers: kcc

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D47085

llvm-svn: 332761
2018-05-18 20:19:53 +00:00
Reid Kleckner 4f9516d617 Revert r332683 & r332684 relating to compiler runtime checks
r332683 passes flags to the compiler without first checking if they are
supported.

llvm-svn: 332754
2018-05-18 18:44:37 +00:00
Walter Lee 0edca4f505 [asan] Explicitly declare memintrinsics interceptors to have C linkage
This is needed on RTEMS.  Also update a comment.

Differential Revision: https://reviews.llvm.org/D47079

llvm-svn: 332746
2018-05-18 18:01:08 +00:00
Walter Lee 927f4ec1ff [asan] Remove an unsigned compare >= 0
This is not needed after we've forked the Myriad version.  Not to
mention it produces a compiler warning.

Differential Revision: https://reviews.llvm.org/D47054

llvm-svn: 332744
2018-05-18 17:52:12 +00:00
Walter Lee e35f57f023 [asan] Add target-specific files for Myriad RTEMS port
Differential Revision: https://reviews.llvm.org/D46468

llvm-svn: 332691
2018-05-18 04:10:12 +00:00
Walter Lee ce26498e3f [asan] Add support for Myriad RTEMS memory map
The Myriad RTEMS memory system has a few unique aspects that
require support in the ASan run-time.

- A limited amount of memory (currently 512M).

- No virtual memory, no memory protection.

- DRAM starts at address 0x80000000.  Other parts of memory may be
  used for MMIO, etc.

- The second highest address bit is the "cache" bit, and 0x80000000
  and 0x84000000 alias to the same memory.

To support the above, we make the following changes:

- Use a ShadowScale of 5, to reduce shadow memory overhead.

- Adjust some existing macros to remove assumption that the lowest
  memory address is 0.

- add a RawAddr macro that on Myriad strips the cache bit from the
  input address, before using the address for shadow memory (for other
  archs this does nothing).

- We must check that an address is in DRAM range before using it to
  index into shadow memory.

Differential Revision: https://reviews.llvm.org/D46456

llvm-svn: 332690
2018-05-18 04:09:45 +00:00
Petr Hosek 83061cc4aa [CMake] Use <UNIX|WINDOWS>_COMMAND with separate_arguments
NATIVE_COMMAND is only available since CMake 3.9.

llvm-svn: 332684
2018-05-18 01:20:47 +00:00
Petr Hosek 8b5fe57d3e [CMake] Detect the compiler runtime and standard library
Rather then requiring the user to specify runtime the compiler
runtime and C++ standard library, or trying to guess them which is
error-prone, use auto-detection by parsing the compiler link output.

Differential Revision: https://reviews.llvm.org/D46857

llvm-svn: 332683
2018-05-18 01:10:09 +00:00
Walter Lee b134dbb3c3 [sanitizer] Trivial portion of the port to Myriad RTEMS
This commit contains the trivial portion of the port of ASan to
Myriad RTEMS.
- Whitelist platform in sanitizer_platform.h, ubsan_platform.h
- Turn off general interception
- Use memset for FastPoisonShadow
- Define interception wrappers
- Set errno symbol correctly
- Enable ASAN_LOW_MEMORY
- Enable preinit array
- Disable slow unwinding
- Use fuchsia offline symbolizer
- Disable common code for: InitializeShadowMemory, CreateMainThread,
  AsanThread::ThreadStart, StartReportDeadlySignal,
  MaybeReportNonExecRegion.

Differential Revision: https://reviews.llvm.org/D46454

llvm-svn: 332681
2018-05-18 00:43:54 +00:00
Petr Hosek 3eb71831e5 [CMake] Reland "Make simple source used for CMake checks a C file"
The source being compiled is plain C, but using .cc extension forces it
to be compiled as C++ which requires a working C++ compiler including
C++ library which may not be the case when we're building compiler-rt
together with libcxx as part of runtimes build.

Differential Revision: https://reviews.llvm.org/D47031

llvm-svn: 332679
2018-05-17 23:50:35 +00:00
Petr Hosek 5a668a054b [CMake] Cleanup find_compiler_rt_library function [NFC]
Rename the output variable and remove the unnecessary set call.

llvm-svn: 332661
2018-05-17 20:35:55 +00:00
Alex Lorenz a54e218d4a [builtins][macos] bump up the the macOS version min in os_version_check tests
This ensures that the tests link with the latest OS.

llvm-svn: 332647
2018-05-17 18:41:38 +00:00
Kostya Serebryany a52d30b2bc [libFuzzer] rename a test from .c to .cpp
llvm-svn: 332558
2018-05-16 23:38:53 +00:00
Walter Lee c5368c230e [asan] Add a magic shadow value for shadw gap
This gives us something to insert into the shadow gap for systems that
don't have memory protection turned on there (i.e. on Myriad).

Differential Revision: https://reviews.llvm.org/D46457

llvm-svn: 332557
2018-05-16 23:36:01 +00:00
Kostya Serebryany e9c6f06cce [libFuzzer] add an experimental flag -focus_function: libFuzzer will try to focus on inputs that trigger that function
llvm-svn: 332554
2018-05-16 23:26:37 +00:00
Walter Lee 3bf63c9201 [asan] Restore check removed by r332033
Needed by fiber handling code, and possibly other code paths.

Differential Revision: https://reviews.llvm.org/D46981

llvm-svn: 332553
2018-05-16 23:23:56 +00:00
Max Moroz d63396097d [libFuzzer] add a symbolic execution puzzle (difficult for today's libFuzzer).
Summary:
This can be solved just in seconds with KLEE. Current libFuzzer
is able to satistfy 101 constraints out of 410 constraints presented during
the first hour of running with -use_value_profile=1 and -max_len=20.
During the next 3 hours, libFuzzer is able to generate ~50 NEW inputs,
bot none of those solve any new constraint.
During the next 20 hours, it didn't find any NEW inputs.

This test might be interesting for experimenting with the data flow tracing
approach started in https://reviews.llvm.org/D46666.

For the solution with KLEE and other information, see
https://github.com/Dor1s/codegate2017-quals-angrybird

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46924

llvm-svn: 332507
2018-05-16 18:19:30 +00:00
Kostya Kortchinsky e5c9e9f0bd [scudo] Quarantine optimization
Summary:
It turns out that the previous code construct was not optimizing the allocation
and deallocation of batches. The class id was read as a class member (even
though a precomputed one) and nothing else was optimized. By changing the
construct this way, the compiler actually optimizes most of the allocation and
deallocation away to only work with a single class id, which not only saves some
CPU but also some code footprint.

Reviewers: alekseyshl, dvyukov

Reviewed By: dvyukov

Subscribers: dvyukov, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46961

llvm-svn: 332502
2018-05-16 18:12:31 +00:00
Kostya Kortchinsky 561228b2fa [sanitizer] Minor 32-bit primary improvements
Summary:
For the 32-bit TransferBatch:
- `SetFromArray` callers have bounds `count`, so relax the `CHECK` to `DCHECK`;
- same for `Add`;
- mark `CopyToArray` as `const`;
For the 32-bit Primary:
- `{Dea,A}llocateBatch` are only called from places that check `class_id`,
  relax the `CHECK` to `DCHECK`;
- same for `AllocateRegion`;
- remove `GetRegionBeginBySizeClass` that is not used;
- use a local variable for the random shuffle state, so that the compiler can
  use a register instead of reading and writing to the `SizeClassInfo` at every
  iteration;
For the 32-bit local cache:
- pass the count to drain instead of doing a `Min` everytime which is at times
  superfluous.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46657

llvm-svn: 332478
2018-05-16 15:13:26 +00:00
Vitaly Buka 75e01fa00b [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock
Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46793

llvm-svn: 332320
2018-05-15 01:39:25 +00:00
Vitaly Buka 3f1fd7988c [asan] Workaround to avoid hangs in Chromium tests
Summary:
For some reasons on Chromium when we start leak checking we get own pid as 1.
After that we see threads with PPID:0 assuming that thread is dead in infinite
loop.

To resolve particularly this case and possible issues like this, when IsAlive check failed to detect thread status, we need to limit the number of SuspendAllThreads
iterations.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46852

llvm-svn: 332319
2018-05-15 01:39:13 +00:00
Kostya Serebryany 2f7edaeb39 [libFuzzer] deprecate equivalence_server
llvm-svn: 332316
2018-05-15 01:15:47 +00:00
Dean Michael Berris 980d93d0e0 [XRay][profiler] Part 2: XRay Function Call Trie
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements a central data structure for capturing statistics
about XRay instrumented function call stacks. The `FunctionCallTrie`
type does the following things:

*  It keeps track of a shadow function call stack of XRay instrumented
   functions as they are entered (function enter event) and as they are
   exited (function exit event).

*  When a function is entered, the shadow stack contains information
   about the entry TSC, and updates the trie (or prefix tree)
   representing the current function call stack. If we haven't
   encountered this function call before, this creates a unique node for
   the function in this position on the stack. We update the list of
   callees of the parent function as well to reflect this newly found
   path.

*  When a function is exited, we compute statistics (TSC deltas,
   function call count frequency) for the associated function(s) up the
   stack as we unwind to find the matching entry event.

This builds upon the XRay `Allocator` and `Array` types in Part 1 of
this series of patches.

Depends on D45756.

Reviewers: echristo, pelikan, kpw

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D45757

llvm-svn: 332313
2018-05-15 00:42:36 +00:00
Vitaly Buka cdb89b72f5 Revert "[sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock"
Tsan tests fail.

This reverts commit r332268

llvm-svn: 332276
2018-05-14 18:53:13 +00:00
Vitaly Buka e5bd326822 [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock
Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46793

llvm-svn: 332268
2018-05-14 18:03:34 +00:00
Dan Liew fe7250c2fa Fix another case where libstdc++ is being inappropriately requested (see
r328775) for all platforms.

Given that this is the second occurance of this problem it seemed worth
fixing this problem in a more generic way. r328775 has been reverted and
now a substitution `%linux_static_libstdcplusplus` has been provided.
This substitution expands to Clang driver arguments to use a static
libstdc++ on Linux and on all other platforms it expands to nothing.

The `asan/TestCases/throw_invoke_test.cc` and
`test/tsan/static_init6.cc` test cases now use this substitution.

rdar://problem/39948818

Differential Revision: https://reviews.llvm.org/D46401

llvm-svn: 332254
2018-05-14 15:48:49 +00:00
Dean Michael Berris 238aa1366e [XRay][compiler-rt] Relocate a DCHECK to the correct location.
Fixes a bad DCHECK where the condition being checked is still valid (for
iterators pointing to sentinels).

Follow-up to D45756.

llvm-svn: 332212
2018-05-14 04:21:12 +00:00
Dean Michael Berris 034b6c1a66 [XRay][compiler-rt] Fixup: Use proper string comparison for DCHECK
Fixes the sanitizer build.

Follow-up to D46574.

llvm-svn: 332211
2018-05-14 04:14:39 +00:00
Dean Michael Berris 845fe8a708 [XRay][compiler-rt] Fixup: Avoid C++11 narrowing in non-x86_64
This should fix non-x86_64 builds where size_t != atomic_uint64_t::Type.

Follow-up to D46574.

llvm-svn: 332209
2018-05-14 03:55:12 +00:00
Dean Michael Berris a2a0e1f353 [XRay][compiler-rt] Support in-memory processing of FDR mode logs
Summary:
This change allows for handling the in-memory data associated with the
FDR mode implementation through the new `__xray_log_process_buffers`
API. With this change, we can now allow users to process the data
in-memory of the process instead of through writing files.

This for example allows users to stream the data of the FDR logging
implementation through network sockets, or through other mechanisms
instead of saving them to local files.

We introduce an FDR-specific flag, for "no_file_flush" which lets the
flushing logic skip opening/writing to files.

This option can be defaulted to `true` when building the compiler-rt
XRay runtime through the `XRAY_FDR_OPTIONS` preprocessor macro.

Reviewers: kpw, echristo, pelikan, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46574

llvm-svn: 332208
2018-05-14 03:35:01 +00:00
Vitaly Buka afce413098 [sanitizer] Replace #if SANITIZER_NETBSD with #if SANITIZER_INTERCEPT___LIBC_MUTEX
llvm-svn: 332190
2018-05-13 05:55:45 +00:00
Vitaly Buka 5e9dd3174d [sanitizer] Fix typo in macro name
llvm-svn: 332171
2018-05-12 03:23:55 +00:00
Walter Lee d8f7b0b5d5 [sanitizer] Allow Fuchsia symbolizer to be reused by Myriad RTEMS
Like Fuchsia, Myriad RTEMS uses an off-line symbolizer -- we just need
a custom backtrace frame string.  Move this definition to
sanitizer_fuchsia.h; the corresponding RTEMS one will be added when we
add sanitizer_rtems.h.

Differential Revision: https://reviews.llvm.org/D46462

llvm-svn: 332157
2018-05-11 23:52:19 +00:00
George Karpenkov 669375814c [libFuzzer] [NFC] Split fuzzer.test into smaller tests
Keeping fuzzer.test together as a gigantic test has no advantages and multiple disadvantages:

 - Worse test parallelization (fuzzer.test is always blocking the test
run on a machine with many cores)
 - Debugging test failures is made more difficult (not clear what fails
from fuzzer.test failing)
 - Makes porting tests to new platforms more difficult (whenever
fuzzer.test fails have to inspect the output to figure out what is going
on, and then restart all of it)
 - Hides dead code (in our case, "Done1000000" FileCheck variable was
never used, DISABLED: not %t-UninstrumentedTest-Uninstrumented was never
compiled, and there was small amount of duplication vs. simple-cmp.test)
 - Makes correspondence between LIT .test files and .cpp files less
straightforward

Differential Revision: https://reviews.llvm.org/D46557

llvm-svn: 332145
2018-05-11 21:29:53 +00:00
George Karpenkov fa57519e31 [libFuzzer] [NFC] Prefix all run lines with %run
This patch does not do anything, but paves the way for future changes,
where %run command will be expanded into a script performing the testing
on device.

Differential Revision: https://reviews.llvm.org/D46553

llvm-svn: 332144
2018-05-11 21:29:51 +00:00
David Major 868b31bf21 [winasan] Update GetInstructionSize for Win10 1803
In Windows version 1803, the first instruction of ntdll!strchr is:
8a01 mov al,byte ptr [rcx]

This is the only needed change for this version as far as I can tell.

Differential Revision: https://reviews.llvm.org/D46458

llvm-svn: 332095
2018-05-11 14:31:14 +00:00
David Major 49fcf17d5f [ASan] Fix range check in AddrIsInHighShadow
This appears to be a copy/paste artifact from `AddrIsInHighMem`. It was caught by Firefox's jit-tests on Win64.

Differential Revision: https://reviews.llvm.org/D46291

llvm-svn: 332092
2018-05-11 14:23:39 +00:00
Kostya Serebryany 6a6e690d24 [libFuzzer] refactor the implementation of -print_coverage
llvm-svn: 332073
2018-05-11 01:17:52 +00:00
Kamil Rytarowski 65e9f1f2c9 Enable SANITIZER_INTERCEPTOR_HOOKS for NetBSD
Summary:
This feature is required for proper libFuzzer support.

Adding SI_NETBSD to the list of OSes, fixes breakage in several
libFuzzer tests. It has been debugged with aid from kcc@.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46729

llvm-svn: 332051
2018-05-10 22:09:24 +00:00
Kuba Mracek 2903a9bb02 [tsan] Add debugging API to retrieve the "external tag" from reports
Differential Revision: https://reviews.llvm.org/D46661

llvm-svn: 332048
2018-05-10 21:46:00 +00:00
Walter Lee ca804caed0 [asan] Enable memtrinsics interception for RTEMS
Replace decltype(memcpy) with decltype(__asan_memcpy) because memcpy
has not been defined in any headers on RTEMS.  Similarly for memmove
and memset.

Differential Revision: https://reviews.llvm.org/D46625

llvm-svn: 332047
2018-05-10 21:45:13 +00:00
Walter Lee 603c7ae232 [sanitizer] Port fast stack unwinder to sparcv8
Differential Revision: https://reviews.llvm.org/D46469

llvm-svn: 332046
2018-05-10 21:40:16 +00:00
Kostya Serebryany 5d95f2782d [libFuzzer] simplify tests, remove one redundant test; NFC
llvm-svn: 332037
2018-05-10 20:37:08 +00:00
Kostya Serebryany d80e821646 [libFuzzer] remove the dump_coverage flag, it hasn't been working with the inline sanitizer coverage anyway
llvm-svn: 332036
2018-05-10 20:24:39 +00:00
Kamil Rytarowski 9788e60736 Setup ORIGIN/NetBSD option in sanitizer tests
Summary:
NetBSD can use the approach that exists in FreeBSD, Linux and SunOS.

Pick the FreeBSD one as marking programs with "-z origin" is useful.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: emaste, fedor.sergeev, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46718

llvm-svn: 332035
2018-05-10 20:13:36 +00:00
Kostya Serebryany 59bf1e8b36 [libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal is weaker than the SanitizerCoverage
llvm-svn: 332034
2018-05-10 20:12:15 +00:00
Walter Lee 3bde962255 [asan] Initialize fake stack during thread init
If detect-stack-use-after-return is on, initialize fake stack during
AsanThread::Init(), rather than lazily.  This is required on Myriad.
From kcc: "There used to be a reason why this was done lazily, but I
don't remember if we still have that reason."  Tested on x86.

Differential Revision: https://reviews.llvm.org/D46626

llvm-svn: 332033
2018-05-10 20:09:03 +00:00
Kostya Serebryany f489e2bfef [libFuzzer] Experimental data flow tracer for fuzz targets.
Summary:
Experimental data flow tracer for fuzz targets.
Allows to tell which bytes of the input affect which functions of the fuzz target.

We previously attempted to use DFSan directly in the libFuzzer process,
and that didn't work nicely.
Now we will try to collect the data flow information for the seed corpus
in a separate process (using this tracer), and then use it in the regular libFuzzer runs.

Reviewers: morehouse, pcc, Dor1s

Reviewed By: morehouse, Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46666

llvm-svn: 332029
2018-05-10 19:59:01 +00:00
Kamil Rytarowski b8238db4c4 Register NetBSD/i386 in asan_mapping.h
Summary:
Introduce kNetBSD_ShadowOffset32 and document
NetBSD/i386 (hosted on amd64 kernel) process virtual
address space ranges.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46708

llvm-svn: 332027
2018-05-10 19:51:40 +00:00
Peter Wu c58988a908 [lsan] Try to fix test failure due to compiler optimization
Summary:
The SanitizerCommon-lsan-x86_64-Linux test failed due to the address of
the very first allocation ending up in the stack through "delete[]".
Workaround this by performing another allocation. The issue was only
present with optimization enabled, the test would pass with -O0.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46650

llvm-svn: 332020
2018-05-10 19:02:32 +00:00
Kamil Rytarowski aee7b0f209 wrong usages of sem_open in the libFuzzer
Summary:
Fixed two non-standard usages of sem_open in the libFuzzer library and
one NetBSD-related modification with test script.


  - The return value to indicate error should be SEM_FAILED instead of
    (void *)-1 (please refer to "RETURN VALUE" section in this [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | page ]]). Actually, SEM_FAILED != (void *)-1 holds in NetBSD.

  - The SharedMemoryRegion::SemName function should return name
    starting with slash. Because the behaviour of name which does not
    start with slash is unspecified as the [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | "DESCRIPTION" section ]] specified:

> If name does not begin with the <slash> character, the effect is implementation-defined.

  - The length of name is limited to 14 in NetBSD, it is suggested to
    reduce the length of equivalence server name in the test script.

Patch by: Yang Zheng

Reviewers: vitalybuka, krytarowski, kcc

Reviewed By: kcc

Subscribers: kcc, #sanitizers, llvm-commits, joerg

Differential Revision: https://reviews.llvm.org/D46622

llvm-svn: 332003
2018-05-10 17:31:06 +00:00
Vitaly Buka 327f5f3a92 [sanitizer] Attempt to fix strace_test.cc on ppc64le
llvm-svn: 331964
2018-05-10 08:16:23 +00:00
Vitaly Buka ebbc754717 [sanitizer] Fix typo in comment
llvm-svn: 331956
2018-05-10 04:21:41 +00:00
Vitaly Buka 5dcc94c177 [sanitizer] Renamed local variable
llvm-svn: 331955
2018-05-10 04:16:44 +00:00
Vitaly Buka edad575a35 [sanitizer] Fix compilation after invalid rebase
llvm-svn: 331954
2018-05-10 04:10:09 +00:00
Vitaly Buka e0c6eadef7 [sanitizer] Don't miss threads by ThreadSuspender
Summary:
Enumerating /proc/<pid>/task/ dir Linux may stop if thread is dead. In this case
we miss some alive threads and can report false memory leaks.
To solve this issue we repeat enumeration if the last thread is dead.
Do detect dead threads same way as proc_task_readdir we use
/proc/<pid>/task/<tid>/status.

Similarly it also ends enumeration of if proc_fill_cache fails, but in this case
Linux sets inode to 1 (Bad block).

And just in case re-list threads if we had to call internal_getdents more than
twice or result takes more than half of the buffer.

Reviewers: eugenis, dvyukov, glider

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46517

llvm-svn: 331953
2018-05-10 04:02:59 +00:00
Kostya Serebryany d790effacb [libFuzzer] add a simple puzzle that is difficult for today's libFuzzer
llvm-svn: 331951
2018-05-10 02:02:41 +00:00
Vitaly Buka c779388c7b [sanitizer] Use all available rounded up capacity
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46617

llvm-svn: 331940
2018-05-09 23:31:05 +00:00
Vitaly Buka ed43f18b55 [lsan] Report unsuspended threads
Summary:
Leak checker needs to suspend all process threads. If we have some running
thread in registry but not suspended we can have false leak report. So we will
report this case here for future debugging.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46663

llvm-svn: 331936
2018-05-09 23:02:14 +00:00
Vitaly Buka cf50425b77 [sanitizer] Fix argument type and remove unneeded vector resize
llvm-svn: 331927
2018-05-09 22:03:52 +00:00
Petr Hosek 1608ca6d64 [CMake] Build shared version of runtimes for Fuchsia
Fuchsia is no longer treated as UNIX which means we need to explicitly
enable building of shared versions of runtimes.

Differential Revision: https://reviews.llvm.org/D46609

llvm-svn: 331922
2018-05-09 21:24:06 +00:00
Vitaly Buka 34794a9669 [sanitizer] Use tid_t in ThreadLister
llvm-svn: 331921
2018-05-09 21:21:26 +00:00
Vitaly Buka d3e55bf7fc [sanitizer] Cleanup sorting functions
llvm-svn: 331915
2018-05-09 20:42:11 +00:00
Kostya Serebryany 73634e4037 [dfsan] add one more sanitizer-coverage hook to the whitelist
llvm-svn: 331908
2018-05-09 18:35:09 +00:00
Matt Morehouse 0a4d213176 [libFuzzer] Disable print-func.test on Darwin.
The try-lock guard change seems to be making this test fail on Mac, but
I haven't been able to reproduce the failure.  Disabling the test on Mac
to fix build bot.

llvm-svn: 331894
2018-05-09 17:05:07 +00:00
Kostya Kortchinsky cb7912cc0f [sanitizer] Correct 64-bit atomic_store on 32-bit "other" platforms
Summary:
I think there might be something to optimize in `atomic_store`.
Currently, if everything goes well (and we have a different new value), we
always iterate 3 times.
For example, `with a = 0`, `oldval = a`, `newval = 42`, we get:
```
oldval = 0, newval = 42, curval = 0  
oldval = 0, newval = 42, curval = 42 
oldval = 42, newval = 42, curval = 42
```
and then it breaks.

Unless I am not seeing something, I don't see a point to the third iteration.
If the current value is the one we want, we should just break.
This means that 2 iterations (with a different newval) should be sufficient to
achieve what we want.

Reviewers: dvyukov, alekseyshl

Reviewed By: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46597

llvm-svn: 331890
2018-05-09 16:20:52 +00:00
Sid Manning ed3065f7a1 Add basic compiler-rt builtins support for hexagon.
Differential Revision: https://reviews.llvm.org/D46364

llvm-svn: 331881
2018-05-09 14:44:54 +00:00
Vitaly Buka fb663789d3 [sanitizer] Remove unneeded blank lines
llvm-svn: 331831
2018-05-09 00:44:26 +00:00
Vitaly Buka 0590a5bd4b [sanitizer] Update .clang-format in compiler-rt
Historically style is Google, but we never used AllowShortIfStatementsOnASingleLine.

llvm-svn: 331829
2018-05-09 00:41:53 +00:00
Matt Morehouse 14cf71a3a5 [libFuzzer] Guard symbolization with try-lock.
Summary:
When out-of-memory or timeout occurs, threads can be stopped during
symbolization, thereby causing a deadlock when the OOM/TO handlers
attempt symbolization.  We avoid this deadlock by skipping symbolization
if another thread is symbolizing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46605

llvm-svn: 331825
2018-05-08 23:45:05 +00:00
Vitaly Buka e21fbe5af2 [sanitizer] Read file to InternalMmapVectorNoCtor
llvm-svn: 331791
2018-05-08 18:35:11 +00:00
Vitaly Buka 6b13684d15 [sanitizer] Close fd on ReadFromFile error
llvm-svn: 331789
2018-05-08 18:32:53 +00:00
Vitaly Buka 7381f26b45 [sanitizer] Fix InternalMmapVectorNoCtor reserve and resize
Remap on reserve of more than the current size.
Don't remap on downsize.

llvm-svn: 331784
2018-05-08 17:59:44 +00:00
Walter Lee 47bc8f1239 [asan] Fix bug introduced by r331647
unmap_shadow_on_exit was inadvertently flipped for non-RTEMS.

llvm-svn: 331737
2018-05-08 05:58:57 +00:00
Vitaly Buka 4e76d7fcb6 [sanitizer] Add InternalMmapVector::swap
llvm-svn: 331736
2018-05-08 04:57:08 +00:00
Vitaly Buka 4199dbd633 [sanitizer] Fix error checking in ThreadLister
llvm-svn: 331735
2018-05-08 04:56:57 +00:00
Igor Kudrin 47db4e6e5b Revert "[sanitizer] Be more accurate when calculating the previous instruction address on ARM."
This reverts commit r331626 because it causes build bot failures:
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/5069

Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 331734
2018-05-08 04:39:00 +00:00
Dean Michael Berris afdaaba272 [XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC)
This change adds some assembler comments to facilitate analysis with
llvm-mca. In particular, we're interested in identifying and later
optimising (reducing) the cost of the key functions in the XRay
implementation using both static analysis (with llvm-mca, etc.) and
dynamic analysis (perf profiling, etc.) of microbenchmarks.

llvm-svn: 331711
2018-05-08 01:57:51 +00:00
Kostya Serebryany 356e584d91 [dfsan] update the abilist for the sanitizer coverage; also remove the fuzz target from this list
llvm-svn: 331710
2018-05-08 01:52:40 +00:00
Vitaly Buka e2953dcbf7 [sanitizer] Simplify ThreadLister interface
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46516

llvm-svn: 331701
2018-05-07 23:29:48 +00:00
Kostya Kortchinsky 440d76c559 [sanitizer] s/TestOnlyInit/Init for the allocator ByteMap (NFC)
Summary:
The `TestOnlyInit` function of `{Flat,TwoLevel}ByteMap` seems to be a misnomer
since the function is used outside of tests as well, namely in
`SizeClassAllocator32::Init`. Rename it to `Init` and update the callers.

Reviewers: alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46408

llvm-svn: 331662
2018-05-07 19:02:19 +00:00
Walter Lee 7b24aea150 [asan] Port asan_malloc_linux.cc to RTEMS
We reuse the allocation interceptors as is.  RTEMS doesn't support
dlsyms.  However, it needs to handle memory allocation requests before
the ASan run-time has been initialized.  We use the dlsym alloc pool
for this purpose, and we increase its size to 4k to support this
usage.

Differential Revision: https://reviews.llvm.org/D46465

llvm-svn: 331649
2018-05-07 16:39:09 +00:00
Walter Lee 5db43c66b7 [sanitizer] On RTEMS, avoid recursion when reporting Mmap failure
Differential Revision: https://reviews.llvm.org/D46463

llvm-svn: 331648
2018-05-07 16:38:45 +00:00
Walter Lee 483eb8640c [asan] Set flags appropriately for RTEMS
Disable both unmap_shadow_on_exit and protect_shadow_gap.

Differential Revision: https://reviews.llvm.org/D46461

llvm-svn: 331647
2018-05-07 16:38:20 +00:00
Walter Lee 29306b9f62 [sanitizer] Add definitions for Myriad RTEMS platform
Introduce two definitions to be used by the Myriad RTEMS port of the
ASan run-time: SANITIZER_MYRIAD2 for the platform and SANITIZER_RTEMS
for the OS.  We expect to use SANITIZER_MYRIAD2 to guard the portion
of the port corresponding to Myriad's unique memory map, and
SANITIZER_RTEMS for most of the rest.

Differential Revision: https://reviews.llvm.org/D46453

llvm-svn: 331646
2018-05-07 16:37:55 +00:00
Igor Kudrin 61de8a5aef [sanitizer] Be more accurate when calculating the previous instruction address on ARM.
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 331626
2018-05-07 10:07:22 +00:00
Vitaly Buka fac095caba [sanitizer] Add operator== and operator!= for InternalMmapVectorNoCtor
llvm-svn: 331619
2018-05-07 06:14:12 +00:00
Vitaly Buka 2a20955169 [sanitizer] Replace InternalScopedBuffer with InternalMmapVector
llvm-svn: 331618
2018-05-07 05:56:36 +00:00
Vitaly Buka 44f55509d7 [sanitizer] Remove reserving constructor from InternalMmapVector
llvm-svn: 331617
2018-05-07 05:56:24 +00:00
Vitaly Buka db4241a046 [sanitizer] Implement InternalScopedBuffer with InternalMmapVector
llvm-svn: 331616
2018-05-07 05:56:12 +00:00
Vitaly Buka 20f49662f6 [sanitizer] Make InternalScopedBuffer::size() behavior similar to vector.
llvm-svn: 331612
2018-05-07 01:08:13 +00:00
Kamil Rytarowski bbe9e2a13c Remove dead sanitizer_procmaps_freebsd.cc
This file has been obsoleted by sanitizer_procmaps_bsd.cc.

llvm-svn: 331581
2018-05-05 03:57:07 +00:00
David Major af79e30ad5 Comment fix.
Test commit!

llvm-svn: 331523
2018-05-04 14:41:49 +00:00
Dean Michael Berris 47e6b8c2c7 [XRay][compiler-rt] Support string-based config for Basic mode.
Summary:
This addresses http://llvm.org/PR36790.

This change allows the XRay Basic Mode implementation to use the
string-based initialization routine provided through
`__xray_log_init_mode(...)`. In the process, we've also deprecated some
flags defined for the `XRAY_OPTIONS` environment variable.

We then introduce another environment variable that can control the XRay
Basic Mode implementation through `XRAY_BASIC_OPTIONS`.

We also rename files from `xray_inmemory_log` to `xray_basic_logging` to
be more in line with the mode implementation.

Depends on D46174.

Reviewers: echristo, kpw, pelikan, eizan

Reviewed By: kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46246

llvm-svn: 331507
2018-05-04 06:27:53 +00:00
Dean Michael Berris 5c389ace7d [XRay][compiler-rt] Support string-based config for FDR mode
Summary:
In this chage we add support for the string-based configuration
mechanism for configuring FDR mode.

We deprecate most of the `xray_fdr_log_*` flags that are set with the
`XRAY_OPTIONS` environment variable. Instead we make the FDR
implementation take defaults from the `XRAY_FDR_OPTIONS` environment
variable, and use the flags defined in `xray_fdr_flags.{h,cc,inc}` for
the options we support.

This change addresses http://llvm.org/PR36790.

Depends on D46173.

Reviewers: eizan, pelikan, kpw, echristo

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D46174

llvm-svn: 331506
2018-05-04 06:13:35 +00:00
Dean Michael Berris f0a6927932 [XRay][compiler-rt+docs] Introduce __xray_log_init_mode(...).
Summary:
This addresses http://llvm.org/PR36790.

The change Deprecates a number of functions and types in
`include/xray/xray_log_interface.h` to recommend using string-based
configuration of XRay through the __xray_log_init_mode(...) function. In
particular, this deprecates the following:

- `__xray_set_log_impl(...)` -- users should instead use the
`__xray_log_register_mode(...)` and `__xray_log_select_mode(...)` APIs.

- `__xray_log_init(...)` -- users should instead use the
`__xray_log_init_mode(...)` function, which also requires using the
`__xray_log_register_mode(...)` and `__xray_log_select_mode(...)`
functionality.

- `__xray::FDRLoggingOptions` -- in following patches, we'll be
migrating the FDR logging implementations (and tests) to use the
string-based configuration. In later stages we'll remove the
`__xray::FDRLoggingOptions` type, and ask users to migrate to using the
string-based configuration mechanism instead.

- `__xray::BasicLoggingOptions` -- same as `__xray::FDRLoggingOptions`,
we'll be removing this type later and instead rely exclusively on the
string-based configuration API.

We also update the documentation to reflect the new advice and remove
some of the deprecated notes.

Reviewers: eizan, kpw, echristo, pelikan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46173

llvm-svn: 331503
2018-05-04 06:01:12 +00:00
Volodymyr Sapsai 3835320617 Revert "Follow-up to r331378. Update tests to allow to use C atomics in C++."
It reverts commit r331484 because it caused test failures

    ThreadSanitizer-x86_64 :: Darwin/gcd-groups-destructor.mm
    ThreadSanitizer-x86_64 :: Darwin/libcxx-shared-ptr-stress.mm
    ThreadSanitizer-x86_64 :: Darwin/xpc-race.mm

Foundation.h transitively includes <atomic>, so we have a case of benign mixing
<stdatomic.h> and <atomic>.

llvm-svn: 331491
2018-05-03 23:06:37 +00:00
Volodymyr Sapsai 6e713d9866 Follow-up to r331378. Update tests to allow to use C atomics in C++.
Reviewers: kubamracek

Reviewed By: kubamracek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D46363

llvm-svn: 331484
2018-05-03 21:30:47 +00:00
Kostya Kortchinsky 3a9245481f [sanitizer] Remove unused 32-bit allocator TransferBatch parameter
Summary:
NFC. Remove an unused parameter in
`SizeClassAllocator32::TransferBatch::SetFromArray`, and thus get rid of the
compilation warning.

Reviewers: alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46397

llvm-svn: 331477
2018-05-03 19:14:07 +00:00
Petr Hosek 716d9949f6 [sanitizer] Fix Fuchsia ReadBinaryName not to crash when uninitialized
If the sanitizer runtime is loaded in a binary that doesn't really
support it, then __sanitizer_startup_hook will never have been
called to initialize StoredArgv. This case can't be supported, but
its failure mode shouldn't be to crash in sanitizer_common internals.

Patch By: mcgrathr

Differential Revision: https://reviews.llvm.org/D46344

llvm-svn: 331382
2018-05-02 18:08:47 +00:00
Matt Morehouse 7764a04af0 [libFuzzer] Don't short-circuit from CrashCallback.
Short-circuiting causes tests to fail on Mac since libFuzzer crashes
rather than exiting with an error code when an unexpected signal
happens.

llvm-svn: 331324
2018-05-02 02:55:28 +00:00
Matt Morehouse 52fd169035 [libFuzzer] Report at most one crash per input.
Summary:
Fixes https://github.com/google/sanitizers/issues/788/, a deadlock
caused by multiple crashes happening at the same time.  Before printing
a crash report, we now test and set an atomic flag.  If the flag was
already set, the crash handler returns immediately.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46277

llvm-svn: 331310
2018-05-01 21:01:53 +00:00
Roman Lebedev e8e95b5b01 [compiler-rt][X86][AMD][Bulldozer] Fix Bulldozer Model 2 detection.
Summary:
The compiler-rt side of D46314

I have discovered an issue by accident.
```
$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          21
Model:               2
Model name:          AMD FX(tm)-8350 Eight-Core Processor
Stepping:            0
CPU MHz:             3584.018
CPU max MHz:         4000.0000
CPU min MHz:         1400.0000
BogoMIPS:            8027.22
Virtualization:      AMD-V
L1d cache:           16K
L1i cache:           64K
L2 cache:            2048K
L3 cache:            8192K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb cpb hw_pstate vmmcall bmi1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold
```
So this is model-2 bulldozer AMD CPU.

GCC agrees:
```
$ echo | gcc -E - -march=native -###
<...>
 /usr/lib/gcc/x86_64-linux-gnu/7/cc1 -E -quiet -imultiarch x86_64-linux-gnu - "-march=bdver2" -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -mno-movbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mlwp -mfma -mfma4 -mxop -mbmi -mno-sgx -mno-bmi2 -mtbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mf16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mxsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps -mno-avx5124vnniw -mno-clwb -mno-mwaitx -mno-clzero -mno-pku -mno-rdpid --param "l1-cache-size=16" --param "l1-cache-line-size=64" --param "l2-cache-size=2048" "-mtune=bdver2"
<...>
```

But clang does not: (look for `bdver1`)
```
$ echo | clang -E - -march=native -###
clang version 7.0.0- (trunk)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
 "/usr/lib/llvm-7/bin/clang" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-E" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "-" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-fuse-init-array" "-target-cpu" "bdver1" "-target-feature" "+sse2" "-target-feature" "+cx16" "-target-feature" "+sahf" "-target-feature" "+tbm" "-target-feature" "-avx512ifma" "-target-feature" "-sha" "-target-feature" "-gfni" "-target-feature" "+fma4" "-target-feature" "-vpclmulqdq" "-target-feature" "+prfchw" "-target-feature" "-bmi2" "-target-feature" "-cldemote" "-target-feature" "-fsgsbase" "-target-feature" "-xsavec" "-target-feature" "+popcnt" "-target-feature" "+aes" "-target-feature" "-avx512bitalg" "-target-feature" "-xsaves" "-target-feature" "-avx512er" "-target-feature" "-avx512vnni" "-target-feature" "-avx512vpopcntdq" "-target-feature" "-clwb" "-target-feature" "-avx512f" "-target-feature" "-clzero" "-target-feature" "-pku" "-target-feature" "+mmx" "-target-feature" "+lwp" "-target-feature" "-rdpid" "-target-feature" "+xop" "-target-feature" "-rdseed" "-target-feature" "-waitpkg" "-target-feature" "-ibt" "-target-feature" "+sse4a" "-target-feature" "-avx512bw" "-target-feature" "-clflushopt" "-target-feature" "+xsave" "-target-feature" "-avx512vbmi2" "-target-feature" "-avx512vl" "-target-feature" "-avx512cd" "-target-feature" "+avx" "-target-feature" "-vaes" "-target-feature" "-rtm" "-target-feature" "+fma" "-target-feature" "+bmi" "-target-feature" "-rdrnd" "-target-feature" "-mwaitx" "-target-feature" "+sse4.1" "-target-feature" "+sse4.2" "-target-feature" "-avx2" "-target-feature" "-wbnoinvd" "-target-feature" "+sse" "-target-feature" "+lzcnt" "-target-feature" "+pclmul" "-target-feature" "-prefetchwt1" "-target-feature" "+f16c" "-target-feature" "+ssse3" "-target-feature" "-sgx" "-target-feature" "-shstk" "-target-feature" "+cmov" "-target-feature" "-avx512vbmi" "-target-feature" "-movbe" "-target-feature" "-xsaveopt" "-target-feature" "-avx512dq" "-target-feature" "-adx" "-target-feature" "-avx512pf" "-target-feature" "+sse3" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/usr/lib/llvm-7/lib/clang/7.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/lib/llvm-7/lib/clang/7.0.0/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/build/llvm-build-Clang-release" "-ferror-limit" "19" "-fmessage-length" "271" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "-" "-x" "c" "-"
```

So clang, unlike gcc, considers this to be `bdver1`.

After some digging, i've come across `getAMDProcessorTypeAndSubtype()` in `Host.cpp`.
I have added the following debug printf after the call to that function in `sys::getHostCPUName()`:
```
errs() << "Family " << Family << " Model " << Model << " Type " << Type "\n";
```
Which produced:
```
Family 21 Model 2 Type 5
```
Which matches the `lscpu` output.

As it was pointed in the review by @craig.topper:
>>! In D46314#1084123, @craig.topper wrote:
> I dont' think this is right. Here is what I found on wikipedia. https://en.wikipedia.org/wiki/List_of_AMD_CPU_microarchitectures.
>
> AMD Bulldozer Family 15h - the successor of 10h/K10. Bulldozer is designed for processors in the 10 to 220W category, implementing XOP, FMA4 and CVT16 instruction sets. Orochi was the first design which implemented it. For Bulldozer, CPUID model numbers are 00h and 01h.
> AMD Piledriver Family 15h (2nd-gen) - successor to Bulldozer. CPUID model numbers are 02h (earliest "Vishera" Piledrivers) and 10h-1Fh.
> AMD Steamroller Family 15h (3rd-gen) - third-generation Bulldozer derived core. CPUID model numbers are 30h-3Fh.
> AMD Excavator Family 15h (4th-gen) - fourth-generation Bulldozer derived core. CPUID model numbers are 60h-6Fh, later updated revisions have model numbers 70h-7Fh.
>
>
> So there's a weird exception where model 2 should go with 0x10-0x1f.

Though It does not help that the code can't be tested at the moment.
With this logical change, the `bdver2` is properly detected.
```
$ echo | /build/llvm-build-Clang-release/bin/clang -E - -march=native -###
clang version 7.0.0 (trunk 331249) (llvm/trunk 331256)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /build/llvm-build-Clang-release/bin
 "/build/llvm-build-Clang-release/bin/clang-7" "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-E" "-disable-free" "-main-file-name" "-" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-fuse-init-array" "-target-cpu" "bdver2" "-target-feature" "+sse2" "-target-feature" "+cx16" "-target-feature" "+sahf" "-target-feature" "+tbm" "-target-feature" "-avx512ifma" "-target-feature" "-sha" "-target-feature" "-gfni" "-target-feature" "+fma4" "-target-feature" "-vpclmulqdq" "-target-feature" "+prfchw" "-target-feature" "-bmi2" "-target-feature" "-cldemote" "-target-feature" "-fsgsbase" "-target-feature" "-xsavec" "-target-feature" "+popcnt" "-target-feature" "+aes" "-target-feature" "-avx512bitalg" "-target-feature" "-movdiri" "-target-feature" "-xsaves" "-target-feature" "-avx512er" "-target-feature" "-avx512vnni" "-target-feature" "-avx512vpopcntdq" "-target-feature" "-clwb" "-target-feature" "-avx512f" "-target-feature" "-clzero" "-target-feature" "-pku" "-target-feature" "+mmx" "-target-feature" "+lwp" "-target-feature" "-rdpid" "-target-feature" "+xop" "-target-feature" "-rdseed" "-target-feature" "-waitpkg" "-target-feature" "-movdir64b" "-target-feature" "-ibt" "-target-feature" "+sse4a" "-target-feature" "-avx512bw" "-target-feature" "-clflushopt" "-target-feature" "+xsave" "-target-feature" "-avx512vbmi2" "-target-feature" "-avx512vl" "-target-feature" "-avx512cd" "-target-feature" "+avx" "-target-feature" "-vaes" "-target-feature" "-rtm" "-target-feature" "+fma" "-target-feature" "+bmi" "-target-feature" "-rdrnd" "-target-feature" "-mwaitx" "-target-feature" "+sse4.1" "-target-feature" "+sse4.2" "-target-feature" "-avx2" "-target-feature" "-wbnoinvd" "-target-feature" "+sse" "-target-feature" "+lzcnt" "-target-feature" "+pclmul" "-target-feature" "-prefetchwt1" "-target-feature" "+f16c" "-target-feature" "+ssse3" "-target-feature" "-sgx" "-target-feature" "-shstk" "-target-feature" "+cmov" "-target-feature" "-avx512vbmi" "-target-feature" "-movbe" "-target-feature" "-xsaveopt" "-target-feature" "-avx512dq" "-target-feature" "-adx" "-target-feature" "-avx512pf" "-target-feature" "+sse3" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/build/llvm-build-Clang-release/lib/clang/7.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/build/llvm-build-Clang-release/lib/clang/7.0.0/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/build/llvm-build-Clang-release" "-ferror-limit" "19" "-fmessage-length" "271" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "-" "-x" "c" "-"
```

Reviewers: craig.topper, asbirlea, rnk, GGanesh, andreadb

Reviewed By: craig.topper

Subscribers: sdardis, dberris, aprantl, arichardson, JDevlieghere, #sanitizers, llvm-commits, cfe-commits, craig.topper

Differential Revision: https://reviews.llvm.org/D46323

llvm-svn: 331295
2018-05-01 18:40:15 +00:00
Dmitry Vyukov 6c33998ce1 tsan: disable trace switching after multithreaded fork
The problem is reported in:
https://github.com/google/sanitizers/issues/945

We already disable as much as possible after multithreaded fork,
trace switching is last place that can hang due to basic
operations (memory accesses, function calls).
Disable it too.

llvm-svn: 331163
2018-04-30 07:28:45 +00:00
Dean Michael Berris 26e81209ef [XRay][profiler] Part 1: XRay Allocator and Array Implementations
Summary:
This change is part of the larger XRay Profiling Mode effort.

Here we implement an arena allocator, for fixed sized buffers used in a
segmented array implementation. This change adds the segmented array
data structure, which relies on the allocator to provide and maintain
the storage for the segmented array.

Key features of the `Allocator` type:

*  It uses cache-aligned blocks, intended to host the actual data. These
   blocks are cache-line-size multiples of contiguous bytes.

*  The `Allocator` has a maximum memory budget, set at construction
   time. This allows us to cap the amount of data each specific
   `Allocator` instance is responsible for.

*  Upon destruction, the `Allocator` will clean up the storage it's
   used, handing it back to the internal allocator used in
   sanitizer_common.

Key features of the `Array` type:

*  Each segmented array is always backed by an `Allocator`, which is
   either user-provided or uses a global allocator.

*  When an `Array` grows, it grows by appending a segment that's
   fixed-sized. The size of each segment is computed by the number of
   elements of type `T` that can fit into cache line multiples.

*  An `Array` does not return memory to the `Allocator`, but it can keep
   track of the current number of "live" objects it stores.

*  When an `Array` is destroyed, it will not return memory to the
   `Allocator`. Users should clean up the `Allocator` independently of
   the `Array`.

*  The `Array` type keeps a freelist of the chunks it's used before, so
   that trimming and growing will re-use previously allocated chunks.

These basic data structures are used by the XRay Profiling Mode
implementation to implement efficient and cache-aware storage for data
that's typically read-and-write heavy for tracking latency information.
We're relying on the cache line characteristics of the architecture to
provide us good data isolation and cache friendliness, when we're
performing operations like searching for elements and/or updating data
hosted in these cache lines.

Reviewers: echristo, pelikan, kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D45756

llvm-svn: 331141
2018-04-29 13:46:30 +00:00
Dmitry Vyukov 21dc68fe7b tsan: improve "destroy of a locked mutex" reports
1. Allow to suppress by current stack.
We generally allow to suppress by all main stacks.
Current is probably the stack one wants to use to
suppress such reports.

2. Fix last lock stack restoration.
We trimmed shadow value by storing it in u32.
This magically worked for the test that provoked
the report on the main thread. But this breaks
for locks in any other threads.

llvm-svn: 331023
2018-04-27 08:59:35 +00:00
Petr Hosek 5d3df3cf44 Revert "[CMake] Make simple source used for CMake checks a C file"
This reverts commit r331003 which breaks sanitizer bots because of
missing 32-bit libstdc++ library.

llvm-svn: 331009
2018-04-27 01:25:32 +00:00
Petr Hosek 887fbc61d4 [CMake] Make simple source used for CMake checks a C file
The source being compiled is plain C, but using .cc extension forces it
to be compiled as C++ which requires a working C++ compiler including
C++ library which may not be the case when we're building compiler-rt
together with libcxx as part of runtimes build.

Differential Revision: https://reviews.llvm.org/D46152

llvm-svn: 331003
2018-04-26 23:35:13 +00:00
Reid Kleckner 824eb0e6a0 [asan] Align __asan_global_start so that it works with LLD
Otherwise LLD will not align the .ASAN$GA section start, and
&__asan_globals + 1 will not be the start of the next real ASan global
metadata in .ASAN$GL.

We discovered this issue when attempting to use LLD on Windows in
Chromium: https://crbug.com/837090

llvm-svn: 330990
2018-04-26 20:46:50 +00:00
Reid Kleckner 8e19bd45a9 Enable full debug info in the ASan runtime on Windows
Clang-cl supports the -gline-tables-only flag, so we were going down
that path. Honestly, we should just go ahead and enable full codeview
support.

llvm-svn: 330989
2018-04-26 20:34:19 +00:00
Petar Jovanovic 152060275f Add s390x to XFAIL for illegal_read/write_test.cc
Follow up to r330840 and r330849.
It seems that s390 is also not distinguishing illegal WRITE and READ memory
access.
Add s390x to XFAIL for the tests.

llvm-svn: 330863
2018-04-25 19:34:48 +00:00
Kostya Kortchinsky d8803d3d92 [scudo] Adding an interface function to print allocator stats
Summary:
This adds `__scudo_print_stats` as an interface function to display the Primary
and Secondary allocator statistics for Scudo.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46016

llvm-svn: 330857
2018-04-25 18:52:29 +00:00
Petar Jovanovic 3517ece21f Finetune supported arches for the tests added in r330840
r330840 introduced two tests that may not be supported on all architectures.
powerpc64 seems to be one of those.

llvm-svn: 330849
2018-04-25 17:34:30 +00:00
Petar Jovanovic 1dd3584ec0 [mips] Implement GetWriteFlag() for mips
The read/write flag is set by manually decoding the instruction that caused
the exception. It is implemented this way because the cause register which
contains the needed flag was removed from the signal context structure which
the user handler receives from the kernel.

Patch by Milos Stojanovic.

Differential Revision: https://reviews.llvm.org/D45768

llvm-svn: 330840
2018-04-25 16:21:00 +00:00
Richard Smith c1aaf8ebb7 Remove 'noexcept's that do not match between header and source file.
This code is ill-formed, but under -fno-exceptions compilers generally accept it (at least, prior to C++17). This allows this code to be built by Clang in C++17 mode.

llvm-svn: 330765
2018-04-24 20:33:37 +00:00
Kostya Kortchinsky efe5afbc3d [sanitizer] More dead code removal
Summary:
The following functions are only used in tests: `SetEnv`,
`SanitizerSetThreadName`, `SanitizerGetThreadName`. I don't think they are
going to be used in the future, and I propose to get rid of them, and associated
tests and include.

Reviewers: alekseyshl, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: dvyukov, vitalybuka, kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45838

llvm-svn: 330724
2018-04-24 14:58:10 +00:00
Rainer Orth 482cef6c3d The OpenBSD UBsan port introduced two typos that broke the Solaris sanitizer build.
The following patch restores it, will shortly commit as obvious.

Differential Revision: https://reviews.llvm.org/D46006

llvm-svn: 330712
2018-04-24 13:00:24 +00:00
Dan Liew dfd5a90a99 [LibFuzzer] Tweak `MutationDispatcher::Mutate_CopyPart` mutation.
It doesn't make sense to non-deterministically choose between
`CopyPart(..)` and `InsertPart(..)` when it is known that
`InsertPart(..)` will fail.

This upstream's a change from JFS solver's fork of LibFuzzer.

Differential Revision: https://reviews.llvm.org/D45693

llvm-svn: 330687
2018-04-24 06:31:09 +00:00
Alex Shlyapnikov cf30f6425a [HWASan] Use dynamic shadow memory on Android only.
There're issues with IFUNC support on other platforms.

Differential Revision: https://reviews.llvm.org/D45847

llvm-svn: 330666
2018-04-24 00:17:02 +00:00
Matt Morehouse f66221c6ec [libFuzzer] Add feature to not use AFL's deferred forkserver.
A small but substantial minority of libFuzzer-based fuzzers run code that
does not play well with fork in global constructors or LLVMFuzzerInitialize.
This patch allows these fuzzers to use afl_driver by allowing them to
opt-out of using AFL's deferred forkserver which deferres calling fork until
after this code.

Patch By: metzman

Differential Revision: https://reviews.llvm.org/D45744

llvm-svn: 330652
2018-04-23 21:36:21 +00:00
Martin Liska d6af13e737 Change kAllocatorSpace for powerpc64.
Fixes issue: https://github.com/google/sanitizers/issues/933

Differential Revision: https://reviews.llvm.org/D45950

llvm-svn: 330650
2018-04-23 21:13:40 +00:00
Alex Shlyapnikov 88b7be0d13 [Sanitizer] Internal Printf string width + left-justify.
Summary:
Example:

  Printf("%-5s", "123");

should yield:

  '123  '

In case Printf's requested string field width is larger than the string
argument length, the resulting string should be padded up to the requested
width.

For the simplicity sake, implementing left-justified (right padding) only.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45906

llvm-svn: 330643
2018-04-23 20:30:50 +00:00
Alex Shlyapnikov d03fb0e3e0 [HWASan] Add files missing in r330624
llvm-svn: 330628
2018-04-23 19:05:12 +00:00
Alex Shlyapnikov 79866cad6f [HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt).
Retire the fixed shadow memory mapping to avoid conflicts with default
process memory mapping (currently manifests on Android).

Tests on AArch64 show <1% performance loss and code size increase,
making it possible to use dynamic shadow memory by default.

Keep the fixed shadow memory mapping around to be able to run
performance comparison tests later.

Re-commiting D45847 with fixed shadow for x86-64.

llvm-svn: 330624
2018-04-23 18:19:23 +00:00
Reid Kleckner 061fcd586d Revert r330474 - "[HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt)."
This commit causes internal errors with ld.bfd 2.24. My guess is that
the ifunc usage in this commit is causing problems. This is the default
system linker on Trusty Tahr, which is from 2014. I claim it's still in
our support window. Maybe we will decide to drop support for it, but
let's get the bots green while we do the investigation and have that
discussion.

Discovered here: https://crbug.com/835864

llvm-svn: 330619
2018-04-23 17:26:33 +00:00
Reid Kleckner 963aba3452 Fix clang-cl warnings in compiler-rt
The profile library was missing some includes and was erroneously using
ftruncate. WinASan was using `= {0}` to initialize structs, which
creates -Wmissing-field-initializers and -Wmissing-braces warnings with
clang. Use `= {}` instead, since this is C++.

llvm-svn: 330616
2018-04-23 17:05:47 +00:00
Kostya Kortchinsky 5a8bdc81d6 [scudo] Read ARM feature bits using Fuchsia APIs.
Summary:
    Fuchsia uses zx_system_get_features in lieu of getauxval.
    
    Use this call when checking for CRC32 support.

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: delcypher, llvm-commits, #sanitizers, kristof.beyls, chrib

Differential Revision: https://reviews.llvm.org/D45896

llvm-svn: 330598
2018-04-23 14:53:13 +00:00