Commit Graph

10690 Commits

Author SHA1 Message Date
Matt Morehouse a4374ab222 [CMake] Remove -Wno-maybe-uninitialized from passthrough vars.
Fixes http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/
buildbot.

llvm-svn: 334458
2018-06-12 00:37:06 +00:00
Peter Wu 1c05c95739 [sanitizer] Add fgets, fputs and puts into sanitizer_common
Summary:
Add fgets, fputs and puts to sanitizer_common. This adds ASAN coverage
for these functions, extends MSAN support from fgets to fputs/puts and
extends TSAN support from puts to fputs.

Fixes: https://github.com/google/sanitizers/issues/952

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46545

llvm-svn: 334450
2018-06-11 22:58:04 +00:00
Alex Shlyapnikov ed32baa84b [Sanitizers] %tool_options -> %env_tool_opts in allocator_returns_null.cc
Use proper substitution in the common allocator_returns_null.cc test.

llvm-svn: 334438
2018-06-11 21:04:24 +00:00
Alex Shlyapnikov 406385505d [Sanitizers] Move allocator_returns_null.cc test to common.
Summary:
Add allocator_returns_null.cc test to sanitizer_common and
remove all sanitizer-specific ones except:
- HWASan is not covered by sanitizer_common
- TSan allocator does not have comprehensive error reporting yet

Reviewers: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47971

llvm-svn: 334433
2018-06-11 19:45:59 +00:00
Alex Shlyapnikov 789494026e [Sanitizers] Move pvalloc overflow tests to common.
Summary:
Now all sanitizers with improved allocator error reporting are covered
by these common tests.

Also, add pvalloc-specific checks to LSan.

HWASan is not covered by sanitizer_common, hence its own pvalloc
and other allocator tests.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47970

llvm-svn: 334424
2018-06-11 17:33:53 +00:00
Kostya Kortchinsky 4410e2c43f [scudo] Improve the scalability of the shared TSD model
Summary:
The shared TSD model in its current form doesn't scale. Here is an example of
rpc2-benchmark (with default parameters, which is threading heavy) on a 72-core
machines (defaulting to a `CompactSizeClassMap` and no Quarantine):
- with tcmalloc: 337K reqs/sec, peak RSS of 338MB;
- with scudo (exclusive): 321K reqs/sec, peak RSS of 637MB;
- with scudo (shared): 241K reqs/sec, peak RSS of 324MB.

This isn't great, since the exclusive model uses a lot of memory, while the
shared model doesn't even come close to be competitive.

This is mostly due to the fact that we are consistently scanning the TSD pool
starting at index 0 for an available TSD, which can result in a lot of failed
lock attempts, and touching some memory that needs not be touched.

This CL attempts to make things better in most situations:
- first, use a thread local variable on Linux (intead of pthread APIs) to store
  the current TSD in the shared model;
- move the locking boolean out of the TSD: this allows the compiler to use a
  register and potentially optimize out a branch instead of reading it from the
  TSD everytime (we also save a tiny bit of memory per TSD);
- 64-bit atomic operations on 32-bit ARM platforms happen to be expensive: so
  store the `Precedence` in a `uptr` instead of a `u64`. We lose some
  nanoseconds of precision and we'll wrap around at some point, but the benefit
  is worth it;
- change a `CHECK` to a `DCHECK`: this should never happen, but if something is
  ever terribly wrong, we'll crash on a near null AV if the TSD happens to be
  null;
- based on an idea by dvyukov@, we are implementing a bound random scan for
  an available TSD. This requires computing the coprimes for the number of TSDs,
  and attempting to lock up to 4 TSDs in an random order before falling back to
  the current one. This is obviously slightly more expansive when we have just
  2 TSDs (barely noticeable) but is otherwise beneficial. The `Precedence` still
  basically corresponds to the moment of the first contention on a TSD. To seed
  on random choice, we use the precedence of the current TSD since it is very
  likely to be non-zero (since we are in the slow path after a failed `tryLock`)

With those modifications, the benchmark yields to:
- with scudo (shared): 330K reqs/sec, peak RSS of 327MB.

So the shared model for this specific situation not only becomes competitive but
outperforms the exclusive model. I experimented with some values greater than 4
for the number of TSDs to attempt to lock and it yielded a decrease in QPS. Just
sticking with the current TSD is also a tad slower. Numbers on platforms with
less cores (eg: Android) remain similar.

Reviewers: alekseyshl, dvyukov, javed.absar

Reviewed By: alekseyshl, dvyukov

Subscribers: srhines, kristof.beyls, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47289

llvm-svn: 334410
2018-06-11 14:50:31 +00:00
Peter Wu 1db004ebdd [ASAN] Fix crash on i?86-linux (32-bit) against glibc 2.27 and later
Summary:
Running sanitized 32-bit x86 programs on glibc 2.27 crashes at startup, with:

    ERROR: AddressSanitizer: SEGV on unknown address 0xf7a8a250 (pc 0xf7f807f4 bp 0xff969fc8 sp 0xff969f7c T16777215)
    The signal is caused by a WRITE memory access.
    #0 0xf7f807f3 in _dl_get_tls_static_info (/lib/ld-linux.so.2+0x127f3)
    #1 0xf7a92599  (/lib/libasan.so.5+0x112599)
    #2 0xf7a80737  (/lib/libasan.so.5+0x100737)
    #3 0xf7f7e14f in _dl_init (/lib/ld-linux.so.2+0x1014f)
    #4 0xf7f6eb49  (/lib/ld-linux.so.2+0xb49)

The problem is that glibc changed the calling convention for the GLIBC_PRIVATE
symbol that sanitizer uses (even when it should not, GLIBC_PRIVATE is exactly
for symbols that can change at any time, be removed etc.), see
https://sourceware.org/ml/libc-alpha/2017-08/msg00497.html

Fixes https://github.com/google/sanitizers/issues/954

Patch By: Jakub Jelinek

Reviewed By: vitalybuka, Lekensteyn

Differential Revison: https://reviews.llvm.org/D44623

llvm-svn: 334363
2018-06-10 11:17:47 +00:00
Alex Shlyapnikov b24e6a9905 [MSan] Switch to common aligned allocs tests.
Summary:
Switch to aligned_alloc and posix_memalign tests in sanitizer_common
and delete the MSan-specific ones.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47968

llvm-svn: 334340
2018-06-09 00:47:37 +00:00
Alex Shlyapnikov 643600566d [MSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47793

llvm-svn: 334338
2018-06-08 23:31:42 +00:00
Alex Shlyapnikov 84f8ec5c72 [Sanitizers] Use proper substitution in common allocator test.
Use %env_tool_ops instead of %tool_options.

llvm-svn: 334332
2018-06-08 21:58:22 +00:00
Walter Lee 5a780ee9a9 [asan, myriad] Use local pool for new/delete when ASan run-time is not up
This can happen on Myriad RTEMS so needs to be handled.

Differential Revision: https://reviews.llvm.org/D47916

llvm-svn: 334329
2018-06-08 21:49:38 +00:00
Vitaly Buka 683e2d636d [asan] Re-enable no_asan_gen_globals for 32-bit
PR22682. Bug in gold was fixed 3 years ago.

llvm-svn: 334319
2018-06-08 21:04:44 +00:00
Alex Shlyapnikov dcf0097962 [Sanitizers] Check alignment != 0 for aligned_alloc and posix_memalign
Summary:
Move the corresponding tests to the common folder (as all of the
sanitizer allocators will support this feature soon) and add the checks
specific to aligned_alloc to ASan and LSan allocators.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47924

llvm-svn: 334316
2018-06-08 20:40:35 +00:00
David Carlier 6dedf2b5d8 [Xray] logging forgotten header
pthread.h missing for pthread_key* functions.

Reviewers: dberris

Reviewed By: dberris

Differential Revison: https://reviews.llvm.org/D47933

llvm-svn: 334272
2018-06-08 08:42:37 +00:00
Dean Michael Berris b81372b56b [XRay][compiler-rt] Cleanup some internal XRay utilities
This change uses 'const' for the retryingWriteAll(...) API and removes
unnecessary 'static' local variables in getting the temporary filename.

llvm-svn: 334267
2018-06-08 07:48:03 +00:00
Dean Michael Berris af4d583ea8 [XRay][compiler-rt] Remove reliance on C++ ABI features
Summary:
This fixes http://llvm.org/PR32274.

This change adds a test to ensure that we're able to link XRay modes and
the runtime to binaries that don't need to depend on the C++ standard
library or a C++ ABI library. In particular, we ensure that this will work
with C programs compiled+linked with XRay.

To make the test pass, we need to change a few things in the XRay
runtime implementations to remove the reliance on C++ ABI features. In
particular, we change the thread-safe function-local-static
initialisation to use pthread_* instead of the C++ features that ensure
non-trivial thread-local/function-local-static initialisation.

Depends on D47696.

Reviewers: dblaikie, jfb, kpw, eizan

Reviewed By: kpw

Subscribers: echristo, eizan, kpw, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46998

llvm-svn: 334262
2018-06-08 04:00:07 +00:00
Alex Shlyapnikov a2670df602 [HWASan] Disable allocator_returns_null.cc test on Android.
Disable test added in D47798 on Android and investigate why "new"
behaves differently there.

llvm-svn: 334255
2018-06-08 00:56:49 +00:00
Alex Shlyapnikov af12729185 [HWASan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47798

llvm-svn: 334248
2018-06-07 23:33:33 +00:00
Kostya Serebryany 111d0b2aec [libFuzzer] When printing NEW_FUNC, use 1-base indexing.
Summary: Otherwise we print things like [0/1] which is visually confusing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47837

llvm-svn: 334234
2018-06-07 21:15:24 +00:00
Eric Fiselier 84be761332 [CMake] Fix Libc++ Modules build.
When building the dylib, the C++ headers are fundamentally non-module.
They require special versions of the headers in order to provide C++03 and
legacy ABI definitions. This causes ODR issues when modules are enabled
during both the build and the usage of the libc++ headers.

This patch fixes the build error by disabling modules when building the
libc++ sources.

llvm-svn: 334220
2018-06-07 19:57:43 +00:00
Douglas Yung 20fa3d2504 Silence a -Wconstant-logical-operand warning.
llvm-svn: 334214
2018-06-07 18:51:47 +00:00
Petr Hosek 157a5c2f09 [Fuzzer] Update the header path for fdio/spawn.h on Fuchsia
The path now includes lib/ prefix.

Differential Revision: https://reviews.llvm.org/D47866

llvm-svn: 334213
2018-06-07 18:41:35 +00:00
Petr Hosek 6367bca339 [CMake] Filter out -z,defs when building custom libc++
-z,defs is incompatible with sanitizers so we need to filter it out
from the linker flags before passing them to the libc++ build.

Differential Revision: https://reviews.llvm.org/D47865

llvm-svn: 334212
2018-06-07 18:27:27 +00:00
Kamil Rytarowski b9c78e41ba Disable recursive interceptors in signal(3)/MSan
Summary:
signal(3) on NetBSD calls internally sigaction(2).

Without disabling the recursive interceptor, there are
false positives about uninitialized memory reads inside libc.

This change fixes execution of such programs as sh(1) and
vmstat(1) in the NetBSD userland.

Sponsored by <The NetBSD Foundation>

Reviewers: eugenis, vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47650

llvm-svn: 334168
2018-06-07 07:55:20 +00:00
Kostya Serebryany 67af99235f [libFuzzer] make the corpus elements aware of their data flow traces
llvm-svn: 334158
2018-06-07 01:40:20 +00:00
Kostya Serebryany 7a88f19099 [libFuzzer] simplify a test, hopefully to fix the bot
llvm-svn: 334156
2018-06-07 01:18:43 +00:00
Vitaly Buka 661d81ef61 [sanitizer] Don't use internal_unlink on Windows
llvm-svn: 334152
2018-06-07 00:26:06 +00:00
Kostya Serebryany 4d9fd7a266 [libFuzzer] remove an experimental flag -use_feature_frequency
llvm-svn: 334146
2018-06-06 23:24:41 +00:00
Petr Hosek 3a32b41024 [Fuzzer] Use private libc++ even for Fuchsia
On Fuchsia, we use libc++ compiled with ASan for our ASan built
executable which means we cannot use the same libc++ for libFuzzer when
building fuzz targets, instead we'll link a custom internal libc++ into
Fuchsia's build of libFuzzer like we already do on Linux.

Differential Revision: https://reviews.llvm.org/D47835

llvm-svn: 334144
2018-06-06 22:59:14 +00:00
Petr Hosek 3cf6db2d56 [CMake] Passthrough additional flags to custom libcxx CMake build
This is needed when we're cross-compiling compiler-rt.

Differential Revision: https://reviews.llvm.org/D47834

llvm-svn: 334139
2018-06-06 22:10:12 +00:00
Vitaly Buka 72fd23c9c5 [sanitizer] Replace deprecated mktemp with mkstemp
llvm-svn: 334138
2018-06-06 22:05:48 +00:00
Vitaly Buka 4ef9117b49 [sanitizer] Cleanup ReadFileToVector and ReadFileToBuffer
Summary:
Added unit-test.
Fixed behavior of max_len argument.
Call read syscall with all available buffer, not just a page.

Reviewers: eugenis

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46618

llvm-svn: 334130
2018-06-06 20:53:43 +00:00
Walter Lee 381072c417 [asan] Fix Myraid RTEMS port broken by r333985
Add nop CheckASLR() function.

llvm-svn: 334102
2018-06-06 15:39:47 +00:00
Ilya Biryukov b7c3745006 Fix compile error with libstdc++.
By adding a ctor to create fuzzer_allocator<T> from fuzzer_allocator<U>.
This mimics construcotrs of std::allocator<T>.

Without the constructors, some versions of libstdc++ can't compile
`vector<bool, fuzzer_allocator<bool>>`.

llvm-svn: 334077
2018-06-06 09:22:19 +00:00
Dean Michael Berris c561970b27 [XRay][compiler-rt] Refactor recursion guard for Basic and FDR Mode
Summary:
This change extracts the recursion guard implementation from FDR Mode
and updates it to do the following:

- Do the atomic operation correctly to be signal-handler safe.

- Make it usable in both FDR and Basic Modes.

Before this change, the recursion guard relied on an unsynchronised read
and write on a volatile thread-local. A signal handler could then run in
between the read and the write, and then be able to run instrumented
code as part of the signal handling. Using an atomic exchange instead
fixes that by doing a proper mutual exclusion even in the presence of
signal handling.

Reviewers: kpw, eizan, jfb

Reviewed By: eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47696

llvm-svn: 334064
2018-06-06 06:07:48 +00:00
Kostya Serebryany 1fd005f552 [libFuzzer] initial implementation of -data_flow_trace. It parses the data flow trace and prints the summary, but doesn't use the information in any other way yet
llvm-svn: 334058
2018-06-06 01:23:29 +00:00
Vitaly Buka b89704fa6f [lsan] Do not check for leaks in the forked process
Summary:
If calling process had threads then forked process will fail to detect
references from them.

Fixes https://github.com/google/sanitizers/issues/836

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47751

llvm-svn: 334036
2018-06-05 18:15:57 +00:00
Alex Shlyapnikov 236c3f9c4a [LSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, detailed and
structured errors were defined and reported under the appropriate conditions.

Reviewers: eugenis

Subscribers: srhines, mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47645

llvm-svn: 334034
2018-06-05 18:02:09 +00:00
Dean Michael Berris 4c1fdcfb97 [XRay][compiler-rt] Use DCHECK instead of assert(...) (NFC)
Use DCHECK instead of assert(...) in the XRay runtime.

llvm-svn: 334002
2018-06-05 10:27:20 +00:00
Dean Michael Berris 13b2fcc4e6 [XRay][compiler-rt] Use static instead of inline (NFC)
We don't actually need to support multiple definitions of the functions
in FDR mode, but rather want to make sure that the implementation-detail
functions are marked as 'static' instead. This allows the inliner to do
its magic better for these functions too, since inline functions must
have a unique address across translation units.

llvm-svn: 334001
2018-06-05 10:18:39 +00:00
Dean Michael Berris abbeb4cbf7 [XRay][compiler-rt] Remove __xray:: in some places (NFC)
This is a cosmetic change to remove unnecessary full-qualifications of
types/functions that are already in the __xray:: namespace.

llvm-svn: 334000
2018-06-05 10:12:58 +00:00
Dean Michael Berris bd37b57b4d [XRay][compiler-rt] Remove namespace __xray_fdr_internal (NFC)
We no longer need the __xray_fdr_internal namespace.

llvm-svn: 333998
2018-06-05 10:01:45 +00:00
Dean Michael Berris 7f88ea1fa2 [XRay][compiler-rt] Merge XRay FDR mode into a single file (NFC)
We planned to have FDR mode's internals unit-tested but it turns out
that we can just use end-to-end testing to verify the implementation.
We're going to move towards that approach more and more going forward,
so we're merging the implementation details of FDR mode into a single
.cc file.

We also avoid globbing in the XRay test helper macro, and instead list
down the files from the lib directory.

llvm-svn: 333986
2018-06-05 08:20:54 +00:00
Kamil Rytarowski 7d260775f3 Introduce CheckASLR() in sanitizers
Summary:
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD.

Introduce a generic CheckASLR() routine, that implements a check for the
current process. This flag depends on the global or per-process settings.

There is no simple way to disable ASLR in the build process from the
level of a sanitizer or during the runtime execution.

With ASLR enabled sanitizers that operate over the process virtual address
space can misbehave usually breaking with cryptic messages.

This check is dummy for !NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: cryptoad, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47442

llvm-svn: 333985
2018-06-05 07:29:23 +00:00
Dean Michael Berris 5eaaff6095 [XRay][compiler-rt] Remove __sanitizer:: from namespace __xray (NFC)
This is a non-functional change that removes the full qualification of
functions in __sanitizer:: being used in __xray.

llvm-svn: 333983
2018-06-05 06:12:42 +00:00
Dean Michael Berris 8ba925d2c3 [XRay][compiler-rt] Remove reliance on C++ ABI from BufferQueue
Summary:
This is part of the work to address http://llvm.org/PR32274.

We remove the calls to array-placement-new and array-delete. This allows
us to rely on the internal memory management provided by
sanitizer_common/sanitizer_internal_allocator.h.

Reviewers: eizan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D47695

llvm-svn: 333982
2018-06-05 03:46:54 +00:00
David Carlier a6303a9866 [TSan] FreeBSD / intercept thr_exit
intercepting thr_exit to terminate threads under FreeBSD.
Unblock few unit tests hanging.

Reviewers: krytarowski, vitalybuka, emaste

Reviewed By: dvyukov		

Differential Revision: https://reviews.llvm.org/D47677

M    lib/tsan/rtl/tsan_interceptors.cc

llvm-svn: 333870
2018-06-04 06:02:38 +00:00
Dean Michael Berris d12cb59c7a [XRay][compiler-rt] Remove RTTI requirement from XRay Builds
XRay doesn't use RTTI and doesn't need it. We disable it explicitly in
the CMake config, similar to how the other sanitizers already do it.

Part of the work to address http://llvm.org/PR32274.

llvm-svn: 333867
2018-06-04 04:22:26 +00:00
Petr Hosek 8908aa4e30 [Fuzzer] Migrate Fuchsia port from launchpad to fdio_spawn
fdio_spawn is the new public API for launching processes in Fuchsia.

Differential Revision: https://reviews.llvm.org/D47649

llvm-svn: 333809
2018-06-02 01:17:10 +00:00
Kostya Serebryany d74d04a6c5 Add weak definitions of trace-cmp hooks to dfsan
Summary:
This allows to build and link the code with e.g.
-fsanitize=dataflow -fsanitize-coverage=trace-pc-guard,pc-table,func,trace-cmp
w/o providing (all) the definitions of trace-cmp hooks.

This is similar to dummy hooks provided by asan/ubsan/msan for the same purpose,
except that some of the hooks need to have the __dfsw_ prefix
since we need dfsan to replace them.

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47605

llvm-svn: 333796
2018-06-01 21:59:25 +00:00
Walter Lee e2f884d73f [asan, myriad] Implement aligned local pool allocation
Extend the local pool allocation support to posix_memalign.

Differential Revision: https://reviews.llvm.org/D47642

llvm-svn: 333788
2018-06-01 21:29:05 +00:00
Walter Lee bf03bfb0a3 [asan, myriad] Configure platform interceptors
Myriad only uses the platform interceptors for memory allocation
routines.  Configure them properly.

Also add a missing guard around aligned alloc interceptor.

Differential Revision: https://reviews.llvm.org/D47641

llvm-svn: 333784
2018-06-01 19:47:41 +00:00
David Carlier 45cdbdac06 [XRay] Fix supposely failing test for FreeBSD
One test was marked as XFAIL for FreeBSD however still running when launching make check-xray

Reviewers: krytarowski, vitalybuka

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D47622

llvm-svn: 333733
2018-06-01 09:15:44 +00:00
Yvan Roux 3b480d1858 [ASAN] Sanitize testsuite for ARM.
Address failures exhibited by ARMv8 bot in Thumb mode:

- Fix logic for fast unwinding support (i.e feature is not available for Thumb)
- Fix Unsupported and Requires rules to handle armv8 as well as soft and hard
  float targets
- Un-xfail passing tests

Differential Revision: https://reviews.llvm.org/D47575

llvm-svn: 333729
2018-06-01 06:23:36 +00:00
Reid Kleckner d44a6eab40 Fix GCC 4.8.5 build by avoiding braced default member initializer
Use internal_memset instead.  Should revive all Linux Chromium ToT bots.

llvm-svn: 333678
2018-05-31 18:26:36 +00:00
Sylvestre Ledru ac804c2238 Define SIZEOF_STRUCT_USTAT for 32bit sparc.
Patch landed on gcc upstream:
27453e962b
Patch by Matthias Klose

llvm-svn: 333644
2018-05-31 12:41:15 +00:00
Stephan Bergmann af3226752b Missing include
(For some reason, my Mac build complained about unknown pthread_once_t and
PTHREAD_ONCE_INIT, but not about pthread_once itself.)

llvm-svn: 333638
2018-05-31 09:26:31 +00:00
Dean Michael Berris d1fe506694 [XRay] Fixup: Remove unnecessary type alias
Follow-up to D45758.

llvm-svn: 333628
2018-05-31 05:25:47 +00:00
Dean Michael Berris 16c865b071 [XRay] Fixup: Explicitly call std::make_tuple(...)
Follow-up to D45758.

llvm-svn: 333627
2018-05-31 05:02:11 +00:00
Dean Michael Berris ca856e07de [XRay] Fixup: Address some warnings breaking build
Follow-up to D45758.

llvm-svn: 333625
2018-05-31 04:55:11 +00:00
Dean Michael Berris 1eb8c206cd [XRay][profiler] Part 3: Profile Collector Service
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements a centralised collector for `FunctionCallTrie`
instances, associated per thread. It maintains a global set of trie
instances which can be retrieved through the XRay API for processing
in-memory buffers (when registered). Future changes will include the
wiring to implement the actual profiling mode implementation.

This central service provides the following functionality:

*  Posting a `FunctionCallTrie` associated with a thread, to the central
   list of tries.

*  Serializing all the posted `FunctionCallTrie` instances into
   in-memory buffers.

*  Resetting the global state of the serialized buffers and tries.

Depends on D45757.

Reviewers: echristo, pelikan, kpw

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D45758

llvm-svn: 333624
2018-05-31 04:33:52 +00:00
Kostya Serebryany 980e45fe55 [libFuzzer] add collect_data_flow.py that allows to run the data-flow tracer several times on subsets of inputs bytes, to overcome DFSan out-of-label failures
llvm-svn: 333616
2018-05-31 01:27:07 +00:00
Reid Kleckner b54ac414d1 [asan] Remove unneeded VirtualQuery from exception handler
We don't use the result of the query, and all tests pass if I remove it.
During startup, ASan spends a fair amount of time in this handler, and
the query is much more expensive than the call to commit the memory.

llvm-svn: 333595
2018-05-30 21:21:18 +00:00
Walter Lee 3dc6cea0e3 [asan, myriad] Simplify main thread handling
On Myriad RTEMS, we don't need to treat the main thread differently.
The existing thread hooks will do the right thing, so get rid of all
the unneeded special logic.

Differential Revision: https://reviews.llvm.org/D47502

llvm-svn: 333504
2018-05-30 04:57:29 +00:00
Walter Lee 0a70e9db03 [asan, myriad] Reset shadow memory during exit
Reset shadow memory during exit.  Also update a cut-and-paste comment,
and do some minor refactoring of InitializeShadowMemory.

Differential Revision: https://reviews.llvm.org/D47501

llvm-svn: 333503
2018-05-30 04:57:04 +00:00
George Burgess IV d30bcab70e Fix a typo; NFC
llvm-svn: 333331
2018-05-26 02:28:43 +00:00
Vitaly Buka 4add8a1f4e [safestack] Lazy initialization of interceptors
Interceptors initialization may try to allocate memory and to call not
initialized allocator.

It's similar to r326025 for CFI.

llvm-svn: 333329
2018-05-26 01:18:32 +00:00
Petr Hosek 796bb7d4ce [Fuzzer] Update _zx_port_wait function use in Fuchsia port
Fuchsia's _zx_port_wait no longer takes the count argument. This
change also updates all symbol uses to use the underscored versions.

Differential Revision: https://reviews.llvm.org/D47403

llvm-svn: 333328
2018-05-26 01:02:34 +00:00
H.J. Lu 11c1c266cc Revert the last test commit
llvm-svn: 333310
2018-05-25 20:51:45 +00:00
H.J. Lu caab7c8cca This is a test commit to verify repository access
llvm-svn: 333309
2018-05-25 20:51:10 +00:00
David Carlier f6311c669f [MemorySanitizer] fix mmap test for oses not implementing MAP_NORESERVE flag
Reviewers: krytarowski, eugenis

Reviewed By: eugenis

Differential Revision: https://review.llvm.org/D47146

llvm-svn: 333306
2018-05-25 20:30:39 +00:00
Vitaly Buka 540d7e528d [libFuzzer] Avoid optimization of "abs(x) < 0"
llvm-svn: 333295
2018-05-25 18:27:12 +00:00
Vitaly Buka b161db099d Revert "[libFuzzer] [NFC] Generalize DSO tests to work even when files are moved."
Breaks libFuzzer tests.

This reverts commit r333243.

llvm-svn: 333257
2018-05-25 07:18:29 +00:00
George Karpenkov 698b0a674d [libFuzzer] [NFC] Generalize fdmask test.
The ordering of stderr/stdout writes might not be guaranteed,
and is irrelevant for this test.

Differential Revision: https://reviews.llvm.org/D47294

llvm-svn: 333245
2018-05-24 23:55:54 +00:00
George Karpenkov b4ccb3ab08 [libFuzzer] [NFC] Do not remove SanitizerLintCheck from libFuzzer tests
With %run commands being present now, we want to make sure that they
stay present.

Differential Revision: https://reviews.llvm.org/D47293

llvm-svn: 333244
2018-05-24 23:55:54 +00:00
George Karpenkov fe44eb4cf1 [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 333243
2018-05-24 23:55:53 +00:00
George Karpenkov 464d1415b2 [libFuzzer] Fix multi-arch build support for fuzzer tests.
The bug was not previously exposed as only one architecture is currently
supported for libFuzzer.

Differential Revision: https://reviews.llvm.org/D47287

llvm-svn: 333242
2018-05-24 23:55:53 +00:00
George Karpenkov 02c85f1489 [libFuzzer] Run libFuzzer unit tests only on host architecture.
LIT tests are used to test other cross-compiled architectures,
unit tests are only run on the host.
NFC now as currently only a single architecture is supported.

Differential Revision: https://reviews.llvm.org/D47286

llvm-svn: 333241
2018-05-24 23:55:52 +00:00
George Karpenkov 0dcb6902f1 [libFuzzer] XFAIL a few libFuzzer tests on iOS.
NFC for now.

Differential Revision: https://reviews.llvm.org/D47284

llvm-svn: 333240
2018-05-24 23:55:52 +00:00
George Karpenkov e7e3fc26ee [libFuzzer] Fix a typo in CMake configuration.
NFC now when libFuzzer supports only one architecture,
will stop being NFC after multiple architectures are supported.

Differential Revision: https://reviews.llvm.org/D47283

llvm-svn: 333239
2018-05-24 23:55:51 +00:00
Azharuddin Mohammed 06e0fd3bc1 Revert "[cmake] [ARM] Check if VFP is supported before including any VFP builtins"
This reverts commit 2a10f5da5acb1b51d0a2ecf13dca0bf1de859db2.

llvm-svn: 333232
2018-05-24 21:36:27 +00:00
Azharuddin Mohammed 80968a9aba [cmake] [ARM] Check if VFP is supported before including any VFP builtins
Summary:
rL325492 disables FPU features when using soft floating point
(-mfloat-abi=soft), which is used internally when building for armv7. This
causes errors with builtins that utililize VFP instructions. With this change
we first check if VFP is enabled (by checking if the preprocessor macro
__VFP_FP__ is defined) before including such builtins.

Reviewers: rengolin, samsonov, compnerd, smeenai, javed.absar, peter.smith

Reviewed By: peter.smith

Subscribers: peter.smith, mgorny, kristof.beyls, chrib, llvm-commits

Differential Revision: https://reviews.llvm.org/D47217

llvm-svn: 333216
2018-05-24 18:53:16 +00:00
Craig Topper 383fe5c866 sanitizer: Use pre-computed size of struct ustat for Linux
<sys/ustat.h> has been removed from glibc 2.28 by:

commit cf2478d53ad7071e84c724a986b56fe17f4f4ca7
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Sun Mar 18 11:28:59 2018 +0800

Deprecate ustat syscall interface
This patch uses pre-computed size of struct ustat for Linux to fix

https://bugs.llvm.org/show_bug.cgi?id=37418

Patch by H.J. Lu.

Differential Revision: https://reviews.llvm.org/D47281

llvm-svn: 333213
2018-05-24 17:59:47 +00:00
Igor Kudrin 13f8a77d79 [tsan] Fix a failure in test java_race_pc on AArch64 after r333071
llvm-svn: 333180
2018-05-24 12:26:20 +00:00
Kostya Serebryany 4d53b744ca [libFuzzer] DataFlow tracer now tags a subset of the input. A separate script merges traces from the subsets
llvm-svn: 333149
2018-05-24 01:43:48 +00:00
Kostya Serebryany 500ca8713c [libFuzzer] fix two off-by-ones (!!) in the data flow tracer
llvm-svn: 333142
2018-05-23 23:55:54 +00:00
Jake Ehrlich 767d92e3fb [fuchsia] Add line buffering in RawWrite
This change causes RawWrite to buffer upto 128 bytes or until
a line is reached. This helps group calls into more readable
lines.

llvm-svn: 333136
2018-05-23 22:27:12 +00:00
Kostya Serebryany 4925392897 [libFuzzer] change the output format for the DataFlow tracer
llvm-svn: 333122
2018-05-23 20:57:11 +00:00
Kostya Serebryany 28fe54fac8 [libFuzzer] add a stress test for the DataFlow tracer
llvm-svn: 333119
2018-05-23 20:23:33 +00:00
Max Moroz fe97441c2c [libFuzzer] Don't complain about lack of interesting inputs when -runs=0.
Summary:
The most common usecase for -runs=0 is for generating code coverage
over some corpus. Coverage reports based on sancov are about to be deprecated,
which means some external coverage solution will be used, e.g. Clang source
based code coverage, which does not use any sancov instrumentations and thus
libFuzzer would consider any input to be not interesting in that case.

Reviewers: kcc

Reviewed By: kcc

Subscribers: alex, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47271

llvm-svn: 333116
2018-05-23 19:42:30 +00:00
Kostya Kortchinsky 08c630457a [sanitizer] Define UINTPTR_MAX
Summary:
`sanitizer_internal_defs.h` didn't have this define, which will be useful in
an upcoming CL.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D47270

llvm-svn: 333109
2018-05-23 18:13:21 +00:00
Kamil Rytarowski 5b1b594f64 Fix internal_mmap() on 32-bit NetBSD platforms
There is need to use internal_syscall64() instead of internal_syscall_ptr().
The offset argument of type off_t is always 64-bit.

llvm-svn: 333075
2018-05-23 10:37:00 +00:00
Igor Kudrin c0099f9be7 [sanitizer] Reland "Be more accurate when calculating the previous instruction address on ARM."
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 333071
2018-05-23 09:18:10 +00:00
Kostya Serebryany 7f5bf83f53 [libFuzzer] modify -print_corpus_stats to print whether the input reaches the focus function
llvm-svn: 333048
2018-05-23 01:42:53 +00:00
Petr Hosek aa6d13e501 [CMake] Support builtins as Clang default rtlib in compiler-rt
Use compiler-rt builtins when selected as default Clang rtlib and avoid
explicitly passing -rtlib= flag to avoid the "argument unused during
compilation" warning.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47115

llvm-svn: 333037
2018-05-22 22:58:48 +00:00
Evgeniy Stepanov 45f8986b80 [asan] Use dynamic allocator space address on Android/AArch64.
Summary:
We need one library to support all of 39, 42 and 48 bit VMAs, and
there is no common address that works for all of them.

Reviewers: kcc, alekseyshl, javed.absar

Subscribers: rengolin, srhines, kubamracek, kristof.beyls, llvm-commits, cryptoad

Differential Revision: https://reviews.llvm.org/D47160

llvm-svn: 333025
2018-05-22 20:44:45 +00:00
Petr Hosek 1ffaa48d4a [CMake] Support libc++ as Clang default stdlib in compiler-rt
Use libc++ when selected as default Clang stdlib and avoid checking
C++ compiler when using the in-tree version of libc++.

This is a partial alternative to D47094 that does not rely on compiler
runtime checks.

Differential Revision: https://reviews.llvm.org/D47100

llvm-svn: 333010
2018-05-22 18:33:27 +00:00
Petr Hosek 4ca5af0721 [CMake] Silence unused variable warning in compiler check
This is breaking the compiler check.

llvm-svn: 332936
2018-05-22 02:53:32 +00:00
Petr Hosek f6dda7cb44 [CMake] Use a different source depending on C++ support
When using system C++ library, assume we have a working C++ compiler and
try to compile a complete C++ program. When using in tree C++ library,
only check the C compiler since the C++ library likely won't have been
built yet at time of running the check.

Differential Revision: https://reviews.llvm.org/D47169

llvm-svn: 332924
2018-05-22 01:01:56 +00:00
Walter Lee ead3b3487b [asan] Make GetCurrentThread RTEMS-friendly
On RTEMS, system and user code all live in a single binary and address
space. There is no clean separation, and instrumented code may
execute before the ASan run-time is initialized (or after it has been
destroyed).

Currently, GetCurrentThread() may crash if it's called before ASan
run-time is initialized. Make it return nullptr instead.

Similarly, fix __asan_handle_no_return so that it gives up rather than
try something that may crash.

Differential Revision: https://reviews.llvm.org/D46459

llvm-svn: 332888
2018-05-21 20:43:36 +00:00
Kostya Serebryany 69c2b71a51 [libFuzzer] reinstate -dump_coverage, which is still in use (reverts r332036)
llvm-svn: 332876
2018-05-21 19:47:00 +00:00
Petr Hosek 702d073642 Revert "[CMake] Reland "Make simple source used for CMake checks a C file""
This reverts commit rCRT332679 which doesn't quite work and will be
replaced by D47100 and D47115 which is a cleaner solution.

llvm-svn: 332871
2018-05-21 19:32:43 +00:00
Kamil Rytarowski 434606c8e4 Align ClearShadowForThreadStackAndTLS for NetBSD/i386
Summary:
The static TLS vector for the main thread on NetBSD/i386 can be
unaligned in terms of the shadow granularity. Align the start of it with
Round Down and end of it with Round Up operations for the shadow
granularity shift.

Example static TLS vector ranges on NetBSD/i386:
tls_begin_=0xfbee7244 tls_end_=0xfbee726c.

ClearShadowForThreadStackAndTLS() is called from the Main Thread
bootstrap functions.

This change restores the NetBSD x86 32-bit (i386) support.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46585

llvm-svn: 332792
2018-05-19 01:20:00 +00:00
Kamil Rytarowski 51daee08cd Port msan_test.cc to NetBSD
Summary:
The changes allows building this file on NetBSD, mostly by
disabling the unsupported functions and adding OS-specific
system includes.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46712

llvm-svn: 332791
2018-05-19 01:10:25 +00:00
Kamil Rytarowski c7fe74576f Do not link with -ldl in NetBSD/ASan tests
Summary:
The dlopen(3) features on NetBSD are in libc.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46713

llvm-svn: 332790
2018-05-19 01:02:51 +00:00
Evgeniy Stepanov 28f330fd6f [msan] Don't check divisor shadow in fdiv.
Summary:
Floating point division by zero or even undef does not have undefined
behavior and may occur due to optimizations.

Fixes https://bugs.llvm.org/show_bug.cgi?id=37523.

Reviewers: kcc

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D47085

llvm-svn: 332761
2018-05-18 20:19:53 +00:00
Reid Kleckner 4f9516d617 Revert r332683 & r332684 relating to compiler runtime checks
r332683 passes flags to the compiler without first checking if they are
supported.

llvm-svn: 332754
2018-05-18 18:44:37 +00:00
Walter Lee 0edca4f505 [asan] Explicitly declare memintrinsics interceptors to have C linkage
This is needed on RTEMS.  Also update a comment.

Differential Revision: https://reviews.llvm.org/D47079

llvm-svn: 332746
2018-05-18 18:01:08 +00:00
Walter Lee 927f4ec1ff [asan] Remove an unsigned compare >= 0
This is not needed after we've forked the Myriad version.  Not to
mention it produces a compiler warning.

Differential Revision: https://reviews.llvm.org/D47054

llvm-svn: 332744
2018-05-18 17:52:12 +00:00
Walter Lee e35f57f023 [asan] Add target-specific files for Myriad RTEMS port
Differential Revision: https://reviews.llvm.org/D46468

llvm-svn: 332691
2018-05-18 04:10:12 +00:00
Walter Lee ce26498e3f [asan] Add support for Myriad RTEMS memory map
The Myriad RTEMS memory system has a few unique aspects that
require support in the ASan run-time.

- A limited amount of memory (currently 512M).

- No virtual memory, no memory protection.

- DRAM starts at address 0x80000000.  Other parts of memory may be
  used for MMIO, etc.

- The second highest address bit is the "cache" bit, and 0x80000000
  and 0x84000000 alias to the same memory.

To support the above, we make the following changes:

- Use a ShadowScale of 5, to reduce shadow memory overhead.

- Adjust some existing macros to remove assumption that the lowest
  memory address is 0.

- add a RawAddr macro that on Myriad strips the cache bit from the
  input address, before using the address for shadow memory (for other
  archs this does nothing).

- We must check that an address is in DRAM range before using it to
  index into shadow memory.

Differential Revision: https://reviews.llvm.org/D46456

llvm-svn: 332690
2018-05-18 04:09:45 +00:00
Petr Hosek 83061cc4aa [CMake] Use <UNIX|WINDOWS>_COMMAND with separate_arguments
NATIVE_COMMAND is only available since CMake 3.9.

llvm-svn: 332684
2018-05-18 01:20:47 +00:00
Petr Hosek 8b5fe57d3e [CMake] Detect the compiler runtime and standard library
Rather then requiring the user to specify runtime the compiler
runtime and C++ standard library, or trying to guess them which is
error-prone, use auto-detection by parsing the compiler link output.

Differential Revision: https://reviews.llvm.org/D46857

llvm-svn: 332683
2018-05-18 01:10:09 +00:00
Walter Lee b134dbb3c3 [sanitizer] Trivial portion of the port to Myriad RTEMS
This commit contains the trivial portion of the port of ASan to
Myriad RTEMS.
- Whitelist platform in sanitizer_platform.h, ubsan_platform.h
- Turn off general interception
- Use memset for FastPoisonShadow
- Define interception wrappers
- Set errno symbol correctly
- Enable ASAN_LOW_MEMORY
- Enable preinit array
- Disable slow unwinding
- Use fuchsia offline symbolizer
- Disable common code for: InitializeShadowMemory, CreateMainThread,
  AsanThread::ThreadStart, StartReportDeadlySignal,
  MaybeReportNonExecRegion.

Differential Revision: https://reviews.llvm.org/D46454

llvm-svn: 332681
2018-05-18 00:43:54 +00:00
Petr Hosek 3eb71831e5 [CMake] Reland "Make simple source used for CMake checks a C file"
The source being compiled is plain C, but using .cc extension forces it
to be compiled as C++ which requires a working C++ compiler including
C++ library which may not be the case when we're building compiler-rt
together with libcxx as part of runtimes build.

Differential Revision: https://reviews.llvm.org/D47031

llvm-svn: 332679
2018-05-17 23:50:35 +00:00
Petr Hosek 5a668a054b [CMake] Cleanup find_compiler_rt_library function [NFC]
Rename the output variable and remove the unnecessary set call.

llvm-svn: 332661
2018-05-17 20:35:55 +00:00
Alex Lorenz a54e218d4a [builtins][macos] bump up the the macOS version min in os_version_check tests
This ensures that the tests link with the latest OS.

llvm-svn: 332647
2018-05-17 18:41:38 +00:00
Kostya Serebryany a52d30b2bc [libFuzzer] rename a test from .c to .cpp
llvm-svn: 332558
2018-05-16 23:38:53 +00:00
Walter Lee c5368c230e [asan] Add a magic shadow value for shadw gap
This gives us something to insert into the shadow gap for systems that
don't have memory protection turned on there (i.e. on Myriad).

Differential Revision: https://reviews.llvm.org/D46457

llvm-svn: 332557
2018-05-16 23:36:01 +00:00
Kostya Serebryany e9c6f06cce [libFuzzer] add an experimental flag -focus_function: libFuzzer will try to focus on inputs that trigger that function
llvm-svn: 332554
2018-05-16 23:26:37 +00:00
Walter Lee 3bf63c9201 [asan] Restore check removed by r332033
Needed by fiber handling code, and possibly other code paths.

Differential Revision: https://reviews.llvm.org/D46981

llvm-svn: 332553
2018-05-16 23:23:56 +00:00
Max Moroz d63396097d [libFuzzer] add a symbolic execution puzzle (difficult for today's libFuzzer).
Summary:
This can be solved just in seconds with KLEE. Current libFuzzer
is able to satistfy 101 constraints out of 410 constraints presented during
the first hour of running with -use_value_profile=1 and -max_len=20.
During the next 3 hours, libFuzzer is able to generate ~50 NEW inputs,
bot none of those solve any new constraint.
During the next 20 hours, it didn't find any NEW inputs.

This test might be interesting for experimenting with the data flow tracing
approach started in https://reviews.llvm.org/D46666.

For the solution with KLEE and other information, see
https://github.com/Dor1s/codegate2017-quals-angrybird

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46924

llvm-svn: 332507
2018-05-16 18:19:30 +00:00
Kostya Kortchinsky e5c9e9f0bd [scudo] Quarantine optimization
Summary:
It turns out that the previous code construct was not optimizing the allocation
and deallocation of batches. The class id was read as a class member (even
though a precomputed one) and nothing else was optimized. By changing the
construct this way, the compiler actually optimizes most of the allocation and
deallocation away to only work with a single class id, which not only saves some
CPU but also some code footprint.

Reviewers: alekseyshl, dvyukov

Reviewed By: dvyukov

Subscribers: dvyukov, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46961

llvm-svn: 332502
2018-05-16 18:12:31 +00:00
Kostya Kortchinsky 561228b2fa [sanitizer] Minor 32-bit primary improvements
Summary:
For the 32-bit TransferBatch:
- `SetFromArray` callers have bounds `count`, so relax the `CHECK` to `DCHECK`;
- same for `Add`;
- mark `CopyToArray` as `const`;
For the 32-bit Primary:
- `{Dea,A}llocateBatch` are only called from places that check `class_id`,
  relax the `CHECK` to `DCHECK`;
- same for `AllocateRegion`;
- remove `GetRegionBeginBySizeClass` that is not used;
- use a local variable for the random shuffle state, so that the compiler can
  use a register instead of reading and writing to the `SizeClassInfo` at every
  iteration;
For the 32-bit local cache:
- pass the count to drain instead of doing a `Min` everytime which is at times
  superfluous.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46657

llvm-svn: 332478
2018-05-16 15:13:26 +00:00
Vitaly Buka 75e01fa00b [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock
Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46793

llvm-svn: 332320
2018-05-15 01:39:25 +00:00
Vitaly Buka 3f1fd7988c [asan] Workaround to avoid hangs in Chromium tests
Summary:
For some reasons on Chromium when we start leak checking we get own pid as 1.
After that we see threads with PPID:0 assuming that thread is dead in infinite
loop.

To resolve particularly this case and possible issues like this, when IsAlive check failed to detect thread status, we need to limit the number of SuspendAllThreads
iterations.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46852

llvm-svn: 332319
2018-05-15 01:39:13 +00:00
Kostya Serebryany 2f7edaeb39 [libFuzzer] deprecate equivalence_server
llvm-svn: 332316
2018-05-15 01:15:47 +00:00
Dean Michael Berris 980d93d0e0 [XRay][profiler] Part 2: XRay Function Call Trie
Summary:
This is part of the larger XRay Profiling Mode effort.

This patch implements a central data structure for capturing statistics
about XRay instrumented function call stacks. The `FunctionCallTrie`
type does the following things:

*  It keeps track of a shadow function call stack of XRay instrumented
   functions as they are entered (function enter event) and as they are
   exited (function exit event).

*  When a function is entered, the shadow stack contains information
   about the entry TSC, and updates the trie (or prefix tree)
   representing the current function call stack. If we haven't
   encountered this function call before, this creates a unique node for
   the function in this position on the stack. We update the list of
   callees of the parent function as well to reflect this newly found
   path.

*  When a function is exited, we compute statistics (TSC deltas,
   function call count frequency) for the associated function(s) up the
   stack as we unwind to find the matching entry event.

This builds upon the XRay `Allocator` and `Array` types in Part 1 of
this series of patches.

Depends on D45756.

Reviewers: echristo, pelikan, kpw

Reviewed By: kpw

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D45757

llvm-svn: 332313
2018-05-15 00:42:36 +00:00
Vitaly Buka cdb89b72f5 Revert "[sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock"
Tsan tests fail.

This reverts commit r332268

llvm-svn: 332276
2018-05-14 18:53:13 +00:00
Vitaly Buka e5bd326822 [sanitizer] Intercept __pthread_mutex_lock and __pthread_mutex_unlock
Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46793

llvm-svn: 332268
2018-05-14 18:03:34 +00:00
Dan Liew fe7250c2fa Fix another case where libstdc++ is being inappropriately requested (see
r328775) for all platforms.

Given that this is the second occurance of this problem it seemed worth
fixing this problem in a more generic way. r328775 has been reverted and
now a substitution `%linux_static_libstdcplusplus` has been provided.
This substitution expands to Clang driver arguments to use a static
libstdc++ on Linux and on all other platforms it expands to nothing.

The `asan/TestCases/throw_invoke_test.cc` and
`test/tsan/static_init6.cc` test cases now use this substitution.

rdar://problem/39948818

Differential Revision: https://reviews.llvm.org/D46401

llvm-svn: 332254
2018-05-14 15:48:49 +00:00
Dean Michael Berris 238aa1366e [XRay][compiler-rt] Relocate a DCHECK to the correct location.
Fixes a bad DCHECK where the condition being checked is still valid (for
iterators pointing to sentinels).

Follow-up to D45756.

llvm-svn: 332212
2018-05-14 04:21:12 +00:00
Dean Michael Berris 034b6c1a66 [XRay][compiler-rt] Fixup: Use proper string comparison for DCHECK
Fixes the sanitizer build.

Follow-up to D46574.

llvm-svn: 332211
2018-05-14 04:14:39 +00:00
Dean Michael Berris 845fe8a708 [XRay][compiler-rt] Fixup: Avoid C++11 narrowing in non-x86_64
This should fix non-x86_64 builds where size_t != atomic_uint64_t::Type.

Follow-up to D46574.

llvm-svn: 332209
2018-05-14 03:55:12 +00:00
Dean Michael Berris a2a0e1f353 [XRay][compiler-rt] Support in-memory processing of FDR mode logs
Summary:
This change allows for handling the in-memory data associated with the
FDR mode implementation through the new `__xray_log_process_buffers`
API. With this change, we can now allow users to process the data
in-memory of the process instead of through writing files.

This for example allows users to stream the data of the FDR logging
implementation through network sockets, or through other mechanisms
instead of saving them to local files.

We introduce an FDR-specific flag, for "no_file_flush" which lets the
flushing logic skip opening/writing to files.

This option can be defaulted to `true` when building the compiler-rt
XRay runtime through the `XRAY_FDR_OPTIONS` preprocessor macro.

Reviewers: kpw, echristo, pelikan, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46574

llvm-svn: 332208
2018-05-14 03:35:01 +00:00
Vitaly Buka afce413098 [sanitizer] Replace #if SANITIZER_NETBSD with #if SANITIZER_INTERCEPT___LIBC_MUTEX
llvm-svn: 332190
2018-05-13 05:55:45 +00:00
Vitaly Buka 5e9dd3174d [sanitizer] Fix typo in macro name
llvm-svn: 332171
2018-05-12 03:23:55 +00:00
Walter Lee d8f7b0b5d5 [sanitizer] Allow Fuchsia symbolizer to be reused by Myriad RTEMS
Like Fuchsia, Myriad RTEMS uses an off-line symbolizer -- we just need
a custom backtrace frame string.  Move this definition to
sanitizer_fuchsia.h; the corresponding RTEMS one will be added when we
add sanitizer_rtems.h.

Differential Revision: https://reviews.llvm.org/D46462

llvm-svn: 332157
2018-05-11 23:52:19 +00:00
George Karpenkov 669375814c [libFuzzer] [NFC] Split fuzzer.test into smaller tests
Keeping fuzzer.test together as a gigantic test has no advantages and multiple disadvantages:

 - Worse test parallelization (fuzzer.test is always blocking the test
run on a machine with many cores)
 - Debugging test failures is made more difficult (not clear what fails
from fuzzer.test failing)
 - Makes porting tests to new platforms more difficult (whenever
fuzzer.test fails have to inspect the output to figure out what is going
on, and then restart all of it)
 - Hides dead code (in our case, "Done1000000" FileCheck variable was
never used, DISABLED: not %t-UninstrumentedTest-Uninstrumented was never
compiled, and there was small amount of duplication vs. simple-cmp.test)
 - Makes correspondence between LIT .test files and .cpp files less
straightforward

Differential Revision: https://reviews.llvm.org/D46557

llvm-svn: 332145
2018-05-11 21:29:53 +00:00
George Karpenkov fa57519e31 [libFuzzer] [NFC] Prefix all run lines with %run
This patch does not do anything, but paves the way for future changes,
where %run command will be expanded into a script performing the testing
on device.

Differential Revision: https://reviews.llvm.org/D46553

llvm-svn: 332144
2018-05-11 21:29:51 +00:00
David Major 868b31bf21 [winasan] Update GetInstructionSize for Win10 1803
In Windows version 1803, the first instruction of ntdll!strchr is:
8a01 mov al,byte ptr [rcx]

This is the only needed change for this version as far as I can tell.

Differential Revision: https://reviews.llvm.org/D46458

llvm-svn: 332095
2018-05-11 14:31:14 +00:00
David Major 49fcf17d5f [ASan] Fix range check in AddrIsInHighShadow
This appears to be a copy/paste artifact from `AddrIsInHighMem`. It was caught by Firefox's jit-tests on Win64.

Differential Revision: https://reviews.llvm.org/D46291

llvm-svn: 332092
2018-05-11 14:23:39 +00:00
Kostya Serebryany 6a6e690d24 [libFuzzer] refactor the implementation of -print_coverage
llvm-svn: 332073
2018-05-11 01:17:52 +00:00
Kamil Rytarowski 65e9f1f2c9 Enable SANITIZER_INTERCEPTOR_HOOKS for NetBSD
Summary:
This feature is required for proper libFuzzer support.

Adding SI_NETBSD to the list of OSes, fixes breakage in several
libFuzzer tests. It has been debugged with aid from kcc@.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46729

llvm-svn: 332051
2018-05-10 22:09:24 +00:00
Kuba Mracek 2903a9bb02 [tsan] Add debugging API to retrieve the "external tag" from reports
Differential Revision: https://reviews.llvm.org/D46661

llvm-svn: 332048
2018-05-10 21:46:00 +00:00
Walter Lee ca804caed0 [asan] Enable memtrinsics interception for RTEMS
Replace decltype(memcpy) with decltype(__asan_memcpy) because memcpy
has not been defined in any headers on RTEMS.  Similarly for memmove
and memset.

Differential Revision: https://reviews.llvm.org/D46625

llvm-svn: 332047
2018-05-10 21:45:13 +00:00
Walter Lee 603c7ae232 [sanitizer] Port fast stack unwinder to sparcv8
Differential Revision: https://reviews.llvm.org/D46469

llvm-svn: 332046
2018-05-10 21:40:16 +00:00
Kostya Serebryany 5d95f2782d [libFuzzer] simplify tests, remove one redundant test; NFC
llvm-svn: 332037
2018-05-10 20:37:08 +00:00
Kostya Serebryany d80e821646 [libFuzzer] remove the dump_coverage flag, it hasn't been working with the inline sanitizer coverage anyway
llvm-svn: 332036
2018-05-10 20:24:39 +00:00
Kamil Rytarowski 9788e60736 Setup ORIGIN/NetBSD option in sanitizer tests
Summary:
NetBSD can use the approach that exists in FreeBSD, Linux and SunOS.

Pick the FreeBSD one as marking programs with "-z origin" is useful.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: emaste, fedor.sergeev, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46718

llvm-svn: 332035
2018-05-10 20:13:36 +00:00
Kostya Serebryany 59bf1e8b36 [libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal is weaker than the SanitizerCoverage
llvm-svn: 332034
2018-05-10 20:12:15 +00:00
Walter Lee 3bde962255 [asan] Initialize fake stack during thread init
If detect-stack-use-after-return is on, initialize fake stack during
AsanThread::Init(), rather than lazily.  This is required on Myriad.
From kcc: "There used to be a reason why this was done lazily, but I
don't remember if we still have that reason."  Tested on x86.

Differential Revision: https://reviews.llvm.org/D46626

llvm-svn: 332033
2018-05-10 20:09:03 +00:00
Kostya Serebryany f489e2bfef [libFuzzer] Experimental data flow tracer for fuzz targets.
Summary:
Experimental data flow tracer for fuzz targets.
Allows to tell which bytes of the input affect which functions of the fuzz target.

We previously attempted to use DFSan directly in the libFuzzer process,
and that didn't work nicely.
Now we will try to collect the data flow information for the seed corpus
in a separate process (using this tracer), and then use it in the regular libFuzzer runs.

Reviewers: morehouse, pcc, Dor1s

Reviewed By: morehouse, Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46666

llvm-svn: 332029
2018-05-10 19:59:01 +00:00
Kamil Rytarowski b8238db4c4 Register NetBSD/i386 in asan_mapping.h
Summary:
Introduce kNetBSD_ShadowOffset32 and document
NetBSD/i386 (hosted on amd64 kernel) process virtual
address space ranges.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D46708

llvm-svn: 332027
2018-05-10 19:51:40 +00:00
Peter Wu c58988a908 [lsan] Try to fix test failure due to compiler optimization
Summary:
The SanitizerCommon-lsan-x86_64-Linux test failed due to the address of
the very first allocation ending up in the stack through "delete[]".
Workaround this by performing another allocation. The issue was only
present with optimization enabled, the test would pass with -O0.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46650

llvm-svn: 332020
2018-05-10 19:02:32 +00:00
Kamil Rytarowski aee7b0f209 wrong usages of sem_open in the libFuzzer
Summary:
Fixed two non-standard usages of sem_open in the libFuzzer library and
one NetBSD-related modification with test script.


  - The return value to indicate error should be SEM_FAILED instead of
    (void *)-1 (please refer to "RETURN VALUE" section in this [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | page ]]). Actually, SEM_FAILED != (void *)-1 holds in NetBSD.

  - The SharedMemoryRegion::SemName function should return name
    starting with slash. Because the behaviour of name which does not
    start with slash is unspecified as the [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | "DESCRIPTION" section ]] specified:

> If name does not begin with the <slash> character, the effect is implementation-defined.

  - The length of name is limited to 14 in NetBSD, it is suggested to
    reduce the length of equivalence server name in the test script.

Patch by: Yang Zheng

Reviewers: vitalybuka, krytarowski, kcc

Reviewed By: kcc

Subscribers: kcc, #sanitizers, llvm-commits, joerg

Differential Revision: https://reviews.llvm.org/D46622

llvm-svn: 332003
2018-05-10 17:31:06 +00:00
Vitaly Buka 327f5f3a92 [sanitizer] Attempt to fix strace_test.cc on ppc64le
llvm-svn: 331964
2018-05-10 08:16:23 +00:00
Vitaly Buka ebbc754717 [sanitizer] Fix typo in comment
llvm-svn: 331956
2018-05-10 04:21:41 +00:00
Vitaly Buka 5dcc94c177 [sanitizer] Renamed local variable
llvm-svn: 331955
2018-05-10 04:16:44 +00:00
Vitaly Buka edad575a35 [sanitizer] Fix compilation after invalid rebase
llvm-svn: 331954
2018-05-10 04:10:09 +00:00
Vitaly Buka e0c6eadef7 [sanitizer] Don't miss threads by ThreadSuspender
Summary:
Enumerating /proc/<pid>/task/ dir Linux may stop if thread is dead. In this case
we miss some alive threads and can report false memory leaks.
To solve this issue we repeat enumeration if the last thread is dead.
Do detect dead threads same way as proc_task_readdir we use
/proc/<pid>/task/<tid>/status.

Similarly it also ends enumeration of if proc_fill_cache fails, but in this case
Linux sets inode to 1 (Bad block).

And just in case re-list threads if we had to call internal_getdents more than
twice or result takes more than half of the buffer.

Reviewers: eugenis, dvyukov, glider

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46517

llvm-svn: 331953
2018-05-10 04:02:59 +00:00
Kostya Serebryany d790effacb [libFuzzer] add a simple puzzle that is difficult for today's libFuzzer
llvm-svn: 331951
2018-05-10 02:02:41 +00:00
Vitaly Buka c779388c7b [sanitizer] Use all available rounded up capacity
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46617

llvm-svn: 331940
2018-05-09 23:31:05 +00:00
Vitaly Buka ed43f18b55 [lsan] Report unsuspended threads
Summary:
Leak checker needs to suspend all process threads. If we have some running
thread in registry but not suspended we can have false leak report. So we will
report this case here for future debugging.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46663

llvm-svn: 331936
2018-05-09 23:02:14 +00:00
Vitaly Buka cf50425b77 [sanitizer] Fix argument type and remove unneeded vector resize
llvm-svn: 331927
2018-05-09 22:03:52 +00:00
Petr Hosek 1608ca6d64 [CMake] Build shared version of runtimes for Fuchsia
Fuchsia is no longer treated as UNIX which means we need to explicitly
enable building of shared versions of runtimes.

Differential Revision: https://reviews.llvm.org/D46609

llvm-svn: 331922
2018-05-09 21:24:06 +00:00
Vitaly Buka 34794a9669 [sanitizer] Use tid_t in ThreadLister
llvm-svn: 331921
2018-05-09 21:21:26 +00:00
Vitaly Buka d3e55bf7fc [sanitizer] Cleanup sorting functions
llvm-svn: 331915
2018-05-09 20:42:11 +00:00
Kostya Serebryany 73634e4037 [dfsan] add one more sanitizer-coverage hook to the whitelist
llvm-svn: 331908
2018-05-09 18:35:09 +00:00
Matt Morehouse 0a4d213176 [libFuzzer] Disable print-func.test on Darwin.
The try-lock guard change seems to be making this test fail on Mac, but
I haven't been able to reproduce the failure.  Disabling the test on Mac
to fix build bot.

llvm-svn: 331894
2018-05-09 17:05:07 +00:00
Kostya Kortchinsky cb7912cc0f [sanitizer] Correct 64-bit atomic_store on 32-bit "other" platforms
Summary:
I think there might be something to optimize in `atomic_store`.
Currently, if everything goes well (and we have a different new value), we
always iterate 3 times.
For example, `with a = 0`, `oldval = a`, `newval = 42`, we get:
```
oldval = 0, newval = 42, curval = 0  
oldval = 0, newval = 42, curval = 42 
oldval = 42, newval = 42, curval = 42
```
and then it breaks.

Unless I am not seeing something, I don't see a point to the third iteration.
If the current value is the one we want, we should just break.
This means that 2 iterations (with a different newval) should be sufficient to
achieve what we want.

Reviewers: dvyukov, alekseyshl

Reviewed By: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46597

llvm-svn: 331890
2018-05-09 16:20:52 +00:00
Sid Manning ed3065f7a1 Add basic compiler-rt builtins support for hexagon.
Differential Revision: https://reviews.llvm.org/D46364

llvm-svn: 331881
2018-05-09 14:44:54 +00:00
Vitaly Buka fb663789d3 [sanitizer] Remove unneeded blank lines
llvm-svn: 331831
2018-05-09 00:44:26 +00:00
Vitaly Buka 0590a5bd4b [sanitizer] Update .clang-format in compiler-rt
Historically style is Google, but we never used AllowShortIfStatementsOnASingleLine.

llvm-svn: 331829
2018-05-09 00:41:53 +00:00
Matt Morehouse 14cf71a3a5 [libFuzzer] Guard symbolization with try-lock.
Summary:
When out-of-memory or timeout occurs, threads can be stopped during
symbolization, thereby causing a deadlock when the OOM/TO handlers
attempt symbolization.  We avoid this deadlock by skipping symbolization
if another thread is symbolizing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46605

llvm-svn: 331825
2018-05-08 23:45:05 +00:00
Vitaly Buka e21fbe5af2 [sanitizer] Read file to InternalMmapVectorNoCtor
llvm-svn: 331791
2018-05-08 18:35:11 +00:00
Vitaly Buka 6b13684d15 [sanitizer] Close fd on ReadFromFile error
llvm-svn: 331789
2018-05-08 18:32:53 +00:00
Vitaly Buka 7381f26b45 [sanitizer] Fix InternalMmapVectorNoCtor reserve and resize
Remap on reserve of more than the current size.
Don't remap on downsize.

llvm-svn: 331784
2018-05-08 17:59:44 +00:00
Walter Lee 47bc8f1239 [asan] Fix bug introduced by r331647
unmap_shadow_on_exit was inadvertently flipped for non-RTEMS.

llvm-svn: 331737
2018-05-08 05:58:57 +00:00
Vitaly Buka 4e76d7fcb6 [sanitizer] Add InternalMmapVector::swap
llvm-svn: 331736
2018-05-08 04:57:08 +00:00
Vitaly Buka 4199dbd633 [sanitizer] Fix error checking in ThreadLister
llvm-svn: 331735
2018-05-08 04:56:57 +00:00
Igor Kudrin 47db4e6e5b Revert "[sanitizer] Be more accurate when calculating the previous instruction address on ARM."
This reverts commit r331626 because it causes build bot failures:
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/5069

Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 331734
2018-05-08 04:39:00 +00:00
Dean Michael Berris afdaaba272 [XRay][compiler-rt] Add llvm-mca assembler annotations for XRay (NFC)
This change adds some assembler comments to facilitate analysis with
llvm-mca. In particular, we're interested in identifying and later
optimising (reducing) the cost of the key functions in the XRay
implementation using both static analysis (with llvm-mca, etc.) and
dynamic analysis (perf profiling, etc.) of microbenchmarks.

llvm-svn: 331711
2018-05-08 01:57:51 +00:00
Kostya Serebryany 356e584d91 [dfsan] update the abilist for the sanitizer coverage; also remove the fuzz target from this list
llvm-svn: 331710
2018-05-08 01:52:40 +00:00
Vitaly Buka e2953dcbf7 [sanitizer] Simplify ThreadLister interface
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46516

llvm-svn: 331701
2018-05-07 23:29:48 +00:00
Kostya Kortchinsky 440d76c559 [sanitizer] s/TestOnlyInit/Init for the allocator ByteMap (NFC)
Summary:
The `TestOnlyInit` function of `{Flat,TwoLevel}ByteMap` seems to be a misnomer
since the function is used outside of tests as well, namely in
`SizeClassAllocator32::Init`. Rename it to `Init` and update the callers.

Reviewers: alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46408

llvm-svn: 331662
2018-05-07 19:02:19 +00:00
Walter Lee 7b24aea150 [asan] Port asan_malloc_linux.cc to RTEMS
We reuse the allocation interceptors as is.  RTEMS doesn't support
dlsyms.  However, it needs to handle memory allocation requests before
the ASan run-time has been initialized.  We use the dlsym alloc pool
for this purpose, and we increase its size to 4k to support this
usage.

Differential Revision: https://reviews.llvm.org/D46465

llvm-svn: 331649
2018-05-07 16:39:09 +00:00
Walter Lee 5db43c66b7 [sanitizer] On RTEMS, avoid recursion when reporting Mmap failure
Differential Revision: https://reviews.llvm.org/D46463

llvm-svn: 331648
2018-05-07 16:38:45 +00:00
Walter Lee 483eb8640c [asan] Set flags appropriately for RTEMS
Disable both unmap_shadow_on_exit and protect_shadow_gap.

Differential Revision: https://reviews.llvm.org/D46461

llvm-svn: 331647
2018-05-07 16:38:20 +00:00
Walter Lee 29306b9f62 [sanitizer] Add definitions for Myriad RTEMS platform
Introduce two definitions to be used by the Myriad RTEMS port of the
ASan run-time: SANITIZER_MYRIAD2 for the platform and SANITIZER_RTEMS
for the OS.  We expect to use SANITIZER_MYRIAD2 to guard the portion
of the port corresponding to Myriad's unique memory map, and
SANITIZER_RTEMS for most of the rest.

Differential Revision: https://reviews.llvm.org/D46453

llvm-svn: 331646
2018-05-07 16:37:55 +00:00
Igor Kudrin 61de8a5aef [sanitizer] Be more accurate when calculating the previous instruction address on ARM.
Differential Revision: https://reviews.llvm.org/D46004

llvm-svn: 331626
2018-05-07 10:07:22 +00:00
Vitaly Buka fac095caba [sanitizer] Add operator== and operator!= for InternalMmapVectorNoCtor
llvm-svn: 331619
2018-05-07 06:14:12 +00:00
Vitaly Buka 2a20955169 [sanitizer] Replace InternalScopedBuffer with InternalMmapVector
llvm-svn: 331618
2018-05-07 05:56:36 +00:00
Vitaly Buka 44f55509d7 [sanitizer] Remove reserving constructor from InternalMmapVector
llvm-svn: 331617
2018-05-07 05:56:24 +00:00
Vitaly Buka db4241a046 [sanitizer] Implement InternalScopedBuffer with InternalMmapVector
llvm-svn: 331616
2018-05-07 05:56:12 +00:00
Vitaly Buka 20f49662f6 [sanitizer] Make InternalScopedBuffer::size() behavior similar to vector.
llvm-svn: 331612
2018-05-07 01:08:13 +00:00
Kamil Rytarowski bbe9e2a13c Remove dead sanitizer_procmaps_freebsd.cc
This file has been obsoleted by sanitizer_procmaps_bsd.cc.

llvm-svn: 331581
2018-05-05 03:57:07 +00:00
David Major af79e30ad5 Comment fix.
Test commit!

llvm-svn: 331523
2018-05-04 14:41:49 +00:00
Dean Michael Berris 47e6b8c2c7 [XRay][compiler-rt] Support string-based config for Basic mode.
Summary:
This addresses http://llvm.org/PR36790.

This change allows the XRay Basic Mode implementation to use the
string-based initialization routine provided through
`__xray_log_init_mode(...)`. In the process, we've also deprecated some
flags defined for the `XRAY_OPTIONS` environment variable.

We then introduce another environment variable that can control the XRay
Basic Mode implementation through `XRAY_BASIC_OPTIONS`.

We also rename files from `xray_inmemory_log` to `xray_basic_logging` to
be more in line with the mode implementation.

Depends on D46174.

Reviewers: echristo, kpw, pelikan, eizan

Reviewed By: kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46246

llvm-svn: 331507
2018-05-04 06:27:53 +00:00
Dean Michael Berris 5c389ace7d [XRay][compiler-rt] Support string-based config for FDR mode
Summary:
In this chage we add support for the string-based configuration
mechanism for configuring FDR mode.

We deprecate most of the `xray_fdr_log_*` flags that are set with the
`XRAY_OPTIONS` environment variable. Instead we make the FDR
implementation take defaults from the `XRAY_FDR_OPTIONS` environment
variable, and use the flags defined in `xray_fdr_flags.{h,cc,inc}` for
the options we support.

This change addresses http://llvm.org/PR36790.

Depends on D46173.

Reviewers: eizan, pelikan, kpw, echristo

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D46174

llvm-svn: 331506
2018-05-04 06:13:35 +00:00
Dean Michael Berris f0a6927932 [XRay][compiler-rt+docs] Introduce __xray_log_init_mode(...).
Summary:
This addresses http://llvm.org/PR36790.

The change Deprecates a number of functions and types in
`include/xray/xray_log_interface.h` to recommend using string-based
configuration of XRay through the __xray_log_init_mode(...) function. In
particular, this deprecates the following:

- `__xray_set_log_impl(...)` -- users should instead use the
`__xray_log_register_mode(...)` and `__xray_log_select_mode(...)` APIs.

- `__xray_log_init(...)` -- users should instead use the
`__xray_log_init_mode(...)` function, which also requires using the
`__xray_log_register_mode(...)` and `__xray_log_select_mode(...)`
functionality.

- `__xray::FDRLoggingOptions` -- in following patches, we'll be
migrating the FDR logging implementations (and tests) to use the
string-based configuration. In later stages we'll remove the
`__xray::FDRLoggingOptions` type, and ask users to migrate to using the
string-based configuration mechanism instead.

- `__xray::BasicLoggingOptions` -- same as `__xray::FDRLoggingOptions`,
we'll be removing this type later and instead rely exclusively on the
string-based configuration API.

We also update the documentation to reflect the new advice and remove
some of the deprecated notes.

Reviewers: eizan, kpw, echristo, pelikan

Reviewed By: kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46173

llvm-svn: 331503
2018-05-04 06:01:12 +00:00
Volodymyr Sapsai 3835320617 Revert "Follow-up to r331378. Update tests to allow to use C atomics in C++."
It reverts commit r331484 because it caused test failures

    ThreadSanitizer-x86_64 :: Darwin/gcd-groups-destructor.mm
    ThreadSanitizer-x86_64 :: Darwin/libcxx-shared-ptr-stress.mm
    ThreadSanitizer-x86_64 :: Darwin/xpc-race.mm

Foundation.h transitively includes <atomic>, so we have a case of benign mixing
<stdatomic.h> and <atomic>.

llvm-svn: 331491
2018-05-03 23:06:37 +00:00
Volodymyr Sapsai 6e713d9866 Follow-up to r331378. Update tests to allow to use C atomics in C++.
Reviewers: kubamracek

Reviewed By: kubamracek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D46363

llvm-svn: 331484
2018-05-03 21:30:47 +00:00
Kostya Kortchinsky 3a9245481f [sanitizer] Remove unused 32-bit allocator TransferBatch parameter
Summary:
NFC. Remove an unused parameter in
`SizeClassAllocator32::TransferBatch::SetFromArray`, and thus get rid of the
compilation warning.

Reviewers: alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46397

llvm-svn: 331477
2018-05-03 19:14:07 +00:00
Petr Hosek 716d9949f6 [sanitizer] Fix Fuchsia ReadBinaryName not to crash when uninitialized
If the sanitizer runtime is loaded in a binary that doesn't really
support it, then __sanitizer_startup_hook will never have been
called to initialize StoredArgv. This case can't be supported, but
its failure mode shouldn't be to crash in sanitizer_common internals.

Patch By: mcgrathr

Differential Revision: https://reviews.llvm.org/D46344

llvm-svn: 331382
2018-05-02 18:08:47 +00:00
Matt Morehouse 7764a04af0 [libFuzzer] Don't short-circuit from CrashCallback.
Short-circuiting causes tests to fail on Mac since libFuzzer crashes
rather than exiting with an error code when an unexpected signal
happens.

llvm-svn: 331324
2018-05-02 02:55:28 +00:00
Matt Morehouse 52fd169035 [libFuzzer] Report at most one crash per input.
Summary:
Fixes https://github.com/google/sanitizers/issues/788/, a deadlock
caused by multiple crashes happening at the same time.  Before printing
a crash report, we now test and set an atomic flag.  If the flag was
already set, the crash handler returns immediately.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46277

llvm-svn: 331310
2018-05-01 21:01:53 +00:00
Roman Lebedev e8e95b5b01 [compiler-rt][X86][AMD][Bulldozer] Fix Bulldozer Model 2 detection.
Summary:
The compiler-rt side of D46314

I have discovered an issue by accident.
```
$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          21
Model:               2
Model name:          AMD FX(tm)-8350 Eight-Core Processor
Stepping:            0
CPU MHz:             3584.018
CPU max MHz:         4000.0000
CPU min MHz:         1400.0000
BogoMIPS:            8027.22
Virtualization:      AMD-V
L1d cache:           16K
L1i cache:           64K
L2 cache:            2048K
L3 cache:            8192K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb cpb hw_pstate vmmcall bmi1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold
```
So this is model-2 bulldozer AMD CPU.

GCC agrees:
```
$ echo | gcc -E - -march=native -###
<...>
 /usr/lib/gcc/x86_64-linux-gnu/7/cc1 -E -quiet -imultiarch x86_64-linux-gnu - "-march=bdver2" -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -mno-movbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mlwp -mfma -mfma4 -mxop -mbmi -mno-sgx -mno-bmi2 -mtbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mf16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mxsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps -mno-avx5124vnniw -mno-clwb -mno-mwaitx -mno-clzero -mno-pku -mno-rdpid --param "l1-cache-size=16" --param "l1-cache-line-size=64" --param "l2-cache-size=2048" "-mtune=bdver2"
<...>
```

But clang does not: (look for `bdver1`)
```
$ echo | clang -E - -march=native -###
clang version 7.0.0- (trunk)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
 "/usr/lib/llvm-7/bin/clang" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-E" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "-" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-fuse-init-array" "-target-cpu" "bdver1" "-target-feature" "+sse2" "-target-feature" "+cx16" "-target-feature" "+sahf" "-target-feature" "+tbm" "-target-feature" "-avx512ifma" "-target-feature" "-sha" "-target-feature" "-gfni" "-target-feature" "+fma4" "-target-feature" "-vpclmulqdq" "-target-feature" "+prfchw" "-target-feature" "-bmi2" "-target-feature" "-cldemote" "-target-feature" "-fsgsbase" "-target-feature" "-xsavec" "-target-feature" "+popcnt" "-target-feature" "+aes" "-target-feature" "-avx512bitalg" "-target-feature" "-xsaves" "-target-feature" "-avx512er" "-target-feature" "-avx512vnni" "-target-feature" "-avx512vpopcntdq" "-target-feature" "-clwb" "-target-feature" "-avx512f" "-target-feature" "-clzero" "-target-feature" "-pku" "-target-feature" "+mmx" "-target-feature" "+lwp" "-target-feature" "-rdpid" "-target-feature" "+xop" "-target-feature" "-rdseed" "-target-feature" "-waitpkg" "-target-feature" "-ibt" "-target-feature" "+sse4a" "-target-feature" "-avx512bw" "-target-feature" "-clflushopt" "-target-feature" "+xsave" "-target-feature" "-avx512vbmi2" "-target-feature" "-avx512vl" "-target-feature" "-avx512cd" "-target-feature" "+avx" "-target-feature" "-vaes" "-target-feature" "-rtm" "-target-feature" "+fma" "-target-feature" "+bmi" "-target-feature" "-rdrnd" "-target-feature" "-mwaitx" "-target-feature" "+sse4.1" "-target-feature" "+sse4.2" "-target-feature" "-avx2" "-target-feature" "-wbnoinvd" "-target-feature" "+sse" "-target-feature" "+lzcnt" "-target-feature" "+pclmul" "-target-feature" "-prefetchwt1" "-target-feature" "+f16c" "-target-feature" "+ssse3" "-target-feature" "-sgx" "-target-feature" "-shstk" "-target-feature" "+cmov" "-target-feature" "-avx512vbmi" "-target-feature" "-movbe" "-target-feature" "-xsaveopt" "-target-feature" "-avx512dq" "-target-feature" "-adx" "-target-feature" "-avx512pf" "-target-feature" "+sse3" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/usr/lib/llvm-7/lib/clang/7.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/lib/llvm-7/lib/clang/7.0.0/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/build/llvm-build-Clang-release" "-ferror-limit" "19" "-fmessage-length" "271" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "-" "-x" "c" "-"
```

So clang, unlike gcc, considers this to be `bdver1`.

After some digging, i've come across `getAMDProcessorTypeAndSubtype()` in `Host.cpp`.
I have added the following debug printf after the call to that function in `sys::getHostCPUName()`:
```
errs() << "Family " << Family << " Model " << Model << " Type " << Type "\n";
```
Which produced:
```
Family 21 Model 2 Type 5
```
Which matches the `lscpu` output.

As it was pointed in the review by @craig.topper:
>>! In D46314#1084123, @craig.topper wrote:
> I dont' think this is right. Here is what I found on wikipedia. https://en.wikipedia.org/wiki/List_of_AMD_CPU_microarchitectures.
>
> AMD Bulldozer Family 15h - the successor of 10h/K10. Bulldozer is designed for processors in the 10 to 220W category, implementing XOP, FMA4 and CVT16 instruction sets. Orochi was the first design which implemented it. For Bulldozer, CPUID model numbers are 00h and 01h.
> AMD Piledriver Family 15h (2nd-gen) - successor to Bulldozer. CPUID model numbers are 02h (earliest "Vishera" Piledrivers) and 10h-1Fh.
> AMD Steamroller Family 15h (3rd-gen) - third-generation Bulldozer derived core. CPUID model numbers are 30h-3Fh.
> AMD Excavator Family 15h (4th-gen) - fourth-generation Bulldozer derived core. CPUID model numbers are 60h-6Fh, later updated revisions have model numbers 70h-7Fh.
>
>
> So there's a weird exception where model 2 should go with 0x10-0x1f.

Though It does not help that the code can't be tested at the moment.
With this logical change, the `bdver2` is properly detected.
```
$ echo | /build/llvm-build-Clang-release/bin/clang -E - -march=native -###
clang version 7.0.0 (trunk 331249) (llvm/trunk 331256)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /build/llvm-build-Clang-release/bin
 "/build/llvm-build-Clang-release/bin/clang-7" "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-E" "-disable-free" "-main-file-name" "-" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-fuse-init-array" "-target-cpu" "bdver2" "-target-feature" "+sse2" "-target-feature" "+cx16" "-target-feature" "+sahf" "-target-feature" "+tbm" "-target-feature" "-avx512ifma" "-target-feature" "-sha" "-target-feature" "-gfni" "-target-feature" "+fma4" "-target-feature" "-vpclmulqdq" "-target-feature" "+prfchw" "-target-feature" "-bmi2" "-target-feature" "-cldemote" "-target-feature" "-fsgsbase" "-target-feature" "-xsavec" "-target-feature" "+popcnt" "-target-feature" "+aes" "-target-feature" "-avx512bitalg" "-target-feature" "-movdiri" "-target-feature" "-xsaves" "-target-feature" "-avx512er" "-target-feature" "-avx512vnni" "-target-feature" "-avx512vpopcntdq" "-target-feature" "-clwb" "-target-feature" "-avx512f" "-target-feature" "-clzero" "-target-feature" "-pku" "-target-feature" "+mmx" "-target-feature" "+lwp" "-target-feature" "-rdpid" "-target-feature" "+xop" "-target-feature" "-rdseed" "-target-feature" "-waitpkg" "-target-feature" "-movdir64b" "-target-feature" "-ibt" "-target-feature" "+sse4a" "-target-feature" "-avx512bw" "-target-feature" "-clflushopt" "-target-feature" "+xsave" "-target-feature" "-avx512vbmi2" "-target-feature" "-avx512vl" "-target-feature" "-avx512cd" "-target-feature" "+avx" "-target-feature" "-vaes" "-target-feature" "-rtm" "-target-feature" "+fma" "-target-feature" "+bmi" "-target-feature" "-rdrnd" "-target-feature" "-mwaitx" "-target-feature" "+sse4.1" "-target-feature" "+sse4.2" "-target-feature" "-avx2" "-target-feature" "-wbnoinvd" "-target-feature" "+sse" "-target-feature" "+lzcnt" "-target-feature" "+pclmul" "-target-feature" "-prefetchwt1" "-target-feature" "+f16c" "-target-feature" "+ssse3" "-target-feature" "-sgx" "-target-feature" "-shstk" "-target-feature" "+cmov" "-target-feature" "-avx512vbmi" "-target-feature" "-movbe" "-target-feature" "-xsaveopt" "-target-feature" "-avx512dq" "-target-feature" "-adx" "-target-feature" "-avx512pf" "-target-feature" "+sse3" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/build/llvm-build-Clang-release/lib/clang/7.0.0" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/build/llvm-build-Clang-release/lib/clang/7.0.0/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/build/llvm-build-Clang-release" "-ferror-limit" "19" "-fmessage-length" "271" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "-" "-x" "c" "-"
```

Reviewers: craig.topper, asbirlea, rnk, GGanesh, andreadb

Reviewed By: craig.topper

Subscribers: sdardis, dberris, aprantl, arichardson, JDevlieghere, #sanitizers, llvm-commits, cfe-commits, craig.topper

Differential Revision: https://reviews.llvm.org/D46323

llvm-svn: 331295
2018-05-01 18:40:15 +00:00
Dmitry Vyukov 6c33998ce1 tsan: disable trace switching after multithreaded fork
The problem is reported in:
https://github.com/google/sanitizers/issues/945

We already disable as much as possible after multithreaded fork,
trace switching is last place that can hang due to basic
operations (memory accesses, function calls).
Disable it too.

llvm-svn: 331163
2018-04-30 07:28:45 +00:00
Dean Michael Berris 26e81209ef [XRay][profiler] Part 1: XRay Allocator and Array Implementations
Summary:
This change is part of the larger XRay Profiling Mode effort.

Here we implement an arena allocator, for fixed sized buffers used in a
segmented array implementation. This change adds the segmented array
data structure, which relies on the allocator to provide and maintain
the storage for the segmented array.

Key features of the `Allocator` type:

*  It uses cache-aligned blocks, intended to host the actual data. These
   blocks are cache-line-size multiples of contiguous bytes.

*  The `Allocator` has a maximum memory budget, set at construction
   time. This allows us to cap the amount of data each specific
   `Allocator` instance is responsible for.

*  Upon destruction, the `Allocator` will clean up the storage it's
   used, handing it back to the internal allocator used in
   sanitizer_common.

Key features of the `Array` type:

*  Each segmented array is always backed by an `Allocator`, which is
   either user-provided or uses a global allocator.

*  When an `Array` grows, it grows by appending a segment that's
   fixed-sized. The size of each segment is computed by the number of
   elements of type `T` that can fit into cache line multiples.

*  An `Array` does not return memory to the `Allocator`, but it can keep
   track of the current number of "live" objects it stores.

*  When an `Array` is destroyed, it will not return memory to the
   `Allocator`. Users should clean up the `Allocator` independently of
   the `Array`.

*  The `Array` type keeps a freelist of the chunks it's used before, so
   that trimming and growing will re-use previously allocated chunks.

These basic data structures are used by the XRay Profiling Mode
implementation to implement efficient and cache-aware storage for data
that's typically read-and-write heavy for tracking latency information.
We're relying on the cache line characteristics of the architecture to
provide us good data isolation and cache friendliness, when we're
performing operations like searching for elements and/or updating data
hosted in these cache lines.

Reviewers: echristo, pelikan, kpw

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D45756

llvm-svn: 331141
2018-04-29 13:46:30 +00:00
Dmitry Vyukov 21dc68fe7b tsan: improve "destroy of a locked mutex" reports
1. Allow to suppress by current stack.
We generally allow to suppress by all main stacks.
Current is probably the stack one wants to use to
suppress such reports.

2. Fix last lock stack restoration.
We trimmed shadow value by storing it in u32.
This magically worked for the test that provoked
the report on the main thread. But this breaks
for locks in any other threads.

llvm-svn: 331023
2018-04-27 08:59:35 +00:00
Petr Hosek 5d3df3cf44 Revert "[CMake] Make simple source used for CMake checks a C file"
This reverts commit r331003 which breaks sanitizer bots because of
missing 32-bit libstdc++ library.

llvm-svn: 331009
2018-04-27 01:25:32 +00:00
Petr Hosek 887fbc61d4 [CMake] Make simple source used for CMake checks a C file
The source being compiled is plain C, but using .cc extension forces it
to be compiled as C++ which requires a working C++ compiler including
C++ library which may not be the case when we're building compiler-rt
together with libcxx as part of runtimes build.

Differential Revision: https://reviews.llvm.org/D46152

llvm-svn: 331003
2018-04-26 23:35:13 +00:00
Reid Kleckner 824eb0e6a0 [asan] Align __asan_global_start so that it works with LLD
Otherwise LLD will not align the .ASAN$GA section start, and
&__asan_globals + 1 will not be the start of the next real ASan global
metadata in .ASAN$GL.

We discovered this issue when attempting to use LLD on Windows in
Chromium: https://crbug.com/837090

llvm-svn: 330990
2018-04-26 20:46:50 +00:00
Reid Kleckner 8e19bd45a9 Enable full debug info in the ASan runtime on Windows
Clang-cl supports the -gline-tables-only flag, so we were going down
that path. Honestly, we should just go ahead and enable full codeview
support.

llvm-svn: 330989
2018-04-26 20:34:19 +00:00
Petar Jovanovic 152060275f Add s390x to XFAIL for illegal_read/write_test.cc
Follow up to r330840 and r330849.
It seems that s390 is also not distinguishing illegal WRITE and READ memory
access.
Add s390x to XFAIL for the tests.

llvm-svn: 330863
2018-04-25 19:34:48 +00:00
Kostya Kortchinsky d8803d3d92 [scudo] Adding an interface function to print allocator stats
Summary:
This adds `__scudo_print_stats` as an interface function to display the Primary
and Secondary allocator statistics for Scudo.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46016

llvm-svn: 330857
2018-04-25 18:52:29 +00:00
Petar Jovanovic 3517ece21f Finetune supported arches for the tests added in r330840
r330840 introduced two tests that may not be supported on all architectures.
powerpc64 seems to be one of those.

llvm-svn: 330849
2018-04-25 17:34:30 +00:00
Petar Jovanovic 1dd3584ec0 [mips] Implement GetWriteFlag() for mips
The read/write flag is set by manually decoding the instruction that caused
the exception. It is implemented this way because the cause register which
contains the needed flag was removed from the signal context structure which
the user handler receives from the kernel.

Patch by Milos Stojanovic.

Differential Revision: https://reviews.llvm.org/D45768

llvm-svn: 330840
2018-04-25 16:21:00 +00:00
Richard Smith c1aaf8ebb7 Remove 'noexcept's that do not match between header and source file.
This code is ill-formed, but under -fno-exceptions compilers generally accept it (at least, prior to C++17). This allows this code to be built by Clang in C++17 mode.

llvm-svn: 330765
2018-04-24 20:33:37 +00:00
Kostya Kortchinsky efe5afbc3d [sanitizer] More dead code removal
Summary:
The following functions are only used in tests: `SetEnv`,
`SanitizerSetThreadName`, `SanitizerGetThreadName`. I don't think they are
going to be used in the future, and I propose to get rid of them, and associated
tests and include.

Reviewers: alekseyshl, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: dvyukov, vitalybuka, kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45838

llvm-svn: 330724
2018-04-24 14:58:10 +00:00
Rainer Orth 482cef6c3d The OpenBSD UBsan port introduced two typos that broke the Solaris sanitizer build.
The following patch restores it, will shortly commit as obvious.

Differential Revision: https://reviews.llvm.org/D46006

llvm-svn: 330712
2018-04-24 13:00:24 +00:00
Dan Liew dfd5a90a99 [LibFuzzer] Tweak `MutationDispatcher::Mutate_CopyPart` mutation.
It doesn't make sense to non-deterministically choose between
`CopyPart(..)` and `InsertPart(..)` when it is known that
`InsertPart(..)` will fail.

This upstream's a change from JFS solver's fork of LibFuzzer.

Differential Revision: https://reviews.llvm.org/D45693

llvm-svn: 330687
2018-04-24 06:31:09 +00:00
Alex Shlyapnikov cf30f6425a [HWASan] Use dynamic shadow memory on Android only.
There're issues with IFUNC support on other platforms.

Differential Revision: https://reviews.llvm.org/D45847

llvm-svn: 330666
2018-04-24 00:17:02 +00:00
Matt Morehouse f66221c6ec [libFuzzer] Add feature to not use AFL's deferred forkserver.
A small but substantial minority of libFuzzer-based fuzzers run code that
does not play well with fork in global constructors or LLVMFuzzerInitialize.
This patch allows these fuzzers to use afl_driver by allowing them to
opt-out of using AFL's deferred forkserver which deferres calling fork until
after this code.

Patch By: metzman

Differential Revision: https://reviews.llvm.org/D45744

llvm-svn: 330652
2018-04-23 21:36:21 +00:00
Martin Liska d6af13e737 Change kAllocatorSpace for powerpc64.
Fixes issue: https://github.com/google/sanitizers/issues/933

Differential Revision: https://reviews.llvm.org/D45950

llvm-svn: 330650
2018-04-23 21:13:40 +00:00
Alex Shlyapnikov 88b7be0d13 [Sanitizer] Internal Printf string width + left-justify.
Summary:
Example:

  Printf("%-5s", "123");

should yield:

  '123  '

In case Printf's requested string field width is larger than the string
argument length, the resulting string should be padded up to the requested
width.

For the simplicity sake, implementing left-justified (right padding) only.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45906

llvm-svn: 330643
2018-04-23 20:30:50 +00:00
Alex Shlyapnikov d03fb0e3e0 [HWASan] Add files missing in r330624
llvm-svn: 330628
2018-04-23 19:05:12 +00:00
Alex Shlyapnikov 79866cad6f [HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt).
Retire the fixed shadow memory mapping to avoid conflicts with default
process memory mapping (currently manifests on Android).

Tests on AArch64 show <1% performance loss and code size increase,
making it possible to use dynamic shadow memory by default.

Keep the fixed shadow memory mapping around to be able to run
performance comparison tests later.

Re-commiting D45847 with fixed shadow for x86-64.

llvm-svn: 330624
2018-04-23 18:19:23 +00:00
Reid Kleckner 061fcd586d Revert r330474 - "[HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt)."
This commit causes internal errors with ld.bfd 2.24. My guess is that
the ifunc usage in this commit is causing problems. This is the default
system linker on Trusty Tahr, which is from 2014. I claim it's still in
our support window. Maybe we will decide to drop support for it, but
let's get the bots green while we do the investigation and have that
discussion.

Discovered here: https://crbug.com/835864

llvm-svn: 330619
2018-04-23 17:26:33 +00:00
Reid Kleckner 963aba3452 Fix clang-cl warnings in compiler-rt
The profile library was missing some includes and was erroneously using
ftruncate. WinASan was using `= {0}` to initialize structs, which
creates -Wmissing-field-initializers and -Wmissing-braces warnings with
clang. Use `= {}` instead, since this is C++.

llvm-svn: 330616
2018-04-23 17:05:47 +00:00
Kostya Kortchinsky 5a8bdc81d6 [scudo] Read ARM feature bits using Fuchsia APIs.
Summary:
    Fuchsia uses zx_system_get_features in lieu of getauxval.
    
    Use this call when checking for CRC32 support.

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: delcypher, llvm-commits, #sanitizers, kristof.beyls, chrib

Differential Revision: https://reviews.llvm.org/D45896

llvm-svn: 330598
2018-04-23 14:53:13 +00:00
Alex Shlyapnikov 8368bd82c3 Revert "[Sanitizer] Internal Printf string precision argument + padding."
This reverts commit r330458.

There are existing code using string precision as 'max len', need more
work.

llvm-svn: 330476
2018-04-20 20:24:02 +00:00
Alex Shlyapnikov c3ec1643f1 [HWASan] Introduce non-zero based and dynamic shadow memory (compiler-rt).
Summary:
Retire the fixed shadow memory mapping to avoid conflicts with default
process memory mapping (currently manifests on Android).

Tests on AArch64 show <1% performance loss and code size increase,
making it possible to use dynamic shadow memory by default.

For the simplicity and unifirmity sake, use dynamic shadow memory mapping
with base address accessed via ifunc resolver on all supported platforms.

Keep the fixed shadow memory mapping around to be able to run
performance comparison tests later.

Complementing D45840.

Reviewers: eugenis

Subscribers: srhines, kubamracek, dberris, mgorny, kristof.beyls, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45847

llvm-svn: 330474
2018-04-20 20:03:57 +00:00
Peter Collingbourne 7b17c9d05c asan: Mark printf-4.c as unsupported on Windows.
Although sprintf is not intercepted on Windows, this test can pass
if sprintf calls memmove, which is intercepted, so we can't XFAIL it.

Differential Revision: https://reviews.llvm.org/D45894

llvm-svn: 330469
2018-04-20 19:07:35 +00:00
Alex Shlyapnikov 5bb2345985 [Sanitizer] Internal Printf string precision argument + padding.
Summary:
Example:
  Printf("%.*s", 5, "123");
should yield:
  '123  '

In case Printf's requested string precision is larger than the string
argument, the resulting string should be padded up to the requested
precision.

For the simplicity sake, implementing right padding only.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45844

llvm-svn: 330458
2018-04-20 18:03:10 +00:00
Hans Wennborg 2473183c15 Revert r330376 "[sanitizer] Generalize atomic_uint8_t, atomic_uint16_t, ... into a template. NFC."
This broke the Windows build, see e.g. http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10130

> Differential Revision: https://reviews.llvm.org/D44246

llvm-svn: 330395
2018-04-20 07:34:59 +00:00
Dan Liew b7a3bc99c9 [LibFuzzer] Report when custom counters are available.
This upstreams a feature from the JFS solver's fork of LibFuzzer.

Differential Revision: https://reviews.llvm.org/D45675

llvm-svn: 330391
2018-04-20 06:46:19 +00:00
Dan Liew 25d0c65ff3 [LibFuzzer] Try to unbreak the `FuzzerMutate.ShuffleBytes1` unit test.
This test is failing on my Linux box. Just increasing the number of
iterations works around this. The divergence is likely due to
our reliance on `std::shuffle()` which is not guaranteed to have
the same behaviour across platforms.

This is a strong argument for us to implement our own shuffle
function to avoid divergence in behaviour across platforms.

Differential Revision: https://reviews.llvm.org/D45767

llvm-svn: 330390
2018-04-20 06:46:14 +00:00
Dan Liew c8879c989e [LibFuzzer] Unbreak the `trace-malloc-unbalanced.test` when using Python 3.
The `unbalanced_allocs.py` script uses Python 2 print statement
and `iteritems()`. Running `2to3` over the script fixes these.

Differential Revision: https://reviews.llvm.org/D45765

llvm-svn: 330389
2018-04-20 06:46:09 +00:00
Petr Hosek dd097445f0 [Fuzzer] Add a missing header in Fuchsia implementation
This is needed for fd_set.

Differential Revision: https://reviews.llvm.org/D45852

llvm-svn: 330383
2018-04-20 00:41:06 +00:00
Kuba Mracek 1707fa3374 [sanitizer] Generalize atomic_uint8_t, atomic_uint16_t, ... into a template. NFC.
Differential Revision: https://reviews.llvm.org/D44246

llvm-svn: 330376
2018-04-19 23:00:43 +00:00
Kostya Kortchinsky 46eab8d9fe [sanitizer] Minor refactor of some ReservedAddressRange functions
Summary:
Some of the functions had spurious conditional statements and checks, and some
intermediary variables that I feel made the code more complicated than it needs
to be. Also, when unmapping the whole range, the range size would be 0, but
the base was set to the address of the end of the range, which sounds prone to
error. I think nulling out the base in this scenario is a better way to go.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45775

llvm-svn: 330355
2018-04-19 18:38:15 +00:00
Petr Hosek 7a31c7ad23 [Fuzzer] Make InterruptHandler non-blocking for Fuchsia
The initial naive approach to simulate SIGINT on Fuchsia was to getchar
and look for ETX. This caused the InterruptHandler thread to lock stdin,
preventing musl's exit() from being able to close the stdio descriptors
and complete. This change uses select() instead.

Patch By: aarongreen

Differential Revision: https://reviews.llvm.org/D45636

llvm-svn: 330328
2018-04-19 14:01:46 +00:00
Dmitry Vyukov 4132b849e5 tsan: fix compiler warnings
vmaSize is uptr, so we need to print it with %zd.

llvm-svn: 330312
2018-04-19 07:42:08 +00:00
Alex Shlyapnikov 02f7841e4a [HWASan] Add "N" suffix to generic __hwasan_load/store.
Summary:
"N" suffix is added by the instrumentation and interface functions
are expected to be exported from the library as __hwasan_loadN* and
__hwasan_storeN*.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45739

llvm-svn: 330297
2018-04-18 22:05:18 +00:00
Kostya Kortchinsky eaeb64a814 [sanitizer] Minor refactor of ThreadDescriptorSize
Summary:
While I was sifting through dead code findings, I stumbled on this function.

First, for `__s390__` it always returned 0 for the 1st call, which doesn't seem
right. 2nd call & beyond would return the correct value though.
Then it duplicated the `atomic_store` multiple times, sometimes with a `if`,
sometimes without. Finally it used a capitalized variable name starting with `k`
which indicates a constant, and it is not.

So:
- rename the static global variable;
- change the atomic functions to their relaxed version;
- move the store to the end, and make sure we return `val` all the time.

Reviewers: alekseyshl, eugenis, koriakin

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45725

llvm-svn: 330268
2018-04-18 15:30:08 +00:00
Keith Wyss 557890126a [XRay][compiler-rt] Add noop patch functions for unsupported arches.
Summary:
Typed event patching is implemented for x86-64, but functions must
be defined for other arches.

Reviewers: dberris, pelikan

Subscribers: nemanjai, javed.absar, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45749

llvm-svn: 330231
2018-04-17 23:19:23 +00:00
Keith Wyss adb092e0ae Implement trampoline and handler for typed xray event tracing.
Summary:
Compiler-rt support first before defining the __xray_typedevent() lowering in
llvm. I'm looking for some early feedback before I touch much more code.

Reviewers: dberris

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43668

llvm-svn: 330218
2018-04-17 21:28:53 +00:00
Kostya Kortchinsky 54764ca235 [sanitizer] Remove low-hanging-fruit dead code
Summary:
Going through the dead code findings, the code removed in this CL appears to be
pretty straightforward to remove, and seems to be some leftover from previous
refactors.

Reviewers: alekseyshl, eugenis

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45704

llvm-svn: 330190
2018-04-17 14:54:50 +00:00
Kostya Kortchinsky 25eae5c879 [sanitizer] Fix OSX build failure post D45457
Summary:
It looks like OSX's UBSan needs a "NoHooks" version of
`RTSanitizerCommonSymbolizer` to work build properly.

Subscribers: kubamracek, mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45696

llvm-svn: 330146
2018-04-16 18:17:22 +00:00
Kostya Kortchinsky 754da0cc7e [sanitizer] Build failures fixes post D45457
Summary: Adding a couple missed RTSanitizerCommonSymbolizer in makefiles.

Subscribers: kubamracek, mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45694

llvm-svn: 330134
2018-04-16 16:58:34 +00:00
Kostya Kortchinsky 596b8b4a22 [sanitizer] Split Symbolizer/StackTraces from core RTSanitizerCommon
Summary:
Host symbolizer & stacktraces related code in their own RT:
`RTSanitizerCommonSymbolizer`, which is "libcdep" by nature. Symbolizer &
stacktraces specific code that used to live in common files is moved to a new
file `sanitizer_symbolizer_report.cc` as is.

The purpose of this is the enforce a separation between code that relies on
symbolization and code that doesn't. This saves the inclusion of spurious code
due to the interface functions with default visibility, and the extra data
associated.

The following sanitizers makefiles were modified & tested locally:
- dfsan: doesn't require the new symbolizer RT
- esan: requires it
- hwasan: requires it
- lsan: requires it
- msan: requires it
- safestack: doesn't require it
- xray: doesn't require it
- tsan: requires it
- ubsan: requires it
- ubsan_minimal: doesn't require it
- scudo: requires it (but not for Fuchsia that has a minimal runtime)

This was tested locally on Linux, Android, Fuchsia.

Reviewers: alekseyshl, eugenis, dberris, kubamracek, vitalybuka, dvyukov, mcgrathr

Reviewed By: alekseyshl, vitalybuka

Subscribers: srhines, kubamracek, mgorny, krytarowski, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45457

llvm-svn: 330131
2018-04-16 16:32:19 +00:00
Dmitry Vyukov 3a99c76e3d tsan: add support for linux/powerpc64 in buildgo.sh
The current implementation of the Go sanitizer only works on x86_64.
Added some modifications to the buildgo.sh script and the Tsan code
to make it work on powerpc64/linux.

Author: cseo (Carlos Eduardo Seo)
Reviewed in: https://reviews.llvm.org/D43025

llvm-svn: 330122
2018-04-16 11:43:00 +00:00
Petr Hosek 1ba8c8a2cc [Fuzzer] Avoid C++ compiler checks for fuzzer C++ library
When bulding libFuzzer as part of the toolchain, C++ library may not
have been fully built yet so the C++ compiler checks will fail, but we
don't care since we don't need C++ library anyway as we're building
our own.

Differential Revision: https://reviews.llvm.org/D45638

llvm-svn: 330075
2018-04-13 23:05:14 +00:00
Kostya Kortchinsky 4563b78b99 [sanitizer] Allow for the allocator "names" to be set by the tools
Summary:
In the same spirit of SanitizerToolName, allow the Primary & Secondary
allocators to have names that can be set by the tools via PrimaryAllocatorName
and SecondaryAllocatorName.

Additionally, set a non-default name for Scudo.

Reviewers: alekseyshl, vitalybuka

Reviewed By: alekseyshl, vitalybuka

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45600

llvm-svn: 330055
2018-04-13 19:21:27 +00:00
Vitaly Buka 753e99df6d [sanitizer] Fix __sanitizer::Vector::Resize vector
Implemented downsizing

Patch by Oleg Doronin

Reviewers: kcc, vitalybuka

Reviewed By: kcc, vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45605

llvm-svn: 330049
2018-04-13 18:41:52 +00:00
Vitaly Buka 63ee534ab1 [sanitizer] Replace cast with unsigned literals in tests
llvm-svn: 330048
2018-04-13 18:41:41 +00:00
Martin Pelikan f6b29caa5b [XRay] [compiler-rt] reformat and clarify CMakeLists.txt [NFC]
Summary:
- last change (+ the Apple support change) missed a lot of indentation
- shorten architecture SOURCES definitions as most fit 1 line/arch
- comment in English what's where, and where the different .a come from
  (using only the word "runtime" in the comment isn't useful, since the
  CMake primitive itself says "runtime" in its name)
- skip unsupported architectures quickly, to avoid extra indentation

Reviewers: dberris, eizan, kpw

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45568

llvm-svn: 329998
2018-04-13 08:33:46 +00:00
Kuba Mracek 296ce3bd23 [tsan] Add interceptors for objc_sync_enter and objc_sync_exit
Objective-C's @synchronize synchronization primitive uses calls to objc_sync_enter and objc_sync_exit runtime functions. In most cases, they end up just calling pthread_mutex_lock/pthread_mutex_unlock, but there are some cases where the synchronization from pthread_mutex_lock/pthread_mutex_unlock interceptors isn't enough. Let's add explicit interceptors for objc_sync_enter and objc_sync_exit to handle all cases.

Differential Revision: https://reviews.llvm.org/D45487

llvm-svn: 329982
2018-04-13 01:05:29 +00:00
Kuba Mracek 0fb14e944c [asan] Reduce flakiness in stack-overflow detection
IsStackOverflow only treats accesses within 512 bytes of SP as stack-overflow. This should really be the size of a page instead.

The scariness_score_test.cc triggers stack overflow with frames that are even larger than a page, which can also trigger a fault that will not be recognized as stack-overflow. Let's just use smaller frames.

llvm-svn: 329980
2018-04-13 00:29:24 +00:00
Evgeniy Stepanov 366ea7115d [asan] Remove malloc_context_size=0 from asan_device_setup.
This line was added in
  r243679 - [asan] Support arm64 devices in asan_device_setup
without any good reason.

llvm-svn: 329962
2018-04-12 22:06:18 +00:00
Kostya Kortchinsky bafba57252 [scudo] Remove duplicate cmake flags
Summary:
Now that common options are propagated again for runtimes build with D45507,
the -f{data,function}-sections flags are now duplicates, remove them.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45575

llvm-svn: 329925
2018-04-12 16:41:57 +00:00
Kostya Kortchinsky daca5ba7dd [sanitizer] Correct name length computation for some Fuchsia vmos
Summary:
This was missed during the review of D38595, but the vmo name size computation
should use internal_strlen, not sizeof, otherwise we end up with 7 character
names.

Reviewers: mcgrathr, flowerhack

Reviewed By: mcgrathr

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45525

llvm-svn: 329842
2018-04-11 18:55:26 +00:00
George Karpenkov 0fc4b7f429 [xray] Fix OS X bots.
OS X has "fat" executables which contain the code for all architectures.

llvm-svn: 329832
2018-04-11 18:00:09 +00:00
Aaron Smith 56e6e0d352 Revert "[cmake] Remove duplicate command line options from build"
The Android sanitizer buildbot is failing with this change and it
looks like an additional change to cmake is necessary to fix the 
build. Reverting this change for now.

llvm-svn: 329828
2018-04-11 17:31:18 +00:00
Aaron Smith 112d6760a8 [cmake] Remove duplicate command line options from build
CMAKE_CXX_FLAGS was added twice to the command line. This causes the command 
line options to be doubled which works until it doesn't as not all options 
can be specified twice. 

For example,

clang-cl foo.c /GS- /GS- -mllvm -small-loop-cost=1 -mllvm -small-loop-cost=1
clang (LLVM option parsing): for the -small-loop-cost option: may only occur zero or one times!

llvm-svn: 329817
2018-04-11 14:56:35 +00:00
Dean Michael Berris 6bb0157dca [XRay][compiler-rt] Fix osx-based builds
This is a follow-up to D45474.

llvm-svn: 329776
2018-04-11 05:16:11 +00:00
Dean Michael Berris 826e666cc7 [XRay][clang+compiler-rt] Support build-time mode selection
Summary:
This patch implements the `-fxray-modes=` flag which allows users
building with XRay instrumentation to decide which modes to pre-package
into the binary being linked. The default is the status quo, which will
link all the available modes.

For this to work we're also breaking apart the mode implementations
(xray-fdr and xray-basic) from the main xray runtime. This gives more
granular control of which modes are pre-packaged, and picked from
clang's invocation.

This fixes llvm.org/PR37066.

Note that in the future, we may change the default for clang to only
contain the profiling implementation under development in D44620, when
that implementation is ready.

Reviewers: echristo, eizan, chandlerc

Reviewed By: echristo

Subscribers: mgorny, mgrang, cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D45474

llvm-svn: 329772
2018-04-11 01:28:25 +00:00
Kostya Kortchinsky 141139e695 [sanitizer] Allow BackgroundThread to not depend on StackDepot v2
Summary:
This is a redo of D45296.
It looks like the random stack-protector issues I was getting were coming from
my Android emulator, and updating everything all around and relaunching stuff
ended up making it go away.
I guess I'll have to see how it behaves on the bots.
Only additional change from the previous CL is some `const` were appropriate.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45461

llvm-svn: 329706
2018-04-10 14:41:40 +00:00
Simon Dardis dfee530748 [compiler-rt][asan][mips] UnXFAIL some consistently passing tests
llvm-svn: 329688
2018-04-10 11:09:17 +00:00
Petr Hosek 00b9248cb3 [sanitizer] Enable threads for libc++ build used by the symbolizer
Disabling threads makes <atomic> unusable, but this is needed by LLVM
libraries that are dependencies of the symbolizer.

Differential Revision: https://reviews.llvm.org/D45424

llvm-svn: 329672
2018-04-10 05:54:36 +00:00
Vitaly Buka 3539c09d3b GetRandom / OpenBSD, using getentropy to get high quality randomness
Summary:
- getentropy presence since late 2014, safe to use.
- guarantees to delivers good random data up to 256 bytes.
- fall back to /dev/urandom as long the buffer is correct.

Patch by David CARLIER

Reviewers: kubamracek, vitalybuka

Reviewed By: vitalybuka

Subscribers: cryptoad, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44866

llvm-svn: 329633
2018-04-09 22:46:40 +00:00
Vitaly Buka 9802089e13 libFuzzer, OpenBSD support
Summary:
- Enabling libfuzzer on OpenBSD
- OpenBSD can t support asan, msan ... the tests can t be run.

Patch by David CARLIER

Reviewers: eugenis, phosek, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44877

llvm-svn: 329631
2018-04-09 22:38:26 +00:00
Peter Collingbourne 6662e9890b shadowcallstack: Make runtime tests compatible with aarch64.
Differential Revision: https://reviews.llvm.org/D45303

llvm-svn: 329614
2018-04-09 20:18:10 +00:00
Kostya Kortchinsky 09873b2bcf [sanitizer] Reverting rCRT329595
Summary:
Even this version seems to mess with Android somehow. Reverting for now while
I figure out what's up.

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45450

llvm-svn: 329606
2018-04-09 19:18:50 +00:00
Kostya Kortchinsky dd7c60c2d2 [sanitizer] Allow BackgroundThread to not depend on StackDepot
Summary:
Still pursuing the ultimate goal of splitting the Symbolizer code from
RTSanitizerCommon core, allow `BackgroundThread` to work even when not linked
with `sanitizer_stackdepot.cc`. There is no reason this function should pull in
the whole `StackDepot` if symbolization is not supported.

Currently this has no functional change as the depot is always linked anyway.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45296

llvm-svn: 329595
2018-04-09 17:25:57 +00:00
Alex Shlyapnikov 40c90bfba8 [ASan] NFC: make use of a new ErrorBase ctor
Summary:
Minor style changes to complement D44404:
- make use of a new ErrorBase ctor
- de-duplicate a comment about VS2013 support

Reviewers: eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45390

llvm-svn: 329586
2018-04-09 16:00:10 +00:00
Matt Morehouse fcc97b29c1 [libFuzzer] Print a correct error message when a directory can't be
opened.

Summary:
Currently if the directory cannot be opened for a reason other than
non-existence (e.g. too many open file descriptors) the error message
printed is incredibly confusing.

Patch By: Alex Gaynor

Reviewers: kcc, morehouse

Reviewed By: morehouse

Subscribers: delcypher, llvm-commits, Sanitizers

Differential Revision: https://reviews.llvm.org/D45322

llvm-svn: 329438
2018-04-06 18:15:24 +00:00
Simon Dardis a1b7447dfd [compiler-rt][dfsan][mips] UnXPASS a consistently passing test
llvm-svn: 329422
2018-04-06 17:03:36 +00:00
Kostya Kortchinsky 0716fb9376 [asan] Fix aligned_alloc test case
Summary:
Under some circumstances (that I haven't dug further into), the first stack
frame for the test looks like:
`#0 0x4e6038 in __interceptor_memalign.localalias.1 ...compiler-rt/lib/asan/asan_malloc_linux.cc:113`
which isn't matched by the current CHECK.

Expand the CHECK to match aligned_alloc or memalign. Hopefully this should fix
the PowerPC issue as well, otherwise we'll bring back the FIXME.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D45281

llvm-svn: 329226
2018-04-04 20:03:16 +00:00
Vlad Tsyrklevich 1781d105ed Add simple runtime tests for shadowcallstack
Summary:
ShadowCallStack does not yet have a runtime provided by compiler-rt, but
this change includes simple tests that make use of a very minimal
runtime in test/shadowcallstack/minimal_runtime.h

Reviewers: pcc, kcc, delcypher, eugenis, filcab

Reviewed By: pcc

Subscribers: kubamracek, mgorny, delcypher, llvm-commits, #sanitizers, kcc

Differential Revision: https://reviews.llvm.org/D44803

llvm-svn: 329210
2018-04-04 17:53:33 +00:00
Petr Hosek fb6a4a7907 [CMake] Support statically linked libc++abi and libunwind
This expands the CMake check to handle both statically and dynamically
linked version of libc++abi and libunwind and matches the implemnetation
used elsewhere in LLVM.

Differential Revision: https://reviews.llvm.org/D45242

llvm-svn: 329205
2018-04-04 16:27:33 +00:00
Kostya Kortchinsky 9336ab6996 [sanitizer] Split stacktrace/symbolizer: Windows unwind
Summary:
The purpose of this set of changes is to separate stackframe/symbolizer support into their own RT within sanitizer_common.
Sanitizers with no use for those could then be built without the extraneous dependencies pulled in by the default visibility interface functions.
I am aiming to do small changes for specific platforms.

In this one, we split the unwind functions from sanitizer_win.cc into their own sanitizer_unwind_win.cc.

Reviewers: alekseyshl, rnk

Reviewed By: alekseyshl, rnk

Subscribers: delcypher, #sanitizers, kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D44799

llvm-svn: 329204
2018-04-04 15:23:30 +00:00
Dean Michael Berris fe79b04a74 [XRay][compiler-rt] Build XRay runtime for OpenBSD
Summary:
This is D45125; the patch enables the build of XRay on OpenBSD. We also
introduce some OpenBSD specific changes to the runtime implementation,
involving how we get the TSC rate through the syscall interface specific
to OpenBSD.

Reviewers: dberris

Authored by: devnexen

Subscribers: dberris, mgorny, krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D45125

llvm-svn: 329189
2018-04-04 13:04:15 +00:00
Kostya Kortchinsky 2c5f944015 [sanitizer] Remove empty Symbolizer PrepareForSandboxing
Summary:
`Symbolizer::PrepareForSandboxing` is empty for all platforms and apparently
has been for a while (D10213). Remove it, and shuffle things around so that the
platform specific code is now in `PlatformPrepareForSandboxing`.

This allows to have one less symbolizer dependency in a common file, which
helps for the upcoming split.

Also remove `SymbolizerPrepareForSandboxing` in tsan_go which appears to not
be used anywhere.

Reviewers: alekseyshl, eugenis, dvyukov, mcgrathr

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44953

llvm-svn: 329094
2018-04-03 18:07:22 +00:00
Rong Xu 1a45e0cccc [profile] Move newly added test to the supported platform
Move test/profile/instrprof-value-merge.c from r328987 to Linux directory.

llvm-svn: 329016
2018-04-02 21:24:07 +00:00
Rong Xu 95ab7582f0 [profile] Fix value profile runtime merging issues
This patch fixes the following issues:
(1) The strong definition of the merge hook function was not working which
breaks the online value profile merging. This patch removes the weak
attribute of VPMergeHook and assigns the value dynamically.
(2) Truncate the proifle file so that we don't have garbage data at the end of
the file.
(3) Add new __llvm_profile_instrument_target_value() interface to do the value
profile update in batch. This is needed as the original incremental by 1
in __llvm_profile_instrument_target() is too slow for online merge.

Differential Revision: https://reviews.llvm.org/D44847

llvm-svn: 328987
2018-04-02 16:57:00 +00:00
Alex Shlyapnikov 0ee5a5577c [ASan] Disable new ASan error reporting tests on various ARMs.
As many other ASan tests already, has to disable these failing tests on
arm, armhf and aarch64 configs.

Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328849
2018-03-30 00:03:36 +00:00
Dan Liew 0f08dff1b5 [asan] Split the `throw_invoke_test.cc` into a Linux specific variant
and the general version to avoid use of libstdc++ on non-Linux
platforms.

This is motivated by the fact that using `libstdc++` is deprecated on
Darwin and maybe removed some day.

Differential Revision: https://reviews.llvm.org/D44733

llvm-svn: 328775
2018-03-29 13:53:50 +00:00
Alex Shlyapnikov b43b558012 [ASan] Disable aligned_alloc-alignment.cc on PPC64.
Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328753
2018-03-29 02:18:57 +00:00
Alex Shlyapnikov 239533319f [ASan] Disable aligned_alloc-alignment.cc on ppc64be
Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328742
2018-03-29 00:09:12 +00:00
Alex Shlyapnikov 5062d7a22d [ASan] Disable aligned_alloc-alignment.cc test on gcc.
This check "CHECK: {{#0 0x.* in .*aligned_alloc}}" fails on ppc64be, gcc
build. Disabling the test for gcc for now.

Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328741
2018-03-28 23:01:04 +00:00
Alex Shlyapnikov b276621d3d [ASan] Disable aligned_alloc-alignment.cc on Android.
Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328734
2018-03-28 22:00:08 +00:00
Alex Shlyapnikov 506b8388ed [ASan] Add aligned_alloc declaration to aligned_alloc-alignment.cc test.
aligned_alloc is not always defined in headers.

Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328726
2018-03-28 19:53:55 +00:00
Alex Shlyapnikov 10f50a44c1 [ASan] Report proper ASan error on allocator failures instead of CHECK(0)-ing
Summary:
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, ASan detailed errors were
defined and reported under the appropriate conditions.

Issue: https://github.com/google/sanitizers/issues/887

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328722
2018-03-28 18:22:40 +00:00
Evgeniy Stepanov 29e6fd6785 [asan] Add vfork to the list of reasons for stack errors.
Until we figure out what to do with it, vfork can cause stack-based
false positives.

llvm-svn: 328681
2018-03-28 00:45:20 +00:00
Alex Shlyapnikov 0e44f5eb8b [HWASan] Make use-after-free c, not c++ test.
To minimize testing surface (remove libstdc++ from the picture, for
one), make use-after-free c, not c++ test.

Differential Revision: https://reviews.llvm.org/D44705

llvm-svn: 328646
2018-03-27 17:45:53 +00:00
Matt Morehouse 86db6dd2ed [libFuzzer] Disable optimization for exit_on_src_pos.test.
Test fails on Darwin with -O2.

llvm-svn: 328634
2018-03-27 16:40:34 +00:00
Matt Morehouse 22a1afd99f [libFuzzer] Place volatile after pointer types.
For a few tests, volatile was placed before the '*' in pointer
declarations, resulting in it applying to the underlying data rather
than the pointer itself.  Placing volatile after the '*' allows us to
switch those tests to -O2.

llvm-svn: 328633
2018-03-27 16:40:20 +00:00
Alex Shlyapnikov b66b8fd0f2 [HWASan] Relax use-after-free.cc test matching rules.
Relax use-after-free.cc test matching rules to cover more malloc/free
interceptor names variations.

Differential Revision: https://reviews.llvm.org/D44705

llvm-svn: 328630
2018-03-27 16:04:40 +00:00
Kostya Kortchinsky eeee252d50 [scudo] Fuchsia minimal shared runtime
Summary:
Fuchsia requires its Scudo shared runtime to not be C++ dependant. Since they
don't use UBSan in conjunction with Scudo, we can just remove the runtime,
and add the extra `nostdinc++` and `nostdlib++` flags. No need for Coverage
either. This allows to keep things going while working on additional splits
of sanitizer_commong and a more minimal runtime.

Reviewers: phosek, flowerhack, alekseyshl

Reviewed By: phosek, alekseyshl

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44791

llvm-svn: 328625
2018-03-27 14:40:39 +00:00
Evgeniy Stepanov 7989d8da20 Revert "[asan] Replace vfork with fork."
Replacing vfork with fork results in significant slowdown of certain
apps (in particular, memcached).

This reverts r327752.

llvm-svn: 328600
2018-03-27 00:31:16 +00:00
Matt Morehouse 93ce24838c [libFuzzer] Do not optimize minimize_two_crashes.test.
Speculative fix for build bot breakage on Mac.

llvm-svn: 328576
2018-03-26 19:36:20 +00:00
Vitaly Buka 48d19f6961 [sanitizer] Make test compatible with Darwin
llvm-svn: 328467
2018-03-26 01:29:48 +00:00
Vitaly Buka f791c9659b [sanitizer] Fix Darwin build
llvm-svn: 328417
2018-03-24 08:13:18 +00:00
Vitaly Buka 8616363017 Mmap interceptor providing mprotect support
Summary:
- Intercepting mprotect calls.
- Fixing forgotten flag check.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Subscribers: delcypher, srhines, kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44777

llvm-svn: 328415
2018-03-24 07:45:24 +00:00
Vitaly Buka eb8a3674ec [sanitizer] Fix strlcpy and strlcat interceptors on Darwin
llvm-svn: 328414
2018-03-24 07:31:59 +00:00
Alex Shlyapnikov 53ddbac32b [HWASan] Fix use-after-free.cc test on x86-64
Differential Revision: https://reviews.llvm.org/D44705

llvm-svn: 328407
2018-03-24 02:10:49 +00:00
Alex Shlyapnikov 3e65a06015 Remove duplicated const qualifier.
llvm-svn: 328403
2018-03-24 00:40:51 +00:00
Alex Shlyapnikov 5cd35ed4fe [HWASan] Port HWASan to Linux x86-64 (compiler-rt)
Summary:
Porting HWASan to Linux x86-64, first of the three patches, compiler-rt part.

The approach is similar to ARM case, trap signal is used to communicate
memory tag check failure. int3 instruction is used to generate a signal,
access parameters are stored in nop [eax + offset] instruction immediately
following the int3 one

Had to add HWASan init on malloc because, due to much less interceptors
defined (most other sanitizers intercept much more and get initalized
via one of those interceptors or don't care about malloc), HWASan was not
initialized yet when libstdc++ was trying to allocate memory for its own
fixed-size heap, which led to CHECK-fail in AllocateFromLocalPool.

Also added the CHECK() failure handler with more detailed message and
stack reporting.

Reviewers: eugenis

Subscribers: kubamracek, dberris, mgorny, kristof.beyls, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44705

llvm-svn: 328385
2018-03-23 23:38:04 +00:00
Matt Morehouse 5317f2e4c9 [libFuzzer] Use OptForFuzzing attribute with -fsanitize=fuzzer.
Summary:
Disables certain CMP optimizations to improve fuzzing signal under -O1
and -O2.

Switches all fuzzer tests to -O2 except for a few leak tests where the
leak is optimized out under -O2.

Reviewers: kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D44798

llvm-svn: 328384
2018-03-23 23:35:28 +00:00
Vitaly Buka b53180846b FreeBSD sanitizer common, intercept couple of more functions
Summary:
Intercepts lstat, acct, access, faccessat and strlcpy/strlcat

Patch by David CARLIER

Reviewers: visa, vitalybuka

Subscribers: krytarowski, fedor.sergeev, srhines, kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44432

llvm-svn: 328376
2018-03-23 21:44:59 +00:00
Vitaly Buka bb096c27bf Revert "Mmap interceptor providing mprotect support"
Breaks Darwin.

This reverts commit r328369.

llvm-svn: 328375
2018-03-23 21:44:59 +00:00
Vitaly Buka e4d34c0d56 Mmap interceptor providing mprotect support
Summary:
- Intercepting mprotect calls.
- Fixing forgotten flag check.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44777

llvm-svn: 328369
2018-03-23 20:59:51 +00:00
Vitaly Buka c038b2f441 [sanitizer] Fix PPC bot
llvm-svn: 328279
2018-03-23 00:15:10 +00:00
Petr Hosek f4ec686bf7 [sanitizer] zx_vmo_write on Fuchsia takes only 4 arguments now
The system call now fails when it cannot write the requested size.
Update the sanitizer runtime Fuchsia implementation accordingly.

Differential Revision: https://reviews.llvm.org/D44770

llvm-svn: 328273
2018-03-22 23:58:37 +00:00
Vitaly Buka 022a82cb15 More OpenBSD fixes
Summary:
- Use internal_syscall_ptr in internal_readlink
- use sigcontext on OpenBSD

Patch by David CARLIER

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44713

llvm-svn: 328239
2018-03-22 20:42:28 +00:00
Kostya Kortchinsky 988fab3f66 [sanitizer] Split coverage into separate RT in sanitizer_common
Summary:
`sanitizer_common`'s coverage support is fairly well separated, and libcdep by
default. Several sanitizers don't make use of coverage, and as far as I can
tell do no benefit from the extra dependencies pulled in by the coverage public
interface functions.

The following sanitizers call `InitializeCoverage` explicitely: MSan, ASan,
LSan, HWAsan, UBSan. On top of this, any sanitizer bundling RTUBSan should
add the coverage RT as well: ASan, Scudo, UBSan, CFI (diag), TSan, MSan, HWAsan.

So in the end the following have no need: DFSan, ESan, CFI, SafeStack (nolibc
anyway), XRay, and the upcoming Scudo minimal runtime.

I tested this with all the sanitizers check-* with gcc & clang, and in
standalone on Linux & Android, and there was no issue. I couldn't test this on
Mac, Fuchsia, BSDs, & Windows for lack of an environment, so adding a bunch of
people for additional scrunity. I couldn't test HWAsan either.

Reviewers: eugenis, vitalybuka, alekseyshl, flowerhack, kubamracek, dberris, rnk, krytarowski

Reviewed By: vitalybuka, alekseyshl, flowerhack, dberris

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44701

llvm-svn: 328204
2018-03-22 15:04:31 +00:00
Vitaly Buka 8980449bb6 Mmap interceptor new option, Write Exec runtime detector
Summary:
Following-up the refactoring of mmap interceptors, adding a new common
option to detect PROT_WRITE|PROT_EXEC pages request.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: krytarowski, #sanitizers

Differential Revision: https://reviews.llvm.org/D44194

llvm-svn: 328151
2018-03-21 21:25:07 +00:00
George Karpenkov 430aa7405f [libFuzzer] Disable equivalence-signals test on Darwin due to persistent failures
llvm-svn: 328125
2018-03-21 17:55:36 +00:00
Nemanja Ivanovic 212227ed10 [PowerPC] Disable failing dynamic tls test cases
Due to recent kernel upgrades, these test case fail on PowerPC buildbots.
This is a known problem on affected kernels.

llvm-svn: 328091
2018-03-21 11:58:37 +00:00
Dmitry Vyukov 625f757837 tsan: fix darwin build after 328079
328079 introduced a weak hook without default implementation.
This broke darwin build:
http://green.lab.llvm.org/green//job/clang-stage1-configure-RA/43731/consoleFull#-119213188149ba4694-19c4-4d7e-bec5-911270d8a58c

Provide default impl for the hook.

llvm-svn: 328082
2018-03-21 09:26:29 +00:00
Dmitry Vyukov f5675e7e2a tsan: support inlined frames in external symbolization
New API passes a callback function to the external symbolizer,
allowing it to add multiple frames to the traceback. Note that
the old interface API will be still supported until the clients
migrate to the new one.

Author: asmundak (Alexander Smundak)
Reviewed in: https://reviews.llvm.org/D44714

llvm-svn: 328079
2018-03-21 08:44:14 +00:00
Vitaly Buka f1fc25eb74 Fix declaration of environ after r327923
llvm-svn: 328077
2018-03-21 07:50:02 +00:00
Mandeep Singh Grang a9e74a9165 Revert "[compiler-rt] Change std::sort to llvm::sort in response to r327219"
This reverts commit 2ee210e1963e03aacc0f71c50e4994bb5c66586e.

llvm-svn: 327936
2018-03-20 01:17:18 +00:00
Mandeep Singh Grang dc09ebf71b [compiler-rt] Change std::sort to llvm::sort in response to r327219
Summary:
r327219 added wrappers to std::sort which randomly shuffle the container before sorting.
This will help in uncovering non-determinism caused due to undefined sorting
order of objects having the same key.

To make use of that infrastructure we need to invoke llvm::sort instead of std::sort.

Reviewers: kcc, rsmith, RKSimon, eugenis

Reviewed By: RKSimon

Subscribers: efriedma, kubamracek, dberris, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44360

llvm-svn: 327929
2018-03-20 00:44:59 +00:00
Vitaly Buka 544a5555c5 OpenBSD UBsan support missing bits
Summary:
Lost bits since the WIP ticket

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, krytarowski, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44599

llvm-svn: 327923
2018-03-19 23:12:14 +00:00
Vitaly Buka edeeb58f2a Cleanup of "extern char **environ" declaration
llvm-svn: 327904
2018-03-19 20:44:06 +00:00
Vitaly Buka d03bc7c26a Fix CMake/MSVC when compiler-rt and llvm are built separately
Summary:
For some reason CMake can't find the `append` macro if LLVM is built separately and imported via `LLVM_CONFIG_PATH`.

Patch by Loo Rong Jie

Reviewers: rnk, vitalybuka

Reviewed By: rnk, vitalybuka

Subscribers: dberris, mgorny, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43458

llvm-svn: 327876
2018-03-19 18:22:35 +00:00
Vitaly Buka ec6327bab8 Revert "Mmap interceptor new option, Write Exec runtime detector"
Breaks Android bot.

This reverts commit r327747.

llvm-svn: 327762
2018-03-17 00:31:41 +00:00
Petr Hosek 961235d335 [Fuzzer] Build the shared memory hooks for Fuchsia
This is needed otherwise we'll get undefined references when trying
to use the libFuzzer built for Fuchsia.

Differential Revision: https://reviews.llvm.org/D44590

llvm-svn: 327759
2018-03-16 22:40:55 +00:00
Vitaly Buka ec55b07824 OpenBSD UBsan support final missing bits
Summary:
One forgotten file change + reordering one header due to clang-format

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Subscribers: kubamracek, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44556

llvm-svn: 327758
2018-03-16 22:29:29 +00:00
Evgeniy Stepanov a4561123de [asan] Replace vfork with fork.
Summary:
vfork is not ASan-friendly because it modifies stack shadow in the
parent process address space. While it is possible to compensate for that with, for example,
__asan_handle_no_return before each call to _exit or execve and friends, simply replacing
vfork with fork looks like by far the easiest solution.

Posix compliant programs can not detect the difference between vfork and fork.

Fixes https://github.com/google/sanitizers/issues/925

Reviewers: kcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D44587

llvm-svn: 327752
2018-03-16 22:15:05 +00:00
Evgeniy Stepanov 9034c02270 [asan] Remove empty fork interceptor.
After a partial revert, ASan somehow ended up with an empty interceptor for fork().

llvm-svn: 327748
2018-03-16 21:31:49 +00:00
Vitaly Buka d2aca7bf89 Mmap interceptor new option, Write Exec runtime detector
Summary: Following-up the refactoring of mmap interceptors, adding a new common option to detect PROT_WRITE|PROT_EXEC pages request.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: krytarowski, #sanitizers

Differential Revision: https://reviews.llvm.org/D44194

llvm-svn: 327747
2018-03-16 21:25:09 +00:00
Jonas Devlieghere bf0ed8e9b8 Revert "Mmap interceptor new option, Write Exec runtime detector"
This reverts r327696 because it is failing on GreenDragon.

  http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/43605/
  http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-expensive/10957/

llvm-svn: 327719
2018-03-16 15:34:09 +00:00
Dmitry Vyukov e63bc9c200 tsan: revert: Update buildgo.sh to pass -isysroot on Darwin.
This commit breaks actual Go runtime build on gomote builders (10.12) with:

xcode-select: error: tool 'xcodebuild' requires Xcode,
but active developer directory '/Library/Developer/CommandLineTools'
is a command line tools instance

Without this part build works fine.
The original commit does not include any explanation as to why
it is needed.

llvm-svn: 327700
2018-03-16 10:20:58 +00:00
Vitaly Buka 1d0c7ff1c4 FreeBSD TSan support update
Summary:
- Disable thread_finalize callback on FreeBSD, fixing couple of unit tests.

Patch by David CARLIER

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: emaste, kubamracek, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44156

llvm-svn: 327697
2018-03-16 08:23:33 +00:00
Vitaly Buka 21f1b69862 Mmap interceptor new option, Write Exec runtime detector
Summary: Following-up the refactoring of mmap interceptors, adding a new common option to detect PROT_WRITE|PROT_EXEC pages request.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: krytarowski, #sanitizers

Differential Revision: https://reviews.llvm.org/D44194

llvm-svn: 327696
2018-03-16 08:22:18 +00:00
Vitaly Buka 764012b497 MSan, FreeBSD few tests fixes
Summary:
pthread_getattr_np_deadlock support
pthread_getname_np unsupported

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: eugenis, srhines, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44085

llvm-svn: 327678
2018-03-15 22:41:28 +00:00
Vitaly Buka fc8bb95724 OpenBSD UBsan support procmaps
Summary: procmaps OpenBSD specifics

Patch by David CARLIER

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: mgorny, emaste, kubamracek, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44050

llvm-svn: 327677
2018-03-15 22:40:47 +00:00
Vitaly Buka 92e6fbf54a OpenBSD UBsan support common functions
Summary: Ripped off OpenBSD specific from the common Linux implementation

Patch by David Carlier

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: emaste, srhines, kubamracek, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44036

llvm-svn: 327674
2018-03-15 22:19:39 +00:00
Martin Pelikan e0207e4948 [TSan] fix Go runtime test on amd64 with PIE
Summary:
Without this diff, the test segfaults.  Examining the generated executable
(which gets auto-deleted likely by cmake/ninja) yields this error message:

ThreadSanitizer failed to allocate 0x4000 (16384) bytes at address 1755558480000 (errno: 12)

Note that the address has more than 47 bits, which on amd64 means special
treatment and therefore points out an overflow.  The allocation came from
__tsan_map_shadow on a .data pointer, which (on my work Debian-based box)
means the 0x550000000000 range.  This doesn't correspond to the constants
mentioned in tsan_platform.h for Go binaries on Linux/amd64.

The diff therefore allocates memory in the sort of area Go programs would,
and prevents the test from crashing.  It would be nice if reviewers kindly
considered other setups and architectures :-)

Reviewers: kcc, dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44071

llvm-svn: 327621
2018-03-15 12:10:43 +00:00
Vitaly Buka f6d8da1425 [sanitizer] Fix off type in mmap64 interceptor
llvm-svn: 327596
2018-03-15 01:13:29 +00:00
Kostya Kortchinsky a51139046e [scudo] Add Chunk::getSize, rework Chunk::getUsableSize
Summary:
Using `getActuallyAllocatedSize` from the Combined resulting in mediocre
compiled code, as the `ClassId != 0` predicament was not propagated there,
resulting in additional branches and dead code. Move the logic in the frontend,
which results in better compiled code. Also I think it makes it slightly easier
to distinguish between the size the user requested, and the size that was
actually allocated by the allocator.

`const` a couple of things as well.

This has no functional impact.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44444

llvm-svn: 327525
2018-03-14 15:50:32 +00:00
Sylvestre Ledru d9a8b6a745 fix some user facing typos / in the comments
llvm-svn: 327402
2018-03-13 14:35:10 +00:00
Alex Shlyapnikov 79a7c4fe73 [Sanitizers] Add more standard compliant posix_memalign implementation for LSan.
Summary:
Add more standard compliant posix_memalign implementation for LSan and
use corresponding sanitizer's posix_memalign implenetations in allocation
wrappers on Mac.

Reviewers: eugenis, fjricci

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44335

llvm-svn: 327338
2018-03-12 21:59:06 +00:00
Evgeniy Stepanov c2e54761c9 [asan] poison_heap=0 should not disable __asan_handle_no_return.
Reviewers: kcc, alekseyshl, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D44339

llvm-svn: 327337
2018-03-12 21:46:55 +00:00
Kostya Kortchinsky e95ef87663 [scudo] Secondary allocator overhaul to support Windows
Summary:
The need for this change stems from the fact that Windows doesn't support
partial unmapping (`MEM_RELEASE` implies the entire allocated region). So we
now have to keep track of the reserved region and the committed region, so that
we can function without the trimming we did when dealing with larger alignments.

Instead of just having a `ReservedAddressRange` per chunk, we introduce a
`LargeChunkHeader` (and `LargeChunk` namespace) that additionally holds the
committed size and the usable size. The former is needed for stats purposes,
the latter is used by the frontend. Requiring both is debatable, we could only
work with the usable size but then be off by up to a page per chunk when
dealing with stats.

Additionally, we introduce more stats since they turned out to be useful for
experiments, and a `PrintStats` function that will be used by the combined
allocator in later patch.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D43949

llvm-svn: 327321
2018-03-12 19:29:38 +00:00
Kostya Kortchinsky 85ecac5f3f [sanitizer] Align & pad the allocator structures to the cacheline size v2
Summary:
This is a new version of D44261, which broke some builds with older gcc, as
they can't align on a constexpr, but rather require an integer (see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56859) among others.

We introduce `SANITIZER_CACHE_LINE_SIZE` in `sanitizer_platform.h` to be
used in `ALIGNED` attributes instead of using directly `kCacheLineSize`.

Reviewers: alekseyshl, thakis

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44326

llvm-svn: 327297
2018-03-12 17:18:26 +00:00
Petr Hosek 0b013e041d [Fuzzer] When building for Fuchsia, add an explicit libc++ dependency
libFuzzer dependes on C++ library, when building for Fuchsia, we
need to ensure that libFuzzer is only being built after libc++
has been built, so we add an explicity dependency on it.

Differential Revision: https://reviews.llvm.org/D44340

llvm-svn: 327196
2018-03-10 02:00:18 +00:00
Dan Liew 0c51c94b00 [asan] Clean up the REQUIRES of `asan/TestCases/strcat-overlap.cc`
by explicitly listing where we expect this test to work.

Because this test invokes undefined behaviour all sorts of things
can happen (e.g. crash, or on some bots asan manages to catch
something). The test seems to pass okay on Darwin (x86_64/x86_64/i386)
and on Linux (x86_64/i386) so explicitly require one of these platforms
to run the test.

llvm-svn: 327185
2018-03-09 23:12:32 +00:00
Peter Collingbourne 43b055f8b1 cfi: Disable simple-pass.cpp on Darwin.
-mretpoline does not work yet on Darwin.

llvm-svn: 327168
2018-03-09 20:41:56 +00:00
Kostya Kortchinsky 112d7a435b [sanitizer] Revert rCRT327145
Summary:
It breaks the Chromium toolchain due to:
```
lib/sanitizer_common/sanitizer_allocator_primary32.h:269:34: error: requested alignment is not an integer constant
   struct ALIGNED(kCacheLineSize) SizeClassInfo {
```

Reviewers: alekseyshl, thakis

Reviewed By: thakis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44320

llvm-svn: 327167
2018-03-09 20:17:12 +00:00
Peter Collingbourne 2974856ad4 Use branch funnels for virtual calls when retpoline mitigation is enabled.
The retpoline mitigation for variant 2 of CVE-2017-5715 inhibits the
branch predictor, and as a result it can lead to a measurable loss of
performance. We can reduce the performance impact of retpolined virtual
calls by replacing them with a special construct known as a branch
funnel, which is an instruction sequence that implements virtual calls
to a set of known targets using a binary tree of direct branches. This
allows the processor to speculately execute valid implementations of the
virtual function without allowing for speculative execution of of calls
to arbitrary addresses.

This patch extends the whole-program devirtualization pass to replace
certain virtual calls with calls to branch funnels, which are
represented using a new llvm.icall.jumptable intrinsic. It also extends
the LowerTypeTests pass to recognize the new intrinsic, generate code
for the branch funnels (x86_64 only for now) and lay out virtual tables
as required for each branch funnel.

The implementation supports full LTO as well as ThinLTO, and extends the
ThinLTO summary format used for whole-program devirtualization to
support branch funnels.

For more details see RFC:
http://lists.llvm.org/pipermail/llvm-dev/2018-January/120672.html

Differential Revision: https://reviews.llvm.org/D42453

llvm-svn: 327163
2018-03-09 19:11:44 +00:00
Kostya Kortchinsky 69df838b52 [sanitizer] Align & pad the allocator structures to the cacheline size
Summary:
Both `SizeClassInfo` structures for the 32-bit primary & `RegionInfo`
structures for the 64-bit primary can be used by different threads, and as such
they should be aligned & padded to the cacheline size to avoid false sharing.
The former was padded but the array was not aligned, the latter was not padded
but we lucked up as the size of the structure was 192 bytes, and aligned by
the properties of `mmap`.

I plan on adding a couple of fields to the `RegionInfo`, and some highly
threaded tests pointed out that without proper padding & alignment, performance
was getting a hit - and it is going away with proper padding.

This patch makes sure that we are properly padded & aligned for both. I used
a template to avoid padding if the size is already a multiple of the cacheline
size. There might be a better way to do this, I am open to suggestions.

Reviewers: alekseyshl, dvyukov

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44261

llvm-svn: 327145
2018-03-09 16:18:38 +00:00
Dan Liew 3aa75a7f9b [asan] Try to unbreak arm build bots by only running the test on
x86_64 and x86_64H (and not in the iOS simulator) where it seems
to work. We need to properly fix the test when time permits.

llvm-svn: 327122
2018-03-09 12:26:29 +00:00
Kuba Mracek ffb5014d9e Mark the -overlap tests are "UNSUPPORTED: android". Android test harness doesn't know how to handle suppression files.
llvm-svn: 327091
2018-03-09 00:41:29 +00:00
Kuba Mracek 08126e7eaa Mark strcat-overlap.cc as "UNSUPPORTED: win32"
llvm-svn: 327081
2018-03-09 00:03:09 +00:00
Kuba Mracek f9ee716299 Move lto-constmerge-odr.cc to Posix (it's failing on Windows).
llvm-svn: 327080
2018-03-08 23:53:29 +00:00
Dan Liew 0faaa99e4d [asan] Fix bug where suppression of overlapping accesses was ignored on
`strcpy()`, `strncpy()`, `strcat()`, and `strncat()`.

rdar://problem/35576899

Differential Revision: https://reviews.llvm.org/D43702

llvm-svn: 327068
2018-03-08 21:50:22 +00:00
Kuba Mracek e70e5fcc7e [asan] Fix a false positive ODR violation due to LTO ConstantMerge pass [compiler-rt part, take 3]
This fixes a false positive ODR violation that is reported by ASan when using LTO. In cases, where two constant globals have the same value, LTO will merge them, which breaks ASan's ODR detection. See the included testcase for an example.

Differential Revision: https://reviews.llvm.org/D43959

llvm-svn: 327062
2018-03-08 21:02:52 +00:00
Kuba Mracek e3dfb9408a More revert of r327031
llvm-svn: 327038
2018-03-08 18:20:03 +00:00
Kuba Mracek 0fe43695e5 Revert r327031
llvm-svn: 327032
2018-03-08 17:31:31 +00:00
Kuba Mracek 653f94df53 [asan] Fix a false positive ODR violation due to LTO ConstantMerge pass [compiler-rt part]
This fixes a false positive ODR violation that is reported by ASan when using LTO. In cases, where two constant globals have the same value, LTO will merge them, which breaks ASan's ODR detection.

Differential Revision: https://reviews.llvm.org/D43959

llvm-svn: 327031
2018-03-08 17:24:47 +00:00
Eugene Zemtsov e465a84f85 Build LLVMDemangle from build_symbolizer.sh
Symbolizer now depends on internal implementation of itaniumDemangle.

llvm-svn: 326987
2018-03-08 04:34:22 +00:00
Eugene Zemtsov 824189f624 Add Demangle lib into internalization list
Symbolizer now depends on internal implementation of itaniumDemangle.

llvm-svn: 326983
2018-03-08 02:57:54 +00:00
Vitaly Buka 1732ee75ff [sanitizer] Fix SANITIZER_INTERCEPT_MMAP and SANITIZER_INTERCEPT_MMAP64 values
llvm-svn: 326938
2018-03-07 19:43:50 +00:00
Petr Hosek 2d19f9036e [Fuzzer] Avoid the unnecessary rebuild of the custom libc++
This changes the add_custom_libcxx macro to resemble the
llvm_ExternalProject_Add. The primary motivation is to avoid
unnecessary libFuzzer rebuilds that are being done on every
Ninja/Make invocation. The libc++ should be only rebuilt whenever
the libc++ source itself changes.

Differential Revision: https://reviews.llvm.org/D43213

llvm-svn: 326921
2018-03-07 18:14:09 +00:00
Kostya Kortchinsky e245ec0cf0 [scudo] Make logging more consistent
Summary:
A few changes related to logging:
- prepend `Scudo` to the error messages so that users can identify that we
  reported an error;
- replace a couple of `Report` calls in the RSS check code with
  `dieWithMessage`/`Print`, mark a condition as `UNLIKELY` in the process;
- change some messages so that they all look more or less the same. This
  includes the `CHECK` message;
- adapt a couple of tests with the new strings.

A couple of side notes: this results in a few 1-line-blocks, for which I left
brackets. There doesn't seem to be any style guide for that, I can remove them
if need be. I didn't use `SanitizerToolName` in the strings, but directly
`Scudo` because we are the only users, I could change that too.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44171

llvm-svn: 326901
2018-03-07 16:22:16 +00:00
Kamil Rytarowski 9b996f03cb Don't intercept mmap64() on NetBSD
Disable SANITIZER_INTERCEPT_MMAP64 for SI_NETBSD.

NetBSD switched to 64-bit offsets almost 30 years ago on 32-bit platforms
and never needed mmap64() concept.

llvm-svn: 326883
2018-03-07 09:16:10 +00:00
George Burgess IV de772ce015 Attempt to appease buildbots
I can't reproduce this build error locally, but it appears
straightforward enough to fix. r326851 renamed two of the params of this
interceptor, but apparently to update their use here.

Failure:
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/4569

llvm-svn: 326876
2018-03-07 07:36:23 +00:00
Adam Nemet 8e0bc894f9 Disable tests from r326852 on Darwin
Darwin needs xlocale.h but I see no precedence in the code how non-Posix
headers like this are handled and I am not experienced with this code.  Just
disable the tests for now to recover the bots.

rdar://38208146

llvm-svn: 326874
2018-03-07 05:33:37 +00:00
Dean Michael Berris 35824efb45 [XRay][compiler-rt] Add APIs for processing logs in memory
Summary:
This change adds APIs to allow logging implementations to provide a
function for iterating through in-memory buffers (if they hold in-memory
buffers) and a way for users to generically deal with these buffers
in-process. These APIs are:

  - __xray_log_set_buffer_iterator(...) and
    __xray_log_remove_buffer_iterator(): installs and removes an
    iterator function that takes an XRayBuffer and yields the next one.

  - __xray_log_process_buffers(...): takes a function pointer that can
    take a mode identifier (string) and an XRayBuffer to process this
    data as they see fit.

The intent is to have the FDR mode implementation's buffers be
available through this `__xray_log_process_buffers(...)` API, so that
they can be streamed from memory instead of flushed to disk (useful for
getting the data to a network, or doing in-process analysis).

Basic mode logging will not support this mechanism as it's designed to
write the data mostly to disk.

Future implementations will may depend on this API as well, to allow for
programmatically working through the XRay buffers exposed to the
users in some fashion.

Reviewers: eizan, kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43495

llvm-svn: 326866
2018-03-07 02:45:14 +00:00
Kuba Mracek cddd6339e9 Fixup r326851: mmap64 interceptor should not be used on Darwin.
llvm-svn: 326864
2018-03-07 02:30:55 +00:00
Vitaly Buka 56bdcc8e1e [sanitizer] Update symbolizer test
llvm-svn: 326855
2018-03-07 00:15:10 +00:00
Vitaly Buka 4580dc1a83 [sanitizer] Extract common code into STRXFRM_INTERCEPTOR_IMPL
llvm-svn: 326854
2018-03-07 00:14:52 +00:00
Vitaly Buka 2e5f44653b [sanitizer] Move strxfrm interceptors into sanitizer_common
llvm-svn: 326853
2018-03-07 00:14:30 +00:00
Vitaly Buka 46520a4dc1 [sanitizer] Add interceptors for wcsxfrm, wcsxfrm_l
Patch by Oliver Chang

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D44133

llvm-svn: 326852
2018-03-07 00:14:12 +00:00
Vitaly Buka 78457e68f4 [sanitizer] Move mmap interceptors into sanitizer_common
Reviewers: devnexen, krytarowski, eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D44125

llvm-svn: 326851
2018-03-07 00:13:54 +00:00
Dean Michael Berris 4e8f3caf95 [XRay][compiler-rt] Make unit tests depend on implementation files
Summary:
This change makes changes to XRay implementation files trigger re-builds
of the unit tests. Prior to this change, the unit tests were not built
and run properly if the implementation files were changed during the
development process. This change forces the dependency on all files in
the XRay include and lib hosted files in compiler-rt.

Caveat is, that new files added to the director(ies) will need a re-run
of CMake to re-generate the fileset.

We think this is an OK compromise, since adding new files may
necessitate editing (or adding) new unit tests. It's also less likely
that we're adding new files without updating the CMake configuration to
include the functionality in the XRay runtime implementation anyway.

Reviewers: pelikan, kpw, nglevin

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D44080

llvm-svn: 326842
2018-03-06 23:06:09 +00:00
Kostya Kortchinsky 7dc5137581 [scudo] Use gc-sections by default
Summary:
If not using `-Wl,--gc-sections`, a whole lot of unused `sanitizer_common` code
and related static variables are pulled into the shared library.
Keep the binary size smaller, and its memory footprint smaller as well, by
using the compiler flags `-ffunction-section` & `-fdata-sections` by default,
as well as the linker flags `-Wl,--gc-sections`.

Current experiments show a large discrepency between binary sizes generated
by gcc (big) and clang (small). I am not sure yet how I can make a test that
would encompass both, so it's an outstanding work item.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: mgorny, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44121

llvm-svn: 326833
2018-03-06 20:13:37 +00:00
Kamil Rytarowski 2f358738b8 OpenBSD sanitizer common, define RLIMIT_AS constant
Summary: define RLIMIT_AS constant until it s defined in the future

Patch by: David Carlier

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44068

llvm-svn: 326768
2018-03-06 08:24:16 +00:00
Petr Hosek 5e3b41d573 [sanitizer] Fix the return type for GetTid in Fuchsia implementation
This is triggering "functions that differ only in their return type
cannot be overloaded" error.

Differential Revision: https://reviews.llvm.org/D44126

llvm-svn: 326759
2018-03-06 02:01:32 +00:00
Kamil Rytarowski a51ccfd49f Remove more references to ptrace_pt_{g,s}et_sigmask
These operations are now obsolete on NetBSD and will be removed.

No functional change for other OSes.

Sponsored by <The NetBSD Foundation>

llvm-svn: 326663
2018-03-03 18:41:38 +00:00
Kamil Rytarowski fc44eb07a7 Fix NetBSD 32-bit build
Replace 'defined(SANITIZER_OPENBSD)' with appropriate 'SANITIZER_OPENBSD'.

This is a fallout from adding OpenBSD partial support.

llvm-svn: 326662
2018-03-03 18:38:20 +00:00
Kamil Rytarowski 9f2a4e212c Remove obsolete ptrace(2) operations from NetBSD
PT_SET_SIGMASK and PT_GET_SIGMASK will be removed from NetBSD
without backward compat (it shortlived in a development branch).

Sponsored by <The NetBSD Foundation>

llvm-svn: 326657
2018-03-03 16:21:13 +00:00
Kamil Rytarowski 980d0f8919 Revert D44035
Broken syntax.

llvm-svn: 326655
2018-03-03 14:50:36 +00:00
Kamil Rytarowski baddcfd76a Hotfix for D44035
Try to fix the syntax.

llvm-svn: 326654
2018-03-03 14:31:06 +00:00
Kamil Rytarowski 13e0063e74 Support OpenBSD in common interceptors
Summary:
Extract  INIT_PTHREAD_ATTR_GET_SCHED from INIT_PTHREAD_ATTR_GET,
as the former is not supported on OpenBSD.

Supported interceptors
getdetachstate, getguardsize, getscope, getstacksize, getgrouplist and getstack
Unsupported
getschedparam, getgroupmembership and getschedpolicy

Patch by David CARLIER

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: fedor.sergeev, srhines, kubamracek, mgorny, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44035

llvm-svn: 326653
2018-03-03 14:13:08 +00:00
Kamil Rytarowski 310cd2509c OpenBSD UBsan support enabling SANITIZER_OPENBSD widely
Summary: Adding OpenBSD platform

Patch by: David CARLIER

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44049

llvm-svn: 326651
2018-03-03 12:12:03 +00:00
Kamil Rytarowski c18102fed5 OpenBSD UBsan support / common
Summary:
Sanitizer common, enable OpenBSD platform.
- Enable common interceptors as possible and create few distinct ones.
- Create necessary sanitizer_struct types.

Patch by: David CARLIER

Reviewers: krytarowski, vitalybuka, visa, kettenis

Reviewed By: vitalybuka

Subscribers: emaste, srhines, kubamracek, mgorny, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43909

llvm-svn: 326650
2018-03-03 12:04:40 +00:00
Kamil Rytarowski 8ec9586851 [msan] Restrict -ftls-model=initial-exec to FreeBSD
Noted by <eugenis> in D43613

llvm-svn: 326649
2018-03-03 12:00:12 +00:00
Kamil Rytarowski d14a955c24 Sanitiser common, using u64 type for GetTid on posix systems
Summary: Moving from ptr to u64 for GetTid posix implementation.
[FreeBSD] Moving from pthread_self to thr_self more appropriate to get thread ID.

Patch by: David CARLIER

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43998

llvm-svn: 326647
2018-03-03 11:50:58 +00:00
Kamil Rytarowski 0d58e0f613 OpenBSD UBsan support, cmake part
Summary:
On OpenBSD no multi arch support
Enabling only UBsan and disabling Asan.

Patch by: David CARLIER

Reviewers: krytarowski, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44017

llvm-svn: 326646
2018-03-03 11:48:54 +00:00
Kamil Rytarowski 3d7fbb052f Adding Msan support to FreeBSD
Summary:
Enabling the memory sanitizer support for FreeBSD, most of unit tests are compatible.
- Adding fstat and stressor_r interceptors.
- Updating the struct link_map access since most likely the struct Obj_Entry had been updated since.
- Disabling few unit tests until further work is needed (or we can assume it can work in real world code).

Patch by: David CARLIER

Reviewers: vitalybuka, krytarowski

Reviewed By: vitalybuka

Subscribers: eugenis, dim, srhines, emaste, kubamracek, mgorny, fedor.sergeev, hintonda, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43080

llvm-svn: 326644
2018-03-03 11:43:11 +00:00
Kamil Rytarowski 7e55245213 OpenBSD Ubsan support / interception
Summary: Interception, enabling OpenBSD platform

Patch by: David CARLIER

Reviewers: krytarowski, vitalybuka, visa

Reviewed By: vitalybuka

Subscribers: srhines, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43893

llvm-svn: 326544
2018-03-02 07:32:30 +00:00
Kamil Rytarowski 4410bbc9ed OpenBSD UBsan support / ubsan part
Summary: UBsan, enable OpenBSD platform

Patch by: David CARLIER

Reviewers: krytarowski, kettenis, visa, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, fedor.sergeev, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43894

llvm-svn: 326543
2018-03-02 07:30:22 +00:00
Shiva Chen 77f19a384c [PATCH] [compiler-rt, RISCV] Support builtins for RISC-V
Summary:
Support builtins for RISC-V, RV32 and RV64.

Reviewers: asb, apazos, mgrang

Differential Revision: https://reviews.llvm.org/D42958

llvm-svn: 326420
2018-03-01 07:47:27 +00:00
Vedant Kumar a4c56923d2 [profile] Test the exported symbol set for empty programs
Programs without any code in them should export the exact same set of
symbols as programs with code, at least on Darwin. This is done to make
text-based API verification possible for certain Darwin frameworks.

llvm-svn: 326351
2018-02-28 19:00:16 +00:00
Mandeep Singh Grang d34e876185 [RISCV] Force enable int128 for compiling long double routines
Summary:
For RISCV32, we must force enable int128 for compiling long double routines using the flag -fforce-enable-int128.

Related clang patch: https://reviews.llvm.org/D43105

Reviewers: asb, kito-cheng, apazos, compnerd, howard.hinnant

Reviewed By: kito-cheng

Subscribers: shiva0217, efriedma, mgorny, hintonda, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43106

llvm-svn: 326346
2018-02-28 18:24:09 +00:00
Kamil Rytarowski ad4632e07a Disable ASan exceptions on NetBSD
This is a workarond for the fallout from D42644:
[asan] Intercept std::rethrow_exception indirectly.

Reported problem on NetBSD/amd64:

$ sh ./projects/compiler-rt/test/sanitizer_common/asan-i386-NetBSD/NetBSD/Output/ttyent.cc.script
/usr/lib/i386/libgcc.a(unwind-dw2.o): In function `_Unwind_RaiseException':
unwind-dw2.c:(.text+0x1b41): multiple definition of `_Unwind_RaiseException'
/public/llvm-build/lib/clang/7.0.0/lib/netbsd/libclang_rt.asan-i386.a(asan_interceptors.cc.o):/public/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:337: first defined here
clang-7.0: error: linker command failed with exit code 1 (use -v to see invocation)

llvm-svn: 326216
2018-02-27 18:05:49 +00:00
Kostya Kortchinsky beeea6227b [scudo] Introduce Chunk::getHeaderSize
Summary:
Instead of using `AlignedChunkHeaderSize`, introduce a `constexpr` function
`getHeaderSize` in the `Chunk` namespace. Switch `RoundUpTo` to a `constexpr`
as well (so we can use it in `constexpr` declarations). Mark a few variables
in the areas touched as `const`.

Overall this has no functional change, and is mostly to make things a bit more
consistent.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D43772

llvm-svn: 326206
2018-02-27 16:14:49 +00:00
Kamil Rytarowski 8992eddbdf Add new interceptors: getnetent(3) family
Summary:
getnetent, getnetbyaddr, getnetbyname - get network entry

Reuse them on NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43543

llvm-svn: 326163
2018-02-27 02:33:30 +00:00
Kamil Rytarowski fc67e66b57 Add new interceptors: getprotoent(3) family
Summary:
getprotoent, getprotobynumber, getprotobyname - get protocol entry

Reuse them on NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43541

llvm-svn: 326162
2018-02-27 02:32:04 +00:00
Kamil Rytarowski 31c34bfe8f Add new interceptors: getttyent(3) family
Summary:
getttyent, getttynam, setttyentpath - get ttys file entry

Reuse them on NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43539

llvm-svn: 326161
2018-02-27 02:30:16 +00:00
Vitaly Buka e9d957de8d [asan] Enable ASAN_INTERCEPT___CXA_THROW for x86 Android
llvm-svn: 326160
2018-02-27 02:14:06 +00:00
Kuba Mracek d2af423bbe Fix build for iOS/ARM ("__Unwind_RaiseException" is not available for armv7).
llvm-svn: 326150
2018-02-26 23:39:04 +00:00
Vitaly Buka 78577341c4 [asan] Fix build for Android
llvm-svn: 326149
2018-02-26 23:23:07 +00:00
Volodymyr Sapsai 9ef00ed487 Fix which Darwin versions have ObjC runtime with full subscripting support.
Update min deployment target in some tests so that they don't try
to link against libarclite and don't fail when it's not available.

rdar://problem/29253617

Reviewers: vsk, kubamracek

Reviewed By: vsk

Subscribers: jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43787

llvm-svn: 326143
2018-02-26 23:04:57 +00:00
Reid Kleckner 54af3e7e14 Re-land "Emit proper CodeView when -gcodeview is passed without the cl driver."
Reverts r326116 and re-lands r326113 with a fix to ASan so that it
enables column info in its test suite.

llvm-svn: 326141
2018-02-26 22:55:33 +00:00
Vitaly Buka 781ef03e10 [asan] Intercept std::rethrow_exception indirectly
Summary:
Fixes Bug 32434
See https://bugs.llvm.org/show_bug.cgi?id=32434

Short summary:
std::rethrow_exception does not use __cxa_throw to rethrow the exception, so if
it is called from uninstrumented code, it will leave the stack poisoned. This
can lead to false positives.

Long description:

For functions which don't return normally (e.g. via exceptions), asan needs to
unpoison the entire stack. It is not known before a call to such a function
where execution will continue, some function which don't contain cleanup code
like destructors might be skipped. After stack unwinding, execution might
continue in uninstrumented code.

If the stack has been poisoned before such a function is called, but the stack
is unwound during the unconventional return, then zombie redzones (entries) for
no longer existing stack variables can remain in the shadow memory. Normally,
this is avoided by asan generating a call to asan_handle_no_return before all
functions marked as [[noreturn]]. This asan_handle_no_return unpoisons the
entire stack. Since these [[noreturn]] functions can be called from
uninstrumented code, asan also introduces interceptor functions which call
asan_handle_no_return before running the original [[noreturn]] function;
for example, cxa_throw is intercepted.

If a [[noreturn]] function is called from uninstrumented code (so the stack is
left poisoned) and additionally, execution continues in uninstrumented code, new
stack variables might be introduced and overlap with the stack variables
which have been removed during stack unwinding. Since the redzones are not
cleared nor overwritten by uninstrumented code, they remain but now contain
invalid data.

Now, if the redzones are checked against the new stack variables, false
positive reports can occur. This can happen for example by the uninstrumented
code calling an intercepted function such as memcpy, or an instrumented
function.

Intercepting std::rethrow_exception directly is not easily possible since it
depends on the C++ standard library implementation (e.g. libcxx vs libstdc++)
and the mangled name it produces for this function. As a rather simple
workaround, we're intercepting _Unwind_RaiseException for libstdc++. For
libcxxabi, we can intercept the ABI function __cxa_rethrow_primary_exception.

Patch by Robert Schneider.

Reviewers: kcc, eugenis, alekseyshl, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D42644

llvm-svn: 326132
2018-02-26 21:40:19 +00:00
Kuba Mracek 473f3fbaf8 Fix-up for r326106: FindAvailableMemoryRange needs a nullptr as its 5th argument.
llvm-svn: 326111
2018-02-26 18:44:43 +00:00
Kuba Mracek 061f3589cc [asan] Be more careful and verbose when allocating dynamic shadow memory
FindAvailableMemoryRange can currently overwrite existing memory (by restricting the VM below addresses that are already used). This patch adds a check to make sure we don't restrict the VM space too much. We are also now more explicit about why the lookup failed and print out verbose values.

Differential Revision: https://reviews.llvm.org/D43318

llvm-svn: 326106
2018-02-26 18:33:21 +00:00
Alex Shlyapnikov 4b30a4261f [MSan] Print current stack on CHECK violation
Summary:
Print current stack on CHECK violation to aid debugging and
match other sanitizers functionality.

Reviewers: eugenis

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43692

llvm-svn: 326105
2018-02-26 18:27:24 +00:00
Kostya Kortchinsky 0c8ecea88b [scudo] Make some tests less Linux-y
Summary:
Start making the Scudo tests less Linux-y:
- `malloc_usable_size` doesn't exist everywhere, so replace them with
  `__sanitizer_get_allocated_size` which we provide;
- move all the `memalign` related tests into `memalign.c` since it's also not
  available everywhere.

I also noticed that the `memalign.c` was missing a line in one of the loops.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D43393

llvm-svn: 326100
2018-02-26 17:14:44 +00:00
Kamil Rytarowski b6195b59a6 Skip two more ioctl interceptors for NetBSD
Don't intercept:
 - HFSC_IF_ATTACH
 - HFSC_MOD_CLASS

These operations conflict on NetBSD 32-bit with:
 - PRIQ_IF_ATTACH
 - PRIQ_MOD_CLASS

Sponsored by <The NetBSD Foundation>

llvm-svn: 326084
2018-02-26 13:00:40 +00:00
Dan Liew 17ad62be94 [ubsan-minimal] Fix the ubsan_minimal debug build (COMPILER_RT_DEBUG=1) on macOS.
`ubsan_minimal` makes use of the `_sanitizer::atomic_load` function.
This function uses the `DCHECK` macro which in debug builds will use
the `_sanitizer::CheckFailed` function.

This function is part of `sanitizer_common` but `ubsan_minimal` doesn't
use this so the implementation is missing which leads to link failures
on macOS when trying to link `libclang_rt.ubsan_minimal_osx_dynamic.dylib`.
This is in contrast to the BFD linker on Linux which doesn't seem to care
about the missing symbol.

A basic implementation of `_sanitizer::CheckFailed` has been added to
the `ubsan_minimal` debug build to avoid the link error. The
implementation could definitely be improved but I don't know which
functions can be used in this context so I decided to restrict myself to
functions only being used in `ubsan_minimal` already.

llvm-svn: 326032
2018-02-24 13:14:44 +00:00
Vitaly Buka 1aa98f4294 [cfi] Lazy initialization of CFI interceptors
Summary:
Interceptors initialization may need to allocate memory. So if we initialize too
early we can crash in non initialized allocator.

Reviewers: pcc, eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43669

llvm-svn: 326025
2018-02-24 06:58:56 +00:00
Kamil Rytarowski c22867f5fc Correct ctype(3) functions with NLS on NetBSD
Summary:
The setlocale(3) function reloads the ctype(3) arrays from
external files. This happens behind the scenes in the internals
of libc (citrus library, runes functions etc).

ctype(3) functions like isspace(3) can be provided with two
variations on NetBSD: inlined or via a global symbol in libc:

```
#if defined(_NETBSD_SOURCE) && !defined(_CTYPE_NOINLINE) && \
    !defined(__cplusplus)
#include <sys/ctype_inline.h>
#else
#include <sys/ctype_bits.h>
#endif
```

The in-lined versions are de-facto array lookup operations.

```
#define isspace(c)      ((int)((_ctype_tab_ + 1)[(c)] & _CTYPE_S))
```

After setting setlocale(3) the ctype(3) arrays (_ctype_tab_,
_toupper_tab_, _tolower_tab_) are reload behind the scenes
and they are required to be marked as initialized.

Set them initialized inside the common setlocale(3) interceptor.

The arrays are of size of 257 elements: 0..255 + 1 (EOF).

This corrects errors on NetBSD/amd64 in applications
prebuilt with MSan.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, dvyukov, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42020

llvm-svn: 326008
2018-02-24 01:01:14 +00:00
Alex Shlyapnikov 524946232b [Sanitizers] Increase allocated chunk limit for LargeMmapAllocator
Summary:
There are applications out there which allocate more than 1 << 18 large chunks
of memory (those handled by LargeMmapAllocator, aka secondary allocator).

For 64 bits, secondary allocator stores allocated chunks in a growing on
demand region of memory, growing in blocks of 128K, up to 1 << 20 chunks total.

Sanitizer internal allocator's secondary uses fixed size array storing up
to 1 << 15 chunks (down to 256K from 2Mb of memory used for that array).

Nothing is changed for 32 bits, chunks are still stored in the fixed size
array (up to 1 << 15 chunks).

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D43693

llvm-svn: 326007
2018-02-24 01:00:27 +00:00
Matt Morehouse ddf352b953 [libFuzzer] Include TEMP_MAX_LEN in Fuzzer::PrintStats.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43597

llvm-svn: 325817
2018-02-22 19:00:17 +00:00
Kamil Rytarowski 663dbfadbf Try to fix the syntax in test
Reported on a buildbot:

Error in XFAIL list:
couldn't parse text: '| arm || aarch64 || mips'
in expression: 'freebsd | arm || aarch64 || mips'

Add || in the place of |

Fallout from D43382

llvm-svn: 325751
2018-02-22 07:17:01 +00:00
Kamil Rytarowski c419df9066 Remove empty files
Left over after D43382

llvm-svn: 325744
2018-02-22 06:11:28 +00:00
Kamil Rytarowski 80c2a79437 Xray instrumentation / enabling more tests
Summary:
The Unix subdirectory mostly allows only on Linux x86_64 but now we can target x86_64 arch in general.

Patch by David CARLIER

Reviewers: krytarowski, dberris, emaste

Reviewed By: krytarowski, dberris, emaste

Subscribers: emaste, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43382

llvm-svn: 325743
2018-02-22 06:08:26 +00:00
Vitaly Buka e075709b41 [sanitizer] Allow to set SANITIZER_CAN_USE_PREINIT_ARRAY externally
llvm-svn: 325720
2018-02-21 21:13:07 +00:00
Evgeniy Stepanov 43271b1803 [hwasan] Fix inline instrumentation.
This patch changes hwasan inline instrumentation:

Fixes address untagging for shadow address calculation (use 0xFF instead of 0x00 for the top byte).
Emits brk instruction instead of hlt for the kernel and user space.
Use 0x900 instead of 0x100 for brk immediate (0x100 - 0x800 are unavailable in the kernel).
Fixes and adds appropriate tests.

Patch by Andrey Konovalov.

Differential Revision: https://reviews.llvm.org/D43135

llvm-svn: 325711
2018-02-21 19:52:23 +00:00
Maxim Kuvyrkov 6b287234ea Fix racy msan test
Summary:
Child thread may finish before pthread_setname_np() and/or
pthread_getname_np() had a chance to run, which causes these functions
to fail with ENOENT (meaning /proc/self/task/[tid]/comm doesn't exist).

Fix by stalling child thread on a mutex.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43548

llvm-svn: 325703
2018-02-21 18:03:50 +00:00
Kamil Rytarowski 8b5d3ebcbe Msan, fixing DTLS_on_tls_get_addr signature empty implementation
Summary: No supported oses normally compiled that code (or not for a long time) probably never caught it.

Patch by: David CARLIER

Reviewers: vitalybuka, krytarowski

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43502

llvm-svn: 325664
2018-02-21 04:16:45 +00:00
Kamil Rytarowski 59a9856e06 Add new interceptor: strmode(3)
Summary:
strmode - convert inode status information into a symbolic string

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43485

llvm-svn: 325588
2018-02-20 15:53:30 +00:00
Kamil Rytarowski 8317565532 Add new interceptor: fgetln(3)
Summary:
fgetln - get a line from a stream

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43482

llvm-svn: 325587
2018-02-20 15:52:08 +00:00
Kamil Rytarowski babcdb3a9a Stop intercepting forkpty(3) and openpty(3) on NetBSD
Summary:
forkpty(3) and openpty(3) are part of `-lutil` and we don't intend to reimplement
this system library in sanitizers. Everybody using these functions will need to use
a precompiled library against MSan or other desired sanitizer.

Restrict these functions to Linux-only.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43490

llvm-svn: 325585
2018-02-20 15:43:07 +00:00
Kamil Rytarowski 7cde46764a Recognize all NetBSD architectures in UBSan
Summary:
Use uniform accessors for Program Pointer,
Stack Pointer and Frame Pointer.

Remove CPU check in UBSan supported platforms
and rely only on the OS type.

This adds NetBSD support in GetPcSpBp() for:
 - ARM
 - ARM64
 - HPPA
 - PowerPC/PowerPC64
 - SPARC/SPARC64
 - MIPS
 - DEC Alpha AXP
 - DEC VAX
 - M68K and M68010
 - SH3
 - IA64
 - OR1K
 - RISCV

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, ro

Reviewed By: vitalybuka

Subscribers: aemerson, jyknight, sdardis, kubamracek, arichardson, llvm-commits, kristof.beyls, fedor.sergeev, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43021

llvm-svn: 325431
2018-02-17 13:35:09 +00:00
Kamil Rytarowski bab5252789 Add initial XRay support for NetBSD
Summary:
Reuse the existing FreeBSD code as it is.

Sponsored by <The NetBSD Foundation>

Reviewers: dberris, rnk, vitalybuka

Reviewed By: dberris

Subscribers: mclow.lists, emaste, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D43370

llvm-svn: 325345
2018-02-16 14:45:20 +00:00
Kamil Rytarowski 015125d7ff Reintroduce FreeBSD support in test/xray
Tested by Douglas Yung.

The original patch from D43278 has been reverted.

New patch by myself.

llvm-svn: 325315
2018-02-16 04:20:33 +00:00
Kamil Rytarowski 515fc27b9e Revert part of SVN r. 325240 in lit.cfg
The original change broke a llvm-clang-lld-x86_64-debian setup.

This change will be investigated and reintroduced in future.

Original commit:

  "Add Xray instrumentation support to FreeBSD"
  https://reviews.llvm.org/D43278

llvm-svn: 325309
2018-02-16 01:16:06 +00:00
Kuba Mracek 9967378eba [asan] Add "arm64" into the list of 64-bit architectures
It looks like get_bits_for_arch doesn't recognize "arm64" as a 64-bit architecture, and it actually matches the "arm" regexp, which marks it as 32-bit. Let's fix that by matching the 64-bit list first and addin "arm64" into the list.

Differential Revision: https://reviews.llvm.org/D43155

llvm-svn: 325300
2018-02-15 22:14:36 +00:00
Alex Shlyapnikov 48296ebf8b [TSan] Fix static TLS boundaries calculations in __tls_get_addr interceptor.
Summary:
DTLS_on_tls_get_addr expects (tls_addr + tls_size) as the last
parameter, static_tls_end.

Reviewers: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D43325

llvm-svn: 325276
2018-02-15 18:58:31 +00:00
Kamil Rytarowski a6affc2fab Try to unbreak Darwin after XRay/FreeBSD commit
Add a fallback definition of getTSCFrequency().

llvm-svn: 325249
2018-02-15 15:24:01 +00:00
Kamil Rytarowski 7d15ab6aef Implement a large part of NetBSD syscalls of netbsd_syscall_hooks.h
This is a follow up of:

  r325206 - Add NetBSD syscall hooks skeleton in sanitizers

Sponsored by <The NetBSD Foundation>

llvm-svn: 325248
2018-02-15 15:15:45 +00:00
Kamil Rytarowski 62b0c5bb88 Enable test/asan for NetBSD
Sponsored by <The NetBSD Foundation>

llvm-svn: 325246
2018-02-15 14:59:59 +00:00
Kamil Rytarowski e7c26e7929 Enable syscall-specific functions in TSan/NetBSD
NetBSD ships now with netbsd_syscall_hooks.h and requires support
for TSan specific features to be enabled.

This is follow up of:

  D42048: Add NetBSD syscall hooks skeleton in sanitizers

Sponsored by <The NetBSD Foundation>

llvm-svn: 325245
2018-02-15 14:58:15 +00:00
Kamil Rytarowski 4d4ed0e288 Add Xray instrumentation support to FreeBSD
Summary:
- Enabling the build.
- Using assembly for the cpuid parts.
- Using thr_self FreeBSD call to get the thread id 

Patch by: David CARLIER

Reviewers: dberris, rnk, krytarowski

Reviewed By: dberris, krytarowski

Subscribers: emaste, stevecheckoway, nglevin, srhines, kubamracek, dberris, mgorny, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D43278

llvm-svn: 325240
2018-02-15 14:17:15 +00:00
Kamil Rytarowski 5ffda0cc1f Teach sanitizer about NetBSD specific ioctl(2) calls
Summary:
Introduce handling of 1200 NetBSD specific ioctl(2) calls.
Over 100 operations are disabled as unavailable or conflicting
with the existing ones (the same operation number).

Add a script that generates the rules to detect ioctls on NetBSD.
The generate_netbsd_ioctls.awk script has been written
in NetBSD awk(1) (patched nawk) and is compatible with gawk.

Generate lib/sanitizer_common/sanitizer_interceptors_ioctl_netbsd.inc
with the awk(1) script.

Update sanitizer_platform_limits_netbsd accordingly to add the needed
definitions.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, eugenis, dvyukov

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, fedor.sergeev, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41636

llvm-svn: 325212
2018-02-15 03:36:16 +00:00
Kamil Rytarowski 443434e45c Prevent recursive MSan interceptors in strftime(3) like functions
Summary:
This corrects intercepting this call on NetBSD without false positives.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42883

llvm-svn: 325208
2018-02-15 02:48:20 +00:00
Kamil Rytarowski a8ab3a6008 Prevent recursive MSan interceptors in fgets(3)
Summary:
This corrects intercepting this call on NetBSD without false positives.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42884

llvm-svn: 325207
2018-02-15 02:47:35 +00:00
Kamil Rytarowski 39628a7ab0 Add NetBSD syscall hooks skeleton in sanitizers
Summary:
Implement the skeleton of NetBSD syscall hooks for use with sanitizers.

Add a script that generates the rules to handle syscalls
on NetBSD: generate_netbsd_syscalls.awk. It has been written
in NetBSD awk(1) (patched nawk) and is compatible with gawk.

Generate lib/sanitizer_common/sanitizer_platform_limits_netbsd.h
that is a public header for applications, and included as:
<sanitizer_common/sanitizer_platform_limits_netbsd.h>.

Generate sanitizer_syscalls_netbsd.inc that defines all the
syscall rules for NetBSD. This file is modeled after the Linux
specific file: sanitizer_common_syscalls.inc.

Start recognizing NetBSD syscalls with existing sanitizers:
ASan, ESan, HWASan, TSan, MSan.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, kcc, dvyukov, eugenis

Reviewed By: vitalybuka

Subscribers: hintonda, kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42048

llvm-svn: 325206
2018-02-15 02:43:02 +00:00
Kamil Rytarowski 1046294bd3 Add new interceptor: lstat(2)
Summary:
lstat - get file status

Use it on NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, eugenis

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42909

llvm-svn: 325199
2018-02-15 02:17:06 +00:00
Vitaly Buka 1fd9f071a2 [sanitizer] Fix flakiness in the tests
Seems tests fails if link already exists.
Also removed duplicated part of tests.

llvm-svn: 325194
2018-02-15 00:40:36 +00:00