Commit Graph

8866 Commits

Author SHA1 Message Date
Peter Collingbourne fdef020d80 hwasan: Ignore loads and stores of size 0.
Now that memory intrinsics are instrumented, it's more likely that
CheckAddressSized will be called with size 0. (It was possible before
with IR like:

  %val = load [0 x i8], [0 x i8]* %ptr

but I don't think clang will generate IR like that and the optimizer
would normally remove it by the time it got anywhere near our pass
anyway). The right thing to do in both cases is to disable the
addressing checks (since the underlying memory intrinsic is a no-op),
so that's what we do.

Differential Revision: https://reviews.llvm.org/D56465

llvm-svn: 350683
2019-01-09 00:44:13 +00:00
David Carlier bfce66fbc1 [Sanitizer] Intercept fdevname on FreeBSD
- Is a file descriptor flavor FreeBSD's specific.
- reentrant version included.

Reviewers: vitalybuka, krytarowski, emaste

Reviewed By: emaste

Differential Revision: https://reviews.llvm.org/D56268

llvm-svn: 350658
2019-01-08 21:49:37 +00:00
Evgeniy Stepanov 43cba99775 Fix Mac compilation.
Provide an implementation of MemoryMappingLayout::Error() for Mac.

llvm-svn: 350601
2019-01-08 02:02:27 +00:00
Evgeniy Stepanov e1caa4ac72 [asan] Support running without /proc
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.

This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.

Reviewers: vitalybuka, pcc, krytarowski

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D56141

llvm-svn: 350590
2019-01-08 01:07:34 +00:00
Julian Lettner a6d29024ed [TSan] Support Objective-C @synchronized with tagged pointers
Summary:
Objective-C employs tagged pointers, that is, small objects/values may be encoded directly in the pointer bits. The resulting pointer is not backed by an allocation/does not point to a valid memory. TSan infrastructure requires a valid address for `Acquire/Release` and `Mutex{Lock/Unlock}`.
This patch establishes such a mapping via a "dummy allocation" for each encountered tagged pointer value.


Reviewers: dcoughlin, kubamracek, dvyukov, delcypher

Reviewed By: dvyukov

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D56238

llvm-svn: 350556
2019-01-07 19:19:23 +00:00
Petr Hosek 5c43496fa3 [compiler-rt][Fuchsia] Replace _zx_vmar_allocate_old call
This is the deprecated legacy interface, replace it with the current
_zx_vmar_allocate one.

Differential Revision: https://reviews.llvm.org/D56360

llvm-svn: 350488
2019-01-06 05:19:05 +00:00
Evgeniy Stepanov 045a83231d Revert "Revert "Switch Android from TLS_SLOT_TSAN(8) to TLS_SLOT_SANITIZER(6)""
This reapplies commit r348984.

llvm-svn: 350449
2019-01-05 00:45:14 +00:00
Peter Collingbourne 1c650debd7 Bring back the pthread_create interceptor, but only on non-aarch64.
We still need the interceptor on non-aarch64 to untag the pthread_t
and pthread_attr_t pointers and disable tagging on allocations done
internally by glibc.

llvm-svn: 350445
2019-01-04 23:24:02 +00:00
Evgeniy Stepanov 9fbc364e16 [sanitizer] Reduce stack depot size on Android.
Summary:
The default setting kTabSizeLog=20 results in an 8Mb global hash table,
almost all of it in private pages. That is not a sane setting in a
mobile, system-wide use case: with ~150 concurrent processes stack
depot will account for more than 1Gb of RAM.

Reviewers: kcc, pcc

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D56333

llvm-svn: 350443
2019-01-04 22:55:04 +00:00
Peter Collingbourne 87f477b5e4 hwasan: Implement lazy thread initialization for the interceptor ABI.
The problem is similar to D55986 but for threads: a process with the
interceptor hwasan library loaded might have some threads started by
instrumented libraries and some by uninstrumented libraries, and we
need to be able to run instrumented code on the latter.

The solution is to perform per-thread initialization lazily. If a
function needs to access shadow memory or add itself to the per-thread
ring buffer its prologue checks to see whether the value in the
sanitizer TLS slot is null, and if so it calls __hwasan_thread_enter
and reloads from the TLS slot. The runtime does the same thing if it
needs to access this data structure.

This change means that the code generator needs to know whether we
are targeting the interceptor runtime, since we don't want to pay
the cost of lazy initialization when targeting a platform with native
hwasan support. A flag -fsanitize-hwaddress-abi={interceptor,platform}
has been introduced for selecting the runtime ABI to target. The
default ABI is set to interceptor since it's assumed that it will
be more common that users will be compiling application code than
platform code.

Because we can no longer assume that the TLS slot is initialized,
the pthread_create interceptor is no longer necessary, so it has
been removed.

Ideally, lazy initialization should only cost one instruction in the
hot path, but at present the call may cause us to spill arguments
to the stack, which means more instructions in the hot path (or
theoretically in the cold path if the spills are moved with shrink
wrapping). With an appropriately chosen calling convention for
the per-thread initialization function (TODO) the hot path should
always need just one instruction and the cold path should need two
instructions with no spilling required.

Differential Revision: https://reviews.llvm.org/D56038

llvm-svn: 350429
2019-01-04 19:27:04 +00:00
Peter Collingbourne fcbcc6114f hwasan: Use system allocator to realloc and free untagged pointers in interceptor mode.
The Android dynamic loader has a non-standard feature that allows
libraries such as the hwasan runtime to interpose symbols even after
the symbol already has a value. The new value of the symbol is used to
relocate libraries loaded after the interposing library, but existing
libraries keep the old value. This behaviour is activated by the
DF_1_GLOBAL flag in DT_FLAGS_1, which is set by passing -z global to
the linker, which is what we already do to link the hwasan runtime.

What this means in practice is that if we have .so files that depend
on interceptor-mode hwasan without the main executable depending on
it, some of the libraries in the process will be using the hwasan
allocator and some will be using the system allocator, and these
allocators need to interact somehow. For example, if an instrumented
library calls a function such as strdup that allocates memory on
behalf of the caller, the instrumented library can reasonably expect
to be able to call free to deallocate the memory.

We can handle that relatively easily with hwasan by using tag 0 to
represent allocations from the system allocator. If hwasan's realloc
or free functions are passed a pointer with tag 0, the system allocator
is called.

One limitation is that this scheme doesn't work in reverse: if an
instrumented library allocates memory, it must free the memory itself
and cannot pass ownership to a system library. In a future change,
we may want to expose an API for calling the system allocator so
that instrumented libraries can safely transfer ownership of memory
to system libraries.

Differential Revision: https://reviews.llvm.org/D55986

llvm-svn: 350427
2019-01-04 19:21:51 +00:00
Evgeniy Stepanov d398471a63 [hwasan] Switch to 64 allocator with a dense size class map.
Summary:
Replace the 32-bit allocator with a 64-bit one with a non-constant
base address, and reduce both the number of size classes and the maximum
size of per-thread caches.

As measured on [1], this reduces average weighted memory overhead
(MaxRSS) from 26% to 12% over stock android allocator. These numbers
include overhead from code instrumentation and hwasan shadow (i.e. not a
pure allocator benchmark).

This switch also enables release-to-OS functionality, which is not
implemented in the 32-bit allocator. I have not seen any effect from
that on the benchmark.

[1] https://android.googlesource.com/platform/system/extras/+/master/memory_replay/

Reviewers: vitalybuka, kcc

Subscribers: kubamracek, cryptoad, llvm-commits

Differential Revision: https://reviews.llvm.org/D56239

llvm-svn: 350370
2019-01-03 23:19:02 +00:00
Julian Lettner ff43aa7e43 [TSan] Enable detection of lock-order-inversions for Objective-C @synchronized
Summary:
@synchronized semantics can be synthesized by using existing mutex_[un]lock operations.

```
@synchronized(obj) {
  // ...
}

=> 
{
  mutex_lock(obj);
  // ...
  mutex_unlock(obj);
}
```
Let me know whether you think this a good idea.

Reviewers: dcoughlin, dvyukov, kubamracek, delcypher

Reviewed By: dvyukov

Subscribers: llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55959

llvm-svn: 350258
2019-01-02 20:10:30 +00:00
David Carlier 205b709366 [Sanitizer] Enable funopen on FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56209

llvm-svn: 350248
2019-01-02 19:07:27 +00:00
Evgeniy Stepanov f5f1fb594b [sanitizer] Android does not provide <fstab.h>
llvm-svn: 350246
2019-01-02 19:05:26 +00:00
Michal Gorny 3d25e8d9f8 [sanitizer_common] Implement funopen*() interceptors for NetBSD
Differential Revision: https://reviews.llvm.org/D56158

llvm-svn: 350233
2019-01-02 17:37:14 +00:00
Michal Gorny 71a75307ae [sanitizer_common] Implement popen, popenve, pclose interceptors
Implement the interceptors for popen(), pclose() and popenve()
functions.  The first two are POSIX, the third one is specific
to NetBSD.  popen() spawns a process and creates a FILE object piping
data from/to that process.  pclose() closes the pipe and waits for
the process to terminate appropriately.

For the purpose of popen(), the COMMON_INTERCEPTOR_FILE_OPEN macro is
modified to allow null path parameter.

Differential Revision: https://reviews.llvm.org/D56157

llvm-svn: 350232
2019-01-02 17:37:09 +00:00
Michal Gorny bb1137da12 [sanitizer_common] Fix devname_r() return type on !NetBSD
Update the interceptor for devname_r() to account for correct return
types on different platforms.  This function returns int on NetBSD
but char* on FreeBSD/OSX.  Noticed by @krytarowski.

Differential Revision: https://reviews.llvm.org/D56150

llvm-svn: 350228
2019-01-02 17:36:50 +00:00
Kamil Rytarowski 646c130a18 Add support for background thread on NetBSD in ASan
Summary:
Change the point of calling MaybeStartBackgroudThread() from AsanInitInternal()
that is too early on NetBSD to a constructor (with aid of C++11 lambda construct).

Enable the code for background thread as is for NetBSD.

Rename test/sanitizer_common/TestCases/Linux/hard_rss_limit_mb_test.cc
to test/sanitizer_common/TestCases/hard_rss_limit_mb_test.cc and allow runs
on NetBSD. This tests passes correctly.

Reviewers: vitalybuka, joerg, eugenis

Reviewed By: eugenis

Subscribers: eugenis, kubamracek, fedor.sergeev, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55887

llvm-svn: 350139
2018-12-29 00:32:07 +00:00
David Carlier adb5f1e3b5 [Sanitizer] Enable getfsent api for Darwin
Reviewers: vitalybuka, kubamracek

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D56138

llvm-svn: 350138
2018-12-29 00:17:21 +00:00
Dan Liew 8c11fb3ed4 Introduce `LocalAddressSpaceView::LoadWritable(...)` and make the `Load(...)` method return a const pointer.
Summary:
This is a follow-up to r346956 (https://reviews.llvm.org/D53975).

The purpose of this change to allow implementers of the
`AddressSpaceView` to be able to distinguish between when a caller wants
read-only memory and when a caller wants writable memory. Being able
distinguish these cases allows implementations to optimize for the
different cases and also provides a way to workaround possible platform
restrictions (e.g. the low level platform interface for reading
out-of-process memory may place memory in read-only pages).

For allocator enumeration in almost all cases read-only is sufficient so
we make `Load(...)` take on this new requirement and introduce the
`LoadWritable(...)` variants for cases where memory needs to be
writable.

The behaviour of `LoadWritable(...)` documented in comments are
deliberately very restrictive so that it will be possible in the future
to implement a simple write-cache (i.e. just a map from target address
to a writable region of memory). These restrictions can be loosened in
the future if necessary by implementing a more sophisticated
write-cache.

rdar://problem/45284065

Reviewers: kcc, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54879

llvm-svn: 350136
2018-12-28 19:30:51 +00:00
Reid Kleckner 1c6dbd6921 Speculative fix for xray assembler error on MachO since r349976
This .file directive wasn't doing anything on MachO, and now that's
diagnosed as an error.

llvm-svn: 350132
2018-12-28 18:53:00 +00:00
David Carlier f11fcc046e [Sanitizer] Intercept arc4random_buf / arc4random_addrandom on FreeBSD/NetBSD
- Disabled on purpose on Android and Darwin platform (for now).
- Darwin supports it, would need interception in its specific code before enabling it.
- Linux does not support it but only via third party library.
- Android supports it via bionic however it is known to have issue with older versions of the implementations. Can be enabled by an Android committer later on if necessary once there is more 'certainity'/been more tested.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56125

llvm-svn: 350123
2018-12-28 16:04:28 +00:00
Evgeniy Stepanov 3408b497f9 Revert "[asan] Support running without /proc.", +1
Revert r350104 "[asan] Fix build on windows."
Revert r350101 "[asan] Support running without /proc."

These changes break Mac build, too.

llvm-svn: 350112
2018-12-28 01:27:18 +00:00
Kamil Rytarowski 10828c39cb Fix a mistake in previous
Assign correct ioctl(2) operation number.

llvm-svn: 350110
2018-12-28 00:17:37 +00:00
Kamil Rytarowski 8e56064542 Update NetBSD ioctl(2) entries with 8.99.28
Add SIOCSETHERCAP.

Add commented out NVMM (NetBSD Virtual Machine Monitor) operations
as this interface is still WIP and a subject to change.

llvm-svn: 350107
2018-12-27 23:35:28 +00:00
Evgeniy Stepanov 1ce7fccbc9 [asan] Fix build on windows.
Reviewers: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D56111

llvm-svn: 350104
2018-12-27 22:16:28 +00:00
Evgeniy Stepanov e9d00896bd Enable posix regex interceptors on linux.
Summary: They happen to work out of the box.

Reviewers: rtrieu, vitalybuka

Subscribers: kubamracek, fedor.sergeev, krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D56088

llvm-svn: 350103
2018-12-27 21:47:26 +00:00
Kamil Rytarowski 93e3b34ceb Reimplement Thread Static Data MSan routines with TLS
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the MSan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.

New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.

Reviewers: joerg, vitalybuka, jfb

Reviewed By: vitalybuka

Subscribers: emaste, jfb, llvm-commits, #sanitizers, mgorny

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55703

llvm-svn: 350102
2018-12-27 21:43:46 +00:00
Evgeniy Stepanov d9df65fd0d [asan] Support running without /proc.
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.

This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.

Reviewers: pcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55874

llvm-svn: 350101
2018-12-27 21:31:52 +00:00
David Carlier 774f7e83c3 [Sanitizer] Add fstab api to FreeBSD/Linux
Interception of /etc/fstab function parsers.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56099

llvm-svn: 350099
2018-12-27 21:25:24 +00:00
Jordan Rupprecht 4009724151 [sanitizer] Avoid memset call in tsan
Summary: Assigning an array of bools to {false, false, false} can, in certain build configurations, lead to a memset call. Use internal_memset to avoid this.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D56107

llvm-svn: 350098
2018-12-27 20:29:10 +00:00
David Carlier 018f505c9e [Sanitizer] Enable FTS api on FreeBSD
Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56097

llvm-svn: 350090
2018-12-27 12:56:54 +00:00
Martin Liska 7240eb3ba9 Do not rely on that subject of ErrorAllocTypeMismatch is a heap address.
Differential Revision: https://reviews.llvm.org/D54856.

llvm-svn: 350085
2018-12-27 08:39:13 +00:00
Evgeniy Stepanov 772bf988cf Revert "[msan] Disable tail call optimization in msan rtl."
This reverts commit r350080, which breaks
* gcc as the host compiler
* some tests on ppc64

llvm-svn: 350083
2018-12-27 03:30:42 +00:00
Evgeniy Stepanov faef9b77de [msan] Disable tail call optimization in msan rtl.
This should give us better error stack traces on the bots, in
particular, for https://bugs.llvm.org/show_bug.cgi?id=40162

llvm-svn: 350080
2018-12-27 00:21:20 +00:00
Michal Gorny a939b40eae [xray] Detect MPROTECT and error out when it's enabled (on NetBSD)
Add a CheckMPROTECT() routine to detect when pax MPROTECT is enabled
on NetBSD, and error xray out when it is.  The solution is adapted
from existing CheckASLR().

Differential Revision: https://reviews.llvm.org/D56049

llvm-svn: 350030
2018-12-23 15:09:28 +00:00
Michal Gorny 470ce63251 [xray] Disable alignas() for thread_local objects on NetBSD
Disable enforcing alignas() for structs that are used as thread_local
data on NetBSD.  The NetBSD ld.so implementation is buggy and does
not enforce correct alignment; however, clang seems to take it for
granted and generates instructions that segv on wrongly aligned objects.
Therefore, disable those alignas() statements on NetBSD until we can
establish a better fix.

Apparently, std::aligned_storage<> does not have any real effect
at the moment, so we can leave it as-is.

Differential Revision: https://reviews.llvm.org/D56000

llvm-svn: 350029
2018-12-23 15:09:20 +00:00
David Carlier 58d3823086 [Sanitizer] Enable POSIX regex api on FreeBSD.
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56009

M    lib/sanitizer_common/sanitizer_common_interceptors.inc
M    lib/sanitizer_common/sanitizer_platform_interceptors.h
M    lib/sanitizer_common/sanitizer_platform_limits_freebsd.cc
M    lib/sanitizer_common/sanitizer_platform_limits_freebsd.h
D    test/sanitizer_common/TestCases/NetBSD/regex.cc
A  + test/sanitizer_common/TestCases/Posix/regex.cc

llvm-svn: 350002
2018-12-22 11:17:27 +00:00
Kamil Rytarowski edbe2b3f02 Add support for LLVM profile for NetBSD
Summary:
NetBSD uses typical UNIX interfaces.

All tests pass except instrprof-dlopen-dlclose-gcov.test, as there
is not supported semantics of atexit(3) in dlopen(3)ed+dlclose(3)d
DSO.

NetBSD also ships an older version of LLVM profile (ABI v.2 predating
ABI v.4 in upstream version) inside libc. That copy has been manually
removed during the porting and testing process of the upstream version
to NetBSD. Otherwise there were conflicts between them two.

Reviewers: joerg, vitalybuka, vsk

Subscribers: srhines, fedor.sergeev, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55997

llvm-svn: 349994
2018-12-22 06:56:19 +00:00
Dan Liew c6027e20d4 Fix comment typo.
llvm-svn: 349961
2018-12-21 21:41:43 +00:00
Dan Liew a8334ed571 Fix `static_assert()` scope in `CombinedAllocator`.
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.

This is a follow up to r349957.

rdar://problem/45284065

llvm-svn: 349960
2018-12-21 21:41:37 +00:00
Dan Liew 09f6d77840 Fix `static_assert()` scope in `SizeClassAllocator32`.
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.

This is a follow up to r349138.

rdar://problem/45284065

llvm-svn: 349959
2018-12-21 21:41:31 +00:00
Dan Liew 14e0d9ed89 Introduce `AddressSpaceView` template parameter to `CombinedAllocator`.
Summary:
This is a follow up to https://reviews.llvm.org/D55764 .

For the ASan and LSan allocatorsthe type declarations have been modified
so that it's possible to create a combined allocator type that
consistently uses a different type of `AddressSpaceView`. We intend to
use this in future patches. For the other sanitizers they just use
`LocalAddressSpaceView` by default because we have no plans to use these
allocators in an out-of-process manner.

rdar://problem/45284065

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov, yln

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55766

llvm-svn: 349957
2018-12-21 21:22:27 +00:00
Dan Liew d2c6af7358 Introduce `AddressSpaceView` template parameter to `SizeClassAllocator64`.
Summary:
This is a follow up patch to r349138.

This patch makes a `AddressSpaceView` a type declaration in the
allocator parameters used by `SizeClassAllocator64`. For ASan, LSan, and
the unit tests the AP64 declarations have been made templated so that
`AddressSpaceView` can be changed at compile time. For the other
sanitizers we just hard-code `LocalAddressSpaceView` because we have no
plans to use these allocators in an out-of-process manner.

rdar://problem/45284065

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55764

llvm-svn: 349954
2018-12-21 21:09:31 +00:00
David Carlier 7c21d95c9d [Sanitizer] Enable strtonum in FreeBSD
Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55993

llvm-svn: 349916
2018-12-21 15:42:24 +00:00
Michal Gorny 1e8e4fc834 [xray] [tests] Detect and handle missing LLVMTestingSupport gracefully
Add a code to properly test for presence of LLVMTestingSupport library
when performing a stand-alone build, and skip tests requiring it when
it is not present.  Since the library is not installed, llvm-config
reported empty --libs for it and the tests failed to link with undefined
references.  Skipping the two fdr_* test files is better than failing to
build, and should be good enough until we find a better solution.

NB: both installing LLVMTestingSupport and building it automatically
from within compiler-rt sources are non-trivial.  The former due to
dependency on gtest, the latter due to tight integration with LLVM
source tree.

Differential Revision: https://reviews.llvm.org/D55891

llvm-svn: 349899
2018-12-21 13:37:30 +00:00
Evgeniy Stepanov 3b7e8b2dbb [sanitizer] Support running without fd 0,1,2.
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.

(2nd attempt, this time without the sanitizer_rtems change)

Reviewers: pcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55801

llvm-svn: 349817
2018-12-20 20:36:33 +00:00
David Carlier 22594ae962 [Sanitizer] Enable vis api on FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55923

llvm-svn: 349762
2018-12-20 14:25:43 +00:00
Ilya Biryukov ea8646ad66 Revert "[sanitizer] Support running without fd 0,1,2."
This reverts commit r349699.
Reason: the commit breaks compilation of sanitizer_rtems.cc when
building for RTEMS.

llvm-svn: 349745
2018-12-20 12:50:03 +00:00
Eugene Leviant d3bd614856 [HWASAN] Add support for memory intrinsics
This is patch complements D55117 implementing __hwasan_mem*
functions in runtime

Differential revision: https://reviews.llvm.org/D55554

llvm-svn: 349730
2018-12-20 09:10:03 +00:00
Evgeniy Stepanov 17e705129e Remove pointless casts.
llvm-svn: 349717
2018-12-20 01:49:21 +00:00
Evgeniy Stepanov f762a9f8f0 [sanitizer] Support running without fd 0,1,2.
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.

Reviewers: pcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55801

llvm-svn: 349699
2018-12-19 23:45:17 +00:00
Kostya Kortchinsky ebaba9df3a [sanitizer] Remove spurious semi-colon
Summary:
An extra ';' at the end of a namespace triggers a pedantic warning:
```
.../sanitizer_common/sanitizer_type_traits.h:42:2: warning: extra ‘;’ [-Wpedantic]
 };  // namespace __sanitizer
```

Reviewers: eugenis, delcypher

Reviewed By: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55849

llvm-svn: 349666
2018-12-19 19:05:29 +00:00
Michal Gorny 0f71c17e7c [sanitizer_common] Fix sha2 interceptors not to use vars in array len
Fix the sha2 interceptor macros to use a constant for array parameter
length rather than referencing the extern variable.  Since the digest
length is provided in hash name, reuse the macro parameter for it.
Verify that the calculated value matches the one provided by system
headers.

Differential Revision: https://reviews.llvm.org/D55811

llvm-svn: 349645
2018-12-19 17:22:14 +00:00
Kamil Rytarowski 342d03e71f Reimplement Thread Static Data ASan routines with TLS
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the ASan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.

New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.

Reviewers: joerg, vitalybuka, jfb

Reviewed By: vitalybuka

Subscribers: dim, emaste, ro, jfb, devnexen, kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55596

llvm-svn: 349619
2018-12-19 11:11:29 +00:00
Dmitry Vyukov 2ba80d6df6 tsan: align default value of detect_deadlocks flag with actual behavior
I tricked myself into thinking that deadlock detection is off by default in TSan by looking at the default value of the detect_deadlocks flag and outdated docs. (Created a pull request to update docs.)
I even managed to confuse others: https://groups.google.com/forum/#!topic/thread-sanitizer/xYvnAYwtoDk

However, the default value is overwritten in code (TSan_flags.cc:InitializeFlags). The TSan/deadlock tests also rely on this

This changes aligns the default value of the flag with the actual default behavior.

Author: yln (Julian Lettner)
Reviewed in: https://reviews.llvm.org/D55846

llvm-svn: 349609
2018-12-19 09:34:13 +00:00
Nico Weber cfa54fb456 Fix a gcc -Wpedantix warning
llvm-svn: 349492
2018-12-18 15:17:01 +00:00
Peter Collingbourne 44ea4f5744 hwasan: Allow range of frame descriptors to be empty.
As of r349413 it's now possible for a binary to contain an empty
hwasan frame section. Handle that case simply by doing nothing.

Differential Revision: https://reviews.llvm.org/D55796

llvm-svn: 349428
2018-12-18 00:48:07 +00:00
Dan Liew b85b15d64a Don't trigger sanitizer initialization from `sysctlbyname` and `sysctl` interceptor.
Summary:
This fixes the `ThreadSanitizer-x86_64-iossim` testsuite which broke
when r348770 (https://reviews.llvm.org/D55473) landed.

The root cause of the problem is that early-on during the iOS simulator
init process a call to `sysctlbyname` is issued. If the TSan initializer
is triggered at this point it will eventually trigger a call to
`__cxa_at_exit(...)`. This call then aborts because the library
implementing this function is not yet had its initialization function
called.

rdar://problem/46696934

Reviewers: kubamracek, george.karpenkov, devnexen, vitalybuka, krytarowski

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55711

llvm-svn: 349402
2018-12-17 21:42:55 +00:00
David Carlier 34144f5ec0 [Sanitizer] capsicum variadic api subset
Reviewers: markj, vitalybuka

Reviewed By: markj

Differential Revision: https://reviews.llvm.org/D55714

llvm-svn: 349392
2018-12-17 20:57:06 +00:00
Kamil Rytarowski ca72239534 Improve the comment in previous
llvm-svn: 349296
2018-12-16 10:24:06 +00:00
Kamil Rytarowski bb66b0dac1 Expand TSan sysroot workaround to NetBSD
https://bugs.llvm.org/show_bug.cgi?id=26651

llvm-svn: 349295
2018-12-16 10:22:30 +00:00
Kamil Rytarowski 17751e1414 Enable SANITIZER_INTERCEPT_PTHREAD_GETNAME_NP for NetBSD
llvm-svn: 349262
2018-12-15 10:41:55 +00:00
Kamil Rytarowski 5748bb4133 Fix internal_sleep() for NetBSD
This is a follow up of a similar fix for Linux from D55692.

llvm-svn: 349257
2018-12-15 08:25:25 +00:00
Peter Collingbourne 1ad24332d1 hwasan: Fix typo: Previosly -> Previously.
llvm-svn: 349254
2018-12-15 07:06:24 +00:00
Kostya Serebryany 1879e8d3fc [libFuzzer] make len_control less aggressive
llvm-svn: 349210
2018-12-14 23:21:31 +00:00
Dan Liew 41fec1bfc5 Introduce `AddressSpaceView` template parameter to `SizeClassAllocator32`, `FlatByteMap`, and `TwoLevelByteMap`.
Summary:
This is a follow up patch to r346956 for the `SizeClassAllocator32`
allocator.

This patch makes `AddressSpaceView` a template parameter both to the
`ByteMap` implementations (but makes `LocalAddressSpaceView` the
default), some `AP32` implementations and is used in `SizeClassAllocator32`.
The actual changes to `ByteMap` implementations and
`SizeClassAllocator32` are very simple. However the patch is large
because it requires changing all the `AP32` definitions, and users of
those definitions.

For ASan and LSan we make `AP32` and `ByteMap` templateds type that take
a single `AddressSpaceView` argument. This has been done because we will
instantiate the allocator with a type that isn't `LocalAddressSpaceView`
in the future patches. For the allocators used in the other sanitizers
(i.e. HWAsan, MSan, Scudo, and TSan) use of `LocalAddressSpaceView` is
hard coded because we do not intend to instantiate the allocators with
any other type.

In the cases where untemplated types have become templated on a single
`AddressSpaceView` parameter (e.g. `PrimaryAllocator`) their name has
been changed to have a `ASVT` suffix (Address Space View Type) to
indicate they are templated.  The only exception to this are the `AP32`
types due to the desire to keep the type name as short as possible.

In order to check that template is instantiated in the correct a way a
`static_assert(...)` has been added that checks that the
`AddressSpaceView` type used by `Params::ByteMap::AddressSpaceView` matches
the `Params::AddressSpaceView`. This uses the new `sanitizer_type_traits.h`
header.

rdar://problem/45284065

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: mgorny, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D54904

llvm-svn: 349138
2018-12-14 09:03:18 +00:00
Fangrui Song c4e6de2e3f [sanitizer] Fix nolibc internal_sleep
Reviewers: kubamracek, vitalybuka

Reviewed By: vitalybuka

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D55692

llvm-svn: 349134
2018-12-14 08:09:43 +00:00
Vlad Tsyrklevich 137e23d536 Windows ASan: Instrument _msize_base()
Summary:
A recent update to the VS toolchain in chromium [1] broke the windows
ASan bot because the new toolchain calls _msize_base() instead of
_msize() in a number of _aligned_* UCRT routines. Instrument
_msize_base() as well.

[1] https://crbug.com/914947

Reviewers: rnk, #sanitizers, vitalybuka

Reviewed By: rnk, #sanitizers, vitalybuka

Subscribers: vitalybuka, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55684

llvm-svn: 349115
2018-12-14 00:39:16 +00:00
Evgeniy Stepanov e6acf2c3b4 Revert "Switch Android from TLS_SLOT_TSAN(8) to TLS_SLOT_SANITIZER(6)"
Breaks sanitizer-android buildbot.

This reverts commit 85e02baff327e7b67ea5b47897302901abb2aa5d.

llvm-svn: 349093
2018-12-13 23:47:59 +00:00
Dan Liew c2e2dd43f0 Implement a small subset of the C++ `type_traits` header inside sanitizer_common so we can avoid depending on system C++ headers.
Summary:
In particular we implement the `is_same<T,U>` templated type. This is
useful for doing compile-time comparison of types in `static_assert`s.
The plan is to use this in another patch (
https://reviews.llvm.org/D54904 ).

Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: mgorny, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54951

llvm-svn: 349077
2018-12-13 19:55:36 +00:00
David Carlier a4ee854259 [Sanitizer] capsicum further support of the API
Reviewers: vitalybuka, krytarowski, emaste

Reviewed By: emaste

Differential Revision: https://reviews.llvm.org/D55622

llvm-svn: 349042
2018-12-13 15:05:24 +00:00
Kamil Rytarowski 0bc12879fc Add a new interceptors for cdbr(3) and cdbw(3) API from NetBSD
Summary:
cdb - formats of the constant database.

cdbr, cdbr_open, cdbr_open_mem, cdbr_entries, cdbr_get, cdbr_find,
cdbr_close - constant database access methods.

cdbw_open, cdbw_put, cdbw_put_data, cdbw_put_key, cdbw_stable_seeder,
cdbw_output, cdbw_close - creates constant databases.

Add a dedicated test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55167

llvm-svn: 349021
2018-12-13 10:19:00 +00:00
Kamil Rytarowski ce86b919da Add new interceptors for vis(3) API in NetBSD
Summary:
Add interceptors for the NetBSD style of vis(3) present inside libc:

 - vis
 - nvis
 - strvis
 - stravis
 - strnvis
 - strvisx
 - strnvisx
 - strenvisx
 - svis
 - snvis
 - strsvis
 - strsnvis
 - strsvisx
 - strsnvisx
 - strsenvisx
 - unvis
 - strunvis
 - strnunvis
 - strunvisx
 - strnunvisx

Add a dedicated test verifying the installed interceptors.

Based on original work by Yang Zheng.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54594

llvm-svn: 349018
2018-12-13 10:14:01 +00:00
Vitaly Buka a257639a69 [asan] Don't check ODR violations for particular types of globals
Summary:
private and internal: should not trigger ODR at all.
unnamed_addr: current ODR checking approach fail and rereport false violation if
a linker merges such globals
linkonce_odr, weak_odr: could cause similar problems and they are already not
instrumented for ELF.

Reviewers: eugenis, kcc

Subscribers: kubamracek, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D55621

llvm-svn: 349015
2018-12-13 09:47:39 +00:00
Evgeniy Stepanov 1020085898 [hwasan] Link ubsan_cxx to shared runtime library.
Summary: This is needed for C++-specific ubsan and cfi error reporting to work.

Reviewers: kcc, vitalybuka

Subscribers: srhines, kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D55589

llvm-svn: 348986
2018-12-12 22:56:00 +00:00
Ryan Prichard b7e1cae526 Switch Android from TLS_SLOT_TSAN(8) to TLS_SLOT_SANITIZER(6)
Summary:
The TLS_SLOT_TSAN slot is available starting in N, but its location (8)
is incompatible with the proposed solution for implementing ELF TLS on
Android (i.e. bump ARM/AArch64 alignment to reserve an 8-word TCB).

Instead, starting in Q, Bionic replaced TLS_SLOT_DLERROR(6) with
TLS_SLOT_SANITIZER(6). Switch compiler-rt to the new slot.

Reviewers: eugenis, srhines, enh

Reviewed By: eugenis

Subscribers: ruiu, srhines, kubamracek, javed.absar, kristof.beyls, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D55581

llvm-svn: 348984
2018-12-12 22:45:12 +00:00
Evgeniy Stepanov 0af6e5facd [hwasan] Verify Android TLS slot at startup.
Summary:
Add a check that TLS_SLOT_TSAN / TLS_SLOT_SANITIZER, whichever
android_get_tls_slot is using, is not conflicting with
TLS_SLOT_DLERROR.

Reviewers: rprichard, vitalybuka

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55587

llvm-svn: 348979
2018-12-12 22:10:52 +00:00
David Carlier 8ec89e6b16 [Sanitizer] Expand FSEEK interception to FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55556

llvm-svn: 348888
2018-12-11 19:08:40 +00:00
Dan Liew a94b00513f Fix bug where we'd try symbolize a second time with the same arguments.
Summary:
Fix bug where we'd try symbolize a second time with the same arguments even though symbolization failed the first time.

This looks like a long standing typo given that the guard for trying
symbolization again is to only try it if symbolization failed using
`binary` and `original_binary != binary`.

Reviewers: kubamracek, glider, samsonov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55504

llvm-svn: 348841
2018-12-11 12:43:44 +00:00
Yi Kong a140d5269e [builtins] Remove trailing whitespaces, NFC
Remove trailing whitespaces so that it is easier to diff the code between
div{s,d,t}f3.c

llvm-svn: 348807
2018-12-10 22:52:59 +00:00
David Carlier 2b26a98a0d [Sanitizer] expand sysctl/getmntinfo/nl_langinfo to Darwin
Reviewers: vitalybuka, krytarowski, kubamracek

Reviewed By: vitalybuka, krytarowski

Differential Revision: https://reviews.llvm.org/D55473

llvm-svn: 348770
2018-12-10 16:29:30 +00:00
Kamil Rytarowski 2eb1f5432f Add data types needed for md2(3)/NetBSD interceptors
Missing part of D55469.

llvm-svn: 348747
2018-12-10 09:09:18 +00:00
Kamil Rytarowski 50bd2ec198 Add interceptors for the sha2(3) from NetBSD
Summary:
SHA224_Init, SHA224_Update, SHA224_Final, SHA224_End, SHA224_File,
SHA224_FileChunk, SHA224_Data, SHA256_Init, SHA256_Update, SHA256_Final,
SHA256_End, SHA256_File, SHA256_FileChunk, SHA256_Data, SHA384_Init,
SHA384_Update, SHA384_Final, SHA384_End, SHA384_File, SHA384_FileChunk,
SHA384_Data, SHA512_Init, SHA512_Update, SHA512_Final, SHA512_End,
SHA512_File, SHA512_FileChunk, SHA512_Data – calculates the NIST Secure
Hash Standard (version 2)

Add tests for new interceptors.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54987

llvm-svn: 348745
2018-12-10 09:06:56 +00:00
Kamil Rytarowski e7971f1bce Add interceptors for md2(3) from NetBSD
Summary:
MD2Init, MD2Update, MD2Final, MD2End, MD2File, MD2Data - calculates the
RSA Data Security, Inc., "MD2" message digest.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55469

llvm-svn: 348744
2018-12-10 09:01:00 +00:00
Kamil Rytarowski 14d36e6e75 Add new interceptors for FILE repositioning stream
Summary:
Add new interceptors for a set of functions to reposition a stream:
fgetpos, fseek, fseeko, fsetpos, ftell, ftello, rewind .

Add a dedicated test.

Enable this interface on NetBSD.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55471

llvm-svn: 348743
2018-12-10 08:56:14 +00:00
Kamil Rytarowski 3ac7a15163 Revert a chunk of previous change in sanitizer_platform_limits_netbsd.h
Undefining INLINE breaks the build.
The invalid change in this file has been overlooked in D55386.

llvm-svn: 348680
2018-12-08 02:47:12 +00:00
Kamil Rytarowski 2e11b93f35 Add interceptors for md5(3) from NetBSD
Summary:
MD5Init, MD5Update, MD5Final, MD5End, MD5File, MD5Data - calculates the
RSA Data Security, Inc., "MD5" message digest.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54993

llvm-svn: 348679
2018-12-08 01:50:18 +00:00
Kamil Rytarowski f9b8569d72 Add interceptors for the rmd160(3) from NetBSD
Summary:
RMD160Init, RMD160Update, RMD160Final, RMD160Transform, RMD160End,
RMD160File, RMD160Data - calculates the ``RIPEMD-160'' message digest.

Add a dedicated test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54991

llvm-svn: 348678
2018-12-08 01:47:29 +00:00
Kamil Rytarowski e9bf12f82f Add interceptors for the md4(3) from NetBSD
Summary:
MD4Init, MD4Update, MD4Final, MD4End, MD4File, MD4Data - calculates the
RSA Data Security, Inc., "MD4" message digest.

Add dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54990

llvm-svn: 348677
2018-12-08 01:43:39 +00:00
Kamil Rytarowski 3f47a6fbd7 Add interceptors for the sha1(3) from NetBSD
Summary:
Add interceptors for:

 - SHA1Init
 - SHA1Update
 - SHA1Final
 - SHA1Transform
 - SHA1End
 - SHA1File
 - SHA1FileChunk
 - SHA1Data

Add a dedicated regression test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: mgorny, llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54927

llvm-svn: 348676
2018-12-08 01:39:47 +00:00
Vitaly Buka 4e23840e30 [sanitizer] Suppress lint warning conflicting with clang-format
llvm-svn: 348673
2018-12-08 01:00:37 +00:00
David Carlier aaa95cbeea Fix style.
llvm-svn: 348672
2018-12-08 00:44:38 +00:00
David Carlier cc3be702b0 Conflict fixes from previous commits.
llvm-svn: 348669
2018-12-08 00:21:40 +00:00
David Carlier a0d0202d89 [Sanitizer] capsicum api subset interception
- For the moment a subset of this api dealing with file descriptors permissions and ioctls.

Reviewers: vitalybuka, krytarowski

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D55368

llvm-svn: 348668
2018-12-08 00:14:04 +00:00
Kamil Rytarowski ae3ae31e9c Add interceptors for the strtoi(3)/strtou(3) from NetBSD
Summary:
strtoi/strtou converts string value to an intmax_t/uintmax_t integer.

Add a dedicated test.

Enable this API for NetBSD.

It's a reworked version of the original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, tomsun.0.7, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54702

llvm-svn: 348663
2018-12-07 22:24:35 +00:00
Dan Liew 5ab96bf57c Fix IOError exception being raised in `asan_symbolize.py`crash when
using `atos` symbolizer on Darwin when the binaries don't exist.

For now we just produce an unsymbolicated stackframe when the binary
doesn't exist.

llvm-svn: 348659
2018-12-07 22:14:20 +00:00
Kamil Rytarowski 2f5fd174c9 Add a new interceptors for statvfs1(2) and fstatvfs1(2) from NetBSD
Summary:
statvfs1, fstatvfs1 - get file system statistics.

While there, use file descriptor related macros in the fstatvfs interceptor.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: dvyukov, kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55031

llvm-svn: 348656
2018-12-07 22:01:16 +00:00
Kamil Rytarowski 592898b306 Add a new interceptor for fparseln(3) from NetBSD
Summary:
fparseln - returns the next logical line from a stream.

Add a dedicated test for this API.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55019

llvm-svn: 348654
2018-12-07 21:50:44 +00:00
Kamil Rytarowski 0fed92a933 Add new interceptor for strtonum(3)
Summary:
strtonum(3) reliably convertss string value to an integer.
This function is used in OpenBSD compat namespace
and is located inside NetBSD's libc.

Add a dedicated test for this interface.

It's a reworked version of the original code by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54527

llvm-svn: 348651
2018-12-07 21:47:36 +00:00
David Carlier a742193309 Missing freebsd files.
A    lib/sanitizer_common/sanitizer_platform_limits_freebsd.cc
A    lib/sanitizer_common/sanitizer_platform_limits_freebsd.h

llvm-svn: 348635
2018-12-07 20:07:49 +00:00
David Carlier da2a653134 [Sanitizer] Separate FreeBSD interception data structures
Reviewers: vitalybuka, krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55386

llvm-svn: 348634
2018-12-07 20:05:55 +00:00
Dean Michael Berris 25d505953a [XRay] Use preallocated memory for XRay profiling
Summary:
This change builds upon D54989, which removes memory allocation from the
critical path of the profiling implementation. This also changes the API
for the profile collection service, to take ownership of the memory and
associated data structures per-thread.

The consolidation of the memory allocation allows us to do two things:

- Limits the amount of memory used by the profiling implementation,
  associating preallocated buffers instead of allocating memory
  on-demand.

- Consolidate the memory initialisation and cleanup by relying on the
  buffer queue's reference counting implementation.

We find a number of places which also display some problematic
behaviour, including:

- Off-by-factor bug in the allocator implementation.

- Unrolling semantics in cases of "memory exhausted" situations, when
  managing the state of the function call trie.

We also add a few test cases which verify our understanding of the
behaviour of the system, with important edge-cases (especially for
memory-exhausted cases) in the segmented array and profile collector
unit tests.

Depends on D54989.

Reviewers: mboerger

Subscribers: dschuff, mgorny, dmgreen, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D55249

llvm-svn: 348568
2018-12-07 06:23:06 +00:00
Dean Michael Berris 190c49bc8f Re-land "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
This reverts commit r348455, with some additional changes:

- Work-around deficiency of gcc-4.8 by duplicating the implementation of
  `AppendEmplace` in `Append`, but instead of using brace-init for the
  copy construction, use a placement new explicitly calling the copy
  constructor.

llvm-svn: 348563
2018-12-07 03:19:13 +00:00
Michal Gorny ef4b600301 [test] Add missing cmake include for building libFuzzer alone
Include CompilerRTCompile in fuzzer tests explicitly.  Otherwise, when
building only libFuzzer, CMake fails due to:

CMake Error at cmake/Modules/AddCompilerRT.cmake:395 (sanitizer_test_compile):
  Unknown CMake command "sanitizer_test_compile".
Call Stack (most recent call first):
  lib/fuzzer/tests/CMakeLists.txt:53 (generate_compiler_rt_tests)

Differential Revision: https://reviews.llvm.org/D55378

llvm-svn: 348524
2018-12-06 20:04:08 +00:00
David Carlier 51e820d0d8 [Sanitizer] getmntinfo support in FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55354

llvm-svn: 348500
2018-12-06 17:04:18 +00:00
Dan Liew f101eb101d Add new `__sanitizer_mz_default_zone()` API which returns the address of the ASan malloc zone. This API will be used for testing in future patches.
Summary:
The name of the function is based on `malloc_default_zone()` found
in Darwin's `malloc/malloc.h` header file.

Reviewers: kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D55064

llvm-svn: 348481
2018-12-06 12:39:00 +00:00
Dean Michael Berris 82f7b21f17 Revert "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
This reverts commits r348438, r348445, and r348449 due to breakages with
gcc-4.8 builds.

llvm-svn: 348455
2018-12-06 03:28:57 +00:00
Dean Michael Berris 889d20715e [XRay] Use a local lvalue as arg to AppendEmplace(...)
This is a follow-up to D54989.

Further work-around gcc-4.8 failing to handle brace-init with temporaries.

llvm-svn: 348449
2018-12-06 02:55:47 +00:00
Dean Michael Berris 400afa0230 [XRay] Use default-constructed struct as argument to Append(...)
This is a follow-up to D54989.

Work-around gcc-4.8 failing to handle brace-init for structs to imply
default-construction of an aggregate, and treats it as an initialiser
list instead.

llvm-svn: 348445
2018-12-06 01:56:27 +00:00
Dean Michael Berris cb447a2604 Re-land r348335 "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
Continuation of D54989.

Additional changes:

  - Use `.AppendEmplace(...)` instead of `.Append(Type{...})` to appease
    GCC 4.8 with confusion on when an initializer_list is used as
    opposed to a temporary aggregate initialized object.

llvm-svn: 348438
2018-12-06 00:25:56 +00:00
David Carlier 82494cb8cf [Sanitizer] nl_langinfo forgotten bit.
M    lib/sanitizer_common/sanitizer_platform_interceptors.h

llvm-svn: 348377
2018-12-05 16:02:26 +00:00
Kamil Rytarowski efadb532bb Add a new interceptor for modctl(2) from NetBSD
Summary:
modctl - controls loadable kernel modules.

Skip tests as this call uses privileged operations.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55025

llvm-svn: 348370
2018-12-05 15:13:20 +00:00
Kamil Rytarowski fab764359f Add a new interceptor for nl_langinfo(3) from NetBSD
Summary:
nl_langinfo - gets locale information.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55015

llvm-svn: 348369
2018-12-05 15:06:53 +00:00
Hans Wennborg 83ff22c297 Revert r348335 "[XRay] Move-only Allocator, FunctionCallTrie, and Array"
.. and also the follow-ups r348336 r348338.

It broke stand-alone compiler-rt builds with GCC 4.8:

In file included from /work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:20:0,
                 from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.h:21,
                 from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:15:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&}; T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71:   required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’
/work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:517:54:   required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget’
     new (AlignedOffset) T{std::forward<Args>(args)...};
     ^
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ThreadTrie&}; T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71:   required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:98:34:   required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ThreadTrie&>((* & args#0))}’ from
‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ThreadTrie’
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ProfileBuffer&}; T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71:   required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]
’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:244:44:   required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ProfileBuffer&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ProfileBuffer’

> Summary:
> This change makes the allocator and function call trie implementations
> move-aware and remove the FunctionCallTrie's reliance on a
> heap-allocated set of allocators.
>
> The change makes it possible to always have storage associated with
> Allocator instances, not necessarily having heap-allocated memory
> obtainable from these allocator instances. We also use thread-local
> uninitialised storage.
>
> We've also re-worked the segmented array implementation to have more
> precondition and post-condition checks when built in debug mode. This
> enables us to better implement some of the operations with surrounding
> documentation as well. The `trim` algorithm now has more documentation
> on the implementation, reducing the requirement to handle special
> conditions, and being more rigorous on the computations involved.
>
> In this change we also introduce an initialisation guard, through which
> we prevent an initialisation operation from racing with a cleanup
> operation.
>
> We also ensure that the ThreadTries array is not destroyed while copies
> into the elements are still being performed by other threads submitting
> profiles.
>
> Note that this change still has an issue with accessing thread-local
> storage from signal handlers that are instrumented with XRay. We also
> learn that with the testing of this patch, that there will be cases
> where calls to mmap(...) (through internal_mmap(...)) might be called in
> signal handlers, but are not async-signal-safe. Subsequent patches will
> address this, by re-using the `BufferQueue` type used in the FDR mode
> implementation for pre-allocated memory segments per active, tracing
> thread.
>
> We still want to land this change despite the known issues, with fixes
> forthcoming.
>
> Reviewers: mboerger, jfb
>
> Subscribers: jfb, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D54989

llvm-svn: 348346
2018-12-05 10:19:55 +00:00
Dean Michael Berris 9e4b7efa0e [XRay] Use uptr instead of uintptr_t
Follow-up to D54989.

llvm-svn: 348338
2018-12-05 07:14:06 +00:00
Dean Michael Berris d49fc9c6fa [XRay] Use deallocateBuffer instead of deallocate
Follow-up to D54989.

llvm-svn: 348336
2018-12-05 07:05:44 +00:00
Dean Michael Berris adc880467d [XRay] Move-only Allocator, FunctionCallTrie, and Array
Summary:
This change makes the allocator and function call trie implementations
move-aware and remove the FunctionCallTrie's reliance on a
heap-allocated set of allocators.

The change makes it possible to always have storage associated with
Allocator instances, not necessarily having heap-allocated memory
obtainable from these allocator instances. We also use thread-local
uninitialised storage.

We've also re-worked the segmented array implementation to have more
precondition and post-condition checks when built in debug mode. This
enables us to better implement some of the operations with surrounding
documentation as well. The `trim` algorithm now has more documentation
on the implementation, reducing the requirement to handle special
conditions, and being more rigorous on the computations involved.

In this change we also introduce an initialisation guard, through which
we prevent an initialisation operation from racing with a cleanup
operation.

We also ensure that the ThreadTries array is not destroyed while copies
into the elements are still being performed by other threads submitting
profiles.

Note that this change still has an issue with accessing thread-local
storage from signal handlers that are instrumented with XRay. We also
learn that with the testing of this patch, that there will be cases
where calls to mmap(...) (through internal_mmap(...)) might be called in
signal handlers, but are not async-signal-safe. Subsequent patches will
address this, by re-using the `BufferQueue` type used in the FDR mode
implementation for pre-allocated memory segments per active, tracing
thread.

We still want to land this change despite the known issues, with fixes
forthcoming.

Reviewers: mboerger, jfb

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54989

llvm-svn: 348335
2018-12-05 06:44:34 +00:00
Kamil Rytarowski b9601a385a Update GET_LINK_MAP_BY_DLOPEN_HANDLE() for NetBSD x86
NetBSD 8.99.26 changed the layout of internal structure
returned by dlopen(3), switch to it.

Set new values for amd64 and i386 based on the results
of &((struct Struct_Obj_Entry*)0)->linkmap.

llvm-svn: 348329
2018-12-05 03:17:21 +00:00
Vitaly Buka 10db3f4779 [asan] Remove use_odr_indicator runtime flag
Summary:
Flag was added for testing 3 years ago. Probably it's time
to simplify code and usage by removing it.

Reviewers: eugenis, m.ostapenko

Subscribers: mehdi_amini, kubamracek, steven_wu, dexonsmith, llvm-commits

Differential Revision: https://reviews.llvm.org/D55254

llvm-svn: 348315
2018-12-04 23:17:32 +00:00
David Carlier eb5bfac7ce Unbreak build due to style.
llvm-svn: 348295
2018-12-04 19:17:26 +00:00
David Carlier 2330a24ca5 [Sanitizer] intercept part of sysctl Api
- Distringuish what FreeBSD/NetBSD can and NetBSD specifics.
- Fixing page size value collection.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D55265	

llvm-svn: 348293
2018-12-04 19:00:38 +00:00
Dan Liew f73b782105 [SanitizerCommon] Test `CombinedAllocator::ForEachChunk()` in unit tests.
Summary:

Previously we weren't testing this function in the unit tests.

Reviewers: kcc, cryptoad, dvyukov, eugenis, kubamracek

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54861

llvm-svn: 348260
2018-12-04 14:03:55 +00:00
Petr Hosek 5c43abcf2d [compiler-rt] Use the new zx_futex_wait for Fuchsia sanitizer runtime
This finishes the soft-transition to the new primitive that implements
priority inheritance.

Differential Revision: https://reviews.llvm.org/D55244

llvm-svn: 348236
2018-12-04 04:07:43 +00:00
Kamil Rytarowski 9d62f4db78 Improve the regerror(3) interceptor
The res returned value might differ with REAL(strlen)(errbuf) + 1,
as the buffer's value is limited with errbuf_size.

Hot fix for D54584.

llvm-svn: 348231
2018-12-04 02:18:18 +00:00
Kamil Rytarowski 5b8d585925 Add interceptors for the sysctl(3) API family from NetBSD
Summary:
Add new interceptors for:

 - sysctl
 - sysctlbyname
 - sysctlgetmibinfo
 - sysctlnametomib
 - asysctl
 - asysctlbyname

Cover the API with a new test file TestCases/NetBSD/sysctl.cc.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: devnexen, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54076

llvm-svn: 348228
2018-12-04 01:51:06 +00:00
Kamil Rytarowski 175d2b8620 Add interceptors for the fts(3) API family from NetBSD
Summary:
fts(3) is API to traverse a file hierarchy.
Cover this interface with interceptors.

Add a test to validate the interface reading
the number of regular files in /etc.

Based on original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54247

llvm-svn: 348227
2018-12-04 01:45:52 +00:00
Kamil Rytarowski 145ac7940e Add new interceptor for regex(3) in NetBSD
Summary:
Add interceptors for the NetBSD style of regex(3) present inside libc:

 - regcomp
 - regexec
 - regerror
 - regfree
 - regnsub
 - regasub

Add a dedicated test verifying the installed interceptors.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54584

llvm-svn: 348224
2018-12-04 01:41:42 +00:00
Eugene Leviant a50c05872e [SanitizerCommon] Remove RenameFile
This function seems to be no longer used by compiler-rt
libraries

Differential revision: https://reviews.llvm.org/D55123

llvm-svn: 348140
2018-12-03 14:03:51 +00:00
Dan Liew 8bffb63497 Introduce a way to allow the ASan dylib on Darwin platforms to be loaded via `dlopen()`.
Summary:

The purpose of this option is provide a way for the ASan dylib
to be loaded via `dlopen()` without triggering most initialization
steps (e.g. shadow memory set up) that normally occur when the
ASan dylib is loaded.

This new functionality is exposed by

- A `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` macro which indicates if the
  feature is supported. This only true for Darwin currently.
- A `HandleDlopenInit()` function which should return true if the library
  is being loaded via `dlopen()` and
  `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` is supported. Platforms that
  support this may perform any initialization they wish inside this
  function.

Although disabling initialization is something that could potentially
apply to other sanitizers it appears to be unnecessary for other
sanitizers so this patch only makes the change for ASan.

rdar://problem/45284065

Reviewers: kubamracek, george.karpenkov, kcc, eugenis, krytarowski

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54469

llvm-svn: 348078
2018-12-01 15:45:42 +00:00
Kamil Rytarowski f130d111b6 Add a new interceptor for getvfsstat(2) from NetBSD
Summary:
getvfsstat - gets list of all mounted file systems.

Add a dedicated test.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55014

llvm-svn: 348027
2018-11-30 19:43:53 +00:00
Reid Kleckner 63f084bd7a Revert r346560 "[winasan] Unpoison the stack in NtTerminateThread"
This reverts r343606 again. The NtTerminateThread interceptor is causing
problems in NaCl:
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/CrWinAsan/1839

I reproduced the problem locally and tried my best to debug them, but
it's beyond me.

llvm-svn: 347933
2018-11-29 23:57:17 +00:00
Julian Lettner 647782c78b (no commit message)
llvm-svn: 347788
2018-11-28 18:17:23 +00:00
Kuba Mracek 265cd31fbf Fix filtering of sanitizer_common unittest architectures on Darwin.
llvm-svn: 347622
2018-11-27 01:37:08 +00:00
Benjamin Kramer a2ada4d1ce [X86][compiler-rt] Add missing semicolon
llvm-svn: 347519
2018-11-24 20:57:03 +00:00
Craig Topper 428caa398c [X86] Make conversion of feature bits into a mask explicitly unsigned by using 1U instead of 1.
llvm-svn: 347517
2018-11-24 20:25:45 +00:00
Craig Topper 62e5924dd6 [X86][compiler-rt] Attempt to fix a warning about a shift amount being negative in a macro expansion.
llvm-svn: 347516
2018-11-24 20:14:03 +00:00
David Carlier 0a01f5d244 Unbreak FreeBSD build.
M    lib/sanitizer_common/sanitizer_platform_limits_posix.cc

llvm-svn: 347451
2018-11-22 10:31:01 +00:00
Petr Hosek e7dec7848b [XRay] Support for Fuchsia
This extends XRay to support Fuchsia.

Differential Revision: https://reviews.llvm.org/D52162

llvm-svn: 347443
2018-11-22 02:00:44 +00:00
Peter Collingbourne 56f3bb4b36 tsan: Update measurements in check_analyze.sh.
These changed as a result of r347379. Unfortunately there was a
regression; filed PR39748 to track it.

Differential Revision: https://reviews.llvm.org/D54821

llvm-svn: 347442
2018-11-22 00:47:37 +00:00
Peter Collingbourne c56109306b tsan: Correct the name of an executable.
llvm-svn: 347439
2018-11-22 00:00:26 +00:00
David Carlier 0c81a62d9d [Sanitizer] Adding setvbuf in supported platforms and other stream buffer functions
- Enabling setvbuf interceptions for non NetBSD platforms.
- setbuf, setbuffer, setlinebuf as well.

Reviewers: vitalybuka, krytarowski	

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D54779

llvm-svn: 347426
2018-11-21 21:17:46 +00:00
Roman Lebedev 1d0c7f563c [compiler-rt][UBSan] silence_unsigned_overflow: do *NOT* ignore *fatal* unsigned overflows
Summary:
D48660 / rL335762 added a `silence_unsigned_overflow` env flag for [[ https://github.com/google/oss-fuzz/pull/1717 | oss-fuzz needs ]],
that allows to silence the reports from unsigned overflows.
It makes sense, it is there because `-fsanitize=integer` sanitizer is not enabled on oss-fuzz,
so this allows to still use it as an interestingness signal, without getting the actual reports.

However there is a slight problem here.
All types of unsigned overflows are ignored.
Even if `-fno-sanitize-recover=unsigned` was used (which means the program will die after the report)
there will still be no report, the program will just silently die.

At the moment there are just two projects on oss-fuzz that care:
* [[ 8eeffa627f/projects/llvm_libcxx/build.sh (L18-L20) | libc++ ]]
* [[ 8eeffa627f/projects/librawspeed/build.sh | RawSpeed ]] (me)

I suppose this could be overridden there ^, but i really don't think this is intended behavior in any case..

Reviewers: kcc, Dor1s, #sanitizers, filcab, vsk, kubamracek

Reviewed By: Dor1s

Subscribers: dberris, mclow.lists, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54771

llvm-svn: 347415
2018-11-21 20:35:43 +00:00
Dmitry Vyukov d0fb5d8b00 tsan: add pthread_tryjoin_np and pthread_timedjoin_np interceptors
Add pthread_tryjoin_np() and pthread_timedjoin_np() interceptors on Linux,
so that ThreadSanitizer can handle programs using these functions.

Author: Yuri Per (yuri)
Reviewed in: https://reviews.llvm.org/D54521

llvm-svn: 347383
2018-11-21 09:31:21 +00:00
David Carlier d931c135f0 Revert "[Sanitizer] intercept setvbuf on other platforms where it is supported"
llvm-svn: 347358
2018-11-20 22:50:31 +00:00
David Carlier 5ffb47b3a1 [Sanitizer] intercept setvbuf on other platforms where it is supported
Unit tests enabled only in platform tested.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski, vitalybuka

Differential Revision: https://reviews.llvm.org/D54739

llvm-svn: 347355
2018-11-20 22:17:23 +00:00
Vitaly Buka afedb893a6 [tsan] Add __cxa_guard_acquire hooks to support cooperative scheduling
Reviewers: dvyukov

Subscribers: krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D54664

llvm-svn: 347336
2018-11-20 18:21:42 +00:00
Dean Michael Berris 388af45f18 [XRay] Add a test for allocator exhaustion
Use a more representative test of allocating small chunks for
oddly-sized (small) objects from an allocator that has a page's worth of
memory.

llvm-svn: 347286
2018-11-20 03:56:04 +00:00
Dean Michael Berris ba02cb58cf [XRay] Move buffer extents back to the heap
Summary:
This change addresses an issue which shows up with the synchronised race
between threads writing into a buffer, and another thread reading the
buffer.

In a lot of cases, we cannot guarantee that threads will always see the
signal to finalise their buffers in time despite the grace periods and
state machine maintained through atomic variables. This change addresses
it by ensuring that the same instance being updated to indicate how much
of the buffer is "used" by the writing thread is the same instance being
read by the thread processing the buffer to be written out to disk or
handled through the iterators.

To do this, we ensure that all the "extents" instances live in their own
the backing store, in a different contiguous page from the
buffer-specific backing store. We also take precautions to ensure that
the atomic variables are cache-line-sized to prevent false-sharing from
unnecessarily causing cache contention on unrelated writes/reads.

It's feasible that we may in the future be able to move the storage of
the extents objects into the single backing store, slightly changing the
way to compute the size(s) of the buffers, but in the meantime we'll
settle for the isolation afforded by having a different backing store
for the extents instances.

Reviewers: mboerger

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54684

llvm-svn: 347280
2018-11-20 01:00:26 +00:00
Petr Hosek 8e0e35a3f5 [compiler-rt] Use zx_futex_wait_deprecated for Fuchsia sanitizer runtime
This change is part of the soft-transition to the new synchronization
primitives which implement priority inheritance.

Differential Revision: https://reviews.llvm.org/D54727

llvm-svn: 347279
2018-11-20 00:55:20 +00:00
Kamil Rytarowski 74abaf8cdc Add interceptor for the setvbuf(3) from NetBSD
Summary:
setvbuf(3) is a routine to setup stream buffering.

Enable the interceptor for NetBSD.

Add dedicated tests for setvbuf(3) and functions
on top of this interface: setbuf, setbuffer, setlinebuf.

Based on original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: devnexen, tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54548

llvm-svn: 347270
2018-11-19 22:44:26 +00:00
Kostya Serebryany 0e799d92fd [hwasan] don't check tail magic when in right_align mode (should fix the bot)
llvm-svn: 347121
2018-11-17 01:02:09 +00:00
Kostya Serebryany c265e7673d [hwasan] implement free_checks_tail_magic=1
Summary:
With free_checks_tail_magic=1 (default) HWASAN
writes magic bytes to the tail of every heap allocation
(last bytes of the last granule, if the last granule is not fully used)
and checks these bytes on free().

This feature will detect buffer overwires within the last granule
at the time of free().

This is an alternative to malloc_align_right=[1289] that should have
fewer compatibility issues. It is also weaker since it doesn't
detect read overflows and reports bugs at free() instead of at access.

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54656

llvm-svn: 347116
2018-11-17 00:25:17 +00:00
Kamil Rytarowski 003b720873 Revert "Cast the 2nd argument of _Unwind_SetIP() to _Unwind_Ptr"
_Unwind_Ptr is unknown on some targets.

Detected on green-dragon-21 (MacPro Late 2013 | OS X 10.14(18A391) | Xcode 10.1(10B61)).

llvm-svn: 347106
2018-11-16 23:00:28 +00:00
Kamil Rytarowski 54eaafafbf Cast the 2nd argument of _Unwind_SetIP() to _Unwind_Ptr
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.

No functional change intended for others.

Cherry-picked chunk from D33878.

llvm-svn: 347102
2018-11-16 22:17:36 +00:00
Kamil Rytarowski 7461cc12fc Cast _Unwind_GetIP() and _Unwind_GetRegionStart() to uintptr_t
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.

No functional change intended for others.

Cherry-picked chunk from D33878.

llvm-svn: 347101
2018-11-16 22:15:09 +00:00
Vitaly Buka c505ceeb3c [sanitizer] Update global_symbols.txt
llvm-svn: 347099
2018-11-16 21:42:22 +00:00
Kamil Rytarowski 56ef906532 Add new interceptor for mi_vector_hash(3)
Summary:
mi_vector_hash(3) provides fast 32bit hash functions.

Add a test for this interface.

Enable the API for NetBSD.

Based on original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54530

llvm-svn: 347088
2018-11-16 19:54:13 +00:00
Kamil Rytarowski 47fb42e6c3 Add new interceptor for getmntinfo(3) from NetBSD
Summary:
getmntinfo gets information about mounted file systems.

Add a dedicated test for new interceptor.

Based on original work by Yang Zheng.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54611

llvm-svn: 347083
2018-11-16 19:47:42 +00:00
Kostya Serebryany ba5c7169c5 [hwasan] optionally right-align heap allocations
Summary:
... so that we can find intra-granule buffer overflows.
The default is still to always align left.
It remains to be seen wether we can enable this mode at scale.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: jfb, dvyukov, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D53789

llvm-svn: 347082
2018-11-16 19:38:48 +00:00
Dan Liew ee6eb9b25a Start adding the supporting code to perform out-of-process allocator
enumeration.

Summary:

This patch introduces the local portion (`LocalAddressSpaceView`) of the
`AddressSpaceView` abstraction and modifies the secondary allocator
so that the `ForEachChunk()` method (and its callees) would work in the
out-of-process case when `AddressSpaceView` is `RemoteAddressSpaceView`.

The `AddressSpaceView` abstraction simply maps pointers from a target
process to a pointer in the local process (via its `Load()` method). For
the local (in-process) case this is a no-op. For the remote
(out-of-process) case this is not a no-op. The implementation of the
out-of-process `RemoteAddressSpaceView` is not included in this patch
and will be introduced later.

This patch is considerably simpler than the `ObjectView` abstraction
used in previous patches but lacks the type safety and stricter memory
management of the `ObjectView` abstraction.

This patch does not introduce any tests because with
`LocalAddressSpaceView` it should be a non functional change and unit
tests already cover the secondary allocator.  When
`RemoteAddressSpaceView` is landed tests will be added to ensure that it
functions as expected.

rdar://problem/45284065

Reviewers: kcc, kubamracek, dvyukov, vitalybuka, cryptoad,
george.karpenkov, morehouse

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D53975

llvm-svn: 346956
2018-11-15 14:20:28 +00:00
George Karpenkov 6fd29477c3 [lsan] [FIXUP] Fixup for http://reviews.llvm.org/D54484
After the change, the tests started failing, as skipped sections can be
equal in size to kMaxSegName.
Changing `<` to `<=` to address the off-by-one problem.

llvm-svn: 346804
2018-11-13 22:17:16 +00:00
George Karpenkov 2e8839de7c [lsan] [NFC] Change ARRAY_SIZE to internal_strnlen
Calling ARRAY_SIZE on a char* will not actually compute it's size, but just the pointer size.
A new Clang warning enabled by default warns about this.

Replaced the call with internal_strnlen.

Differential Revision: https://reviews.llvm.org/D54484

llvm-svn: 346792
2018-11-13 20:19:38 +00:00
Kamil Rytarowski 4a282024db Correct atexit(3) support in MSan/NetBSD
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.

Changes:

 - Split paths of handling intercepted __cxa_atexit() and atexit(3).
   This affects all supported Operating Systems.
 - Add a local stack-like structure to hold the __cxa_atexit() context.
   atexit(3) is documented in the C standard as calling callback from the
   earliest to the oldest entry. This path also fixes potential ABI
   problem of passing an argument to a function from the atexit(3)
   callback mechanism.
 - Allow usage of global vars with ctors in interceptors.
   This allows to use Vector without automatic cleaning up the structures.

This code has been modeled after TSan implementation for the same functions.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc

Reviewed By: vitalybuka

Subscribers: delcypher, devnexen, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40714

llvm-svn: 346579
2018-11-10 02:17:32 +00:00
Reid Kleckner 6c0bb3758e Re-land r343606 "[winasan] Unpoison the stack in NtTerminateThread"
This change was reverted because it caused some nacl tests in chromium
to fail. I attempted to reproduce those problems locally, but I was
unable to. Let's reland this and let Chromium's test infrastructure
discover any problems.

llvm-svn: 346560
2018-11-09 22:06:59 +00:00
Evgeniy Stepanov 771d6d37ca [hwasan] Add entire report to abort message on Android.
Summary:
When reporting a fatal error, collect and add the entire report text to
android_set_abort_message so that it can be found in the tombstone.

Reviewers: kcc, vitalybuka

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D54284

llvm-svn: 346557
2018-11-09 21:54:03 +00:00
Dean Michael Berris 9e089fca7e [XRay] Add a test for function id encoding/decoding (NFC)
Increase test coverage for function enter/exit encoding/decoding.

llvm-svn: 346477
2018-11-09 07:43:30 +00:00
Dean Michael Berris ead47aab78 [XRay] Add a static assertion on size of metadata payload (NFC)
This change adds a static check to ensure that all data metadata record
payloads don't go past the available buffers in Metadata records.

llvm-svn: 346476
2018-11-09 07:16:45 +00:00
Dean Michael Berris e39a89fbfb [XRay] Fix enter function tracing for record unwriting
Summary:
Before this change, we could run into a situation where we may try to
undo tail exit records after writing metadata records before a function
enter event. This change rectifies that by resetting the tail exit
counter after writing the metadata records.

Reviewers: mboerger

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D54292

llvm-svn: 346475
2018-11-09 06:49:00 +00:00
Dean Michael Berris aeae545064 [XRay] Add atomic fences around non-atomic reads and writes
Summary:
We need these fences to ensure that other threads attempting to read
bytes in the buffer will see thw writes committed before the extents are
updated. Without these, the writes can be un-committed by the time the
buffer extents counter is updated -- the fences should ensure that the
records written into the log have completed by the time we observe the
buffer extents from different threads.

Reviewers: mboerger

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54291

llvm-svn: 346474
2018-11-09 06:39:45 +00:00
Dean Michael Berris da375a67f8 [XRay] Improve FDR trace handling and error messaging
Summary:
This change covers a number of things spanning LLVM and compiler-rt,
which are related in a non-trivial way.

In LLVM, we have a library that handles the FDR mode even log loading,
which uses C++'s runtime polymorphism feature to better faithfully
represent the events that are written down by the FDR mode runtime. We
do this by interpreting a trace that's serliased in a common format
agreed upon by both the trace loading library and the FDR mode runtime.
This library is under active development, which consists of features
allowing us to reconstitute a higher-level event log.

This event log is used by the conversion and visualisation tools we have
for interpreting XRay traces.

One of the tools we have is a diagnostic tool in llvm-xray called
`fdr-dump` which we've been using to debug our expectations of what the
FDR runtime should be writing and what the logical FDR event log
structures are. We use this fairly extensively to reason about why some
non-trivial traces we're generating with FDR mode runtimes fail to
convert or fail to parse correctly.

One of these failures we've found in manual debugging of some of the
traces we've seen involve an inconsistency between the buffer extents (a
record indicating how many bytes to follow are part of a logical
thread's event log) and the record of the bytes written into the log --
sometimes it turns out the data could be garbage, due to buffers being
recycled, but sometimes we're seeing the buffer extent indicating a log
is "shorter" than the actual records associated with the buffer. This
case happens particularly with function entry records with a call
argument.

This change for now updates the FDR mode runtime to write the bytes for
the function call and arg record before updating the buffer extents
atomically, allowing multiple threads to see a consistent view of the
data in the buffer using the atomic counter associated with a buffer.
What we're trying to prevent here is partial updates where we see the
intermediary updates to the buffer extents (function record size then
call argument record size) becoming observable from another thread, for
instance, one doing the serialization/flushing.

To do both diagnose this issue properly, we need to be able to honour
the extents being set in the `BufferExtents` records marking the
beginning of the logical buffers when reading an FDR trace. Since LLVM
doesn't use C++'s RTTI mechanism, we instead follow the advice in the
documentation for LLVM Style RTTI
(https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html). We then rely on
this RTTI feature to ensure that our file-based record producer (our
streaming "deserializer") can honour the extents of individual buffers
as we interpret traces.

This also sets us up to be able to eventually do smart
skipping/continuation of FDR logs, seeking instead to find BufferExtents
records in cases where we find potentially recoverable errors. In the
meantime, we make this change to operate in a strict mode when reading
logical buffers with extent records.

Reviewers: mboerger

Subscribers: hiraditya, llvm-commits, jfb

Differential Revision: https://reviews.llvm.org/D54201

llvm-svn: 346473
2018-11-09 06:26:48 +00:00
Evgeniy Stepanov 4680386c34 [hwasan] Remove dead code.
llvm-svn: 346452
2018-11-09 00:03:39 +00:00
Dan Liew 7383b4fba4 Introduce `sanitizer_malloc_introspect_t` for Darwin which is a sub-class of Darwin's `malloc_introspection_t` and use it when setting up the malloc zone.
Summary:
Currently `sanitizer_malloc_introspection_t` just adds a version field
which is used to version the allocator ABI. The current allocator ABI
version is returned by the new `GetMallocZoneAllocatorEnumerationVersion()` function.

The motivation behind this change is to allow external processes to
determine the allocator ABI of a sanitized process.

rdar://problem/45284065

Reviewers: kubamracek, george.karpenkov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54045

llvm-svn: 346420
2018-11-08 17:50:34 +00:00
Kamil Rytarowski e06fb482a9 Split lgammal() from INIT_LGAMMAL
Summary:
Introduce SANITIZER_INTERCEPT_LGAMMAL
dedicated for lgammal(). Disable it for NetBSD
as this routine is not implemented in this OS.

Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54054

llvm-svn: 346352
2018-11-07 20:19:53 +00:00
Kamil Rytarowski c4dfb42897 Split remquol() from INIT_REMQUO
Summary:
Introduce SANITIZER_INTERCEPT_REMQUOL
dedicated for remquol(). Disable for for NetBSD
as this routine is not implemented in this OS.

Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54051

llvm-svn: 346351
2018-11-07 20:18:36 +00:00
Petr Hosek 5dc0fba2e3 [CMake][ASan][HWASan] Set an explicit dependency on libc++ headers
We have seen failing builds due to a race condition between
RTAsan_dynamic and libc++ headers builds, specifically libc++
headers depend on __config and if this header hasn't been copied
into the final location, including other headers will typically
result in failure. To avoid this race, we add an explicit dependency
on libc++ headers which ensures that they've been copied into place
before the sanitizer object library build starts.

Differential Revision: https://reviews.llvm.org/D54198

llvm-svn: 346339
2018-11-07 18:35:44 +00:00
Marco Castelluccio ed4bb266c8 [GCOV] Close file mapping handle on Windows, so flushed gcda files can be removed while the process is in execution
llvm-svn: 346300
2018-11-07 09:38:26 +00:00
Dean Michael Berris 59439dd069 [XRay] Use TSC delta encoding for custom/typed events
Summary:
This change updates the version number for FDR logs to 5, and update the
trace processing to support changes in the custom event records.

In the runtime, since we're already writing down the record preamble to
handle CPU migrations and TSC wraparound, we can use the same TSC delta
encoding in the custom event and typed event records that we use in
function event records. We do the same change to typed events (which
were unsupported before this change in the trace processing) which now
show up in the trace.

Future changes should increase our testing coverage to make custom and
typed events as first class entities in the FDR mode log processing
tools.

This change is also a good example of how we end up supporting new
record types in the FDR mode implementation. This shows the places where
new record types are added and supported.

Depends on D54139.

Reviewers: mboerger

Subscribers: hiraditya, arphaman, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54140

llvm-svn: 346293
2018-11-07 04:37:42 +00:00
Evgeniy Stepanov d657f109d7 [sanitizer] Only set soft coredump limit.
Summary: If user wants to raise it back, let them.

Reviewers: kcc, vitalybuka

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D54190

llvm-svn: 346284
2018-11-07 01:03:26 +00:00
Jonathan Metzman 39b6ba9f33 [fuzzer] Read files as binary
Summary: Read corpus files as binary to avoid automatic conversions

Reviewers: Dor1s, morehouse

Reviewed By: Dor1s, morehouse

Differential Revision: https://reviews.llvm.org/D54180

llvm-svn: 346279
2018-11-06 23:25:25 +00:00
Vitaly Buka 34e6f0b872 [sanitizer] Return headed to fix build after r346258
llvm-svn: 346270
2018-11-06 21:49:53 +00:00
Benjamin Kramer b23fded901 [dfsan] Fix build after r346262
compiler-rt/lib/dfsan/dfsan.cc:426:3: error: call to 'InitializePlatformEarly' is ambiguous
  InitializePlatformEarly();
  ^~~~~~~~~~~~~~~~~~~~~~~
compiler-rt/lib/dfsan/../sanitizer_common/sanitizer_common.h:901:6: note: candidate function
void InitializePlatformEarly();
     ^
compiler-rt/lib/dfsan/dfsan.cc:391:13: note: candidate function
static void InitializePlatformEarly() {
            ^

llvm-svn: 346264
2018-11-06 20:42:19 +00:00
Kuba Mracek c4b8eb53c4 [sanitizer] Use "fast mmap" kernel flag for shadow memory on macOS 10.13.4+
This speeds up process startup and teardown and also reduces lock contention when running multiple ASanified/TSanified processes simultaneously. Should greatly improve lit testing time.

Differential Revision: https://reviews.llvm.org/D48445

llvm-svn: 346262
2018-11-06 19:55:19 +00:00
Vitaly Buka bcee83da3e [sanitizer] Fix nolibc unittests broken by r346215
Subscribers: kubamracek, krytarowski, fedor.sergeev, llvm-commits

Differential Revision: https://reviews.llvm.org/D54163

llvm-svn: 346258
2018-11-06 19:23:22 +00:00
Vitaly Buka ecb2eb46bc [sanitizer] Add char **GetEnviron() on all other platforms
Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D54165

llvm-svn: 346257
2018-11-06 19:23:12 +00:00
Kamil Rytarowski cc297272c7 Prioritize the constructor call of __local_xray_dyninit()
Summary:
For platforms without preinit support (such as NetBSD/amd64) the
initialization routine __xray_init() was called in non-deterministic order
compared to other constructors. This caused breakage failures
as xray routines attempted to execute code with assumption of
being initialized, which was no always true.

Use GCC/Clang extension to set maximal priority to the constructor
calling __xray_init(). This code switches away from C++ lambda form,
as it did not allow to specify this compiler extension.

Reviewers: dberris, joerg

Reviewed By: dberris

Subscribers: llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54136

llvm-svn: 346222
2018-11-06 10:59:21 +00:00
Benjamin Kramer 178d26fa18 [sanitizer] Use AT_EXECFN in ReExec() if available
execve("/proc/self/exe") will not work if the binary relies on
$EXEC_ORIGIN in an rpath. Query AT_EXECFN instead, which will give the
same string that the current binary was exec'd with.

Differential Revision: https://reviews.llvm.org/D54113

llvm-svn: 346215
2018-11-06 08:53:38 +00:00
Dean Michael Berris 25f8d204b8 [XRay] Update XRayRecord to support Custom/Typed Events
Summary:
This change cuts across LLVM and compiler-rt to add support for
rendering custom events in the XRayRecord type, to allow for including
user-provided annotations in the output YAML (as raw bytes).

This work enables us to add custom event and typed event records into
the `llvm::xray::Trace` type for user-provided events. This can then be
programmatically handled through the C++ API and can be included in some
of the tooling as well. For now we support printing the raw data we
encounter in the custom events in the converted output.

Future work will allow us to start interpreting these custom and typed
events through a yet-to-be-defined API for extending the trace analysis
library.

Reviewers: mboerger

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D54139

llvm-svn: 346214
2018-11-06 08:51:37 +00:00
Kamil Rytarowski 2e61186c42 Follow Windows' approach for NetBSD in AlarmCallback()
NetBSD's current behavior needs to ignore check for
InFuzzingThread() in Fuzzer::AlarmCallback().

llvm-svn: 346194
2018-11-06 01:28:01 +00:00
Rainer Orth 682ee6c3bc [Sanitizers] Disable SANITIZER_CAN_FAST_UNWIND on all SPARC targets
While testing my to-be-submitted Solaris sanitizer support on gcc mainline, I ran into
an issue on Solaris/SPARC (sparc-sun-solaris2.11). Initially libasan.so failed to link:

  Undefined                       first referenced
   symbol                             in file
  __sanitizer::BufferedStackTrace::FastUnwindStack(unsigned long, unsigned long, unsigned long, unsigned long, unsigned int) /var/gcc/gcc-9.0.0-20181024/11.5-gcc-gas/sparc-sun-solaris2.11/./libsanitizer/asan/.libs/libasan.so

This happens because SANITIZER_CAN_FAST_UNWIND is enabled on non-Linux
SPARC targets (cf. sanitizer_stacktrace.h), but the guard around the SPARCv8-only
definition in sanitizer_stacktrace_sparc.cc only works with clang:

clang predefines __sparcv8__ on non-Solaris, and __sparcv8 only on Solaris
gcc predefines __sparcv8 on Solaris, but __sparc_v8__ on non-Solaris

The attached patch allows for all three variants.

However, disabling SANITIZER_CAN_FAST_UNWIND on all SPARC targets
fixes a couple of testsuite failures in the Solaris asan testsuite, so for now it's better
to keep it disabled everywhere.

This allowed the libsanitizer build to complete and gave reasonable (though slightly
worse than on Solaris/x86) testsuite results.

Differential Revision: https://reviews.llvm.org/D54099

llvm-svn: 346155
2018-11-05 19:22:54 +00:00
Rainer Orth 1a9b072338 [Sanitizers] Solaris largefile fixes
While testing the Solaris libsanitizer port on GCC mainline, I found that
I'd messed up
the largefile checks in various ways, some of which showed as compile failures
(wrong structure sizes and member offsets), others at runtime, some of those only
on sparc as a big-endian target.

This patch fixes all of them:

- OFF_T is now correctly defined for 32-bit largefile and traditional
  environments, and 64-bit.

- The definition of __sanitizer_dirent now checks the correct conditionals.

- sanitizer_procmaps_solaris.cc undefines _FILE_OFFSET_BITS: before
  Solaris 11.4 <procfs.h> doesn't even compile with largefile support
  enabled, but the use at hand doesn't need it anyway while g++ 9 will
  define _FILE_OFFSET_BITS=64 out of the box.

- With full largefile support enabled, one needs to use e.g. mmap64
  instead of mmap; this is hidden behind macros.

With this patch I could bootstrap gcc mainline on both sparc-sun-solaris2.11 and
i386-pc-solaris2.11.  In addition, I've successfully built llvm on
i386-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D54101

llvm-svn: 346153
2018-11-05 19:19:15 +00:00
Martin Liska 5f60861be7 Fix build on sparc64-linux-gnu.
Differential Revision: https://reviews.llvm.org/D54030

llvm-svn: 346129
2018-11-05 12:27:43 +00:00
Dean Michael Berris 1e255e7a7b [XRay] Update TSC math to handle wraparound
Summary:
Prior to this change, we can run into situations where the TSC we're
getting when exiting a function is less than the TSC we got when
entering it. This would sometimes cause the counter for cumulative call
times overflow, which was erroneously also being stored as a signed
64-bit integer.

This change addresses both these issues while adding provisions for
tracking CPU migrations. We do this because moving from one CPU to
another doesn't guarantee that the timestamp counter for some
architectures aren't guaranteed to be synchronised. For the moment, we
leave the provisions there until we can update the data format to
include the counting of CPU migrations we can catch.

We update the necessary tests as well, ensuring that our expectations
for the cycle accounting to be met in case of counter wraparound.

Reviewers: mboerger

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D54088

llvm-svn: 346116
2018-11-05 05:43:22 +00:00
Kamil Rytarowski 538d678737 Disable recursive interception for tzset in MSan
This fixes sanitization of this library call on NetBSD.
Extracted as a part of D42050.

llvm-svn: 346098
2018-11-04 15:30:38 +00:00
Petr Hosek 466f0f028a [compiler-rt][Fuzzer] Fix the fuzzer test build
This fixes the issue introduced in r345765 which changed the way in
which the embedded libc++ is being built but omitted tests.

Differential Revision: https://reviews.llvm.org/D54058

llvm-svn: 346052
2018-11-02 23:29:13 +00:00
Kamil Rytarowski 3095874d3c Try to unbreak the build of sanitizers on !NetBSD
Include the build of unpoison_passwd() and unpoison_group() for
SANITIZER_INTERCEPT_FGETPWENT_R and SANITIZER_INTERCEPT_FGETGRENT_R.
 static void unpoison_passwd(

llvm-svn: 346042
2018-11-02 21:06:05 +00:00
Kamil Rytarowski 8deda5fbd8 Split getpwent and fgetgrent functions in interceptors
Summary:
NetBSD does not ship with fgetpwent_r() and fgetgrent_r().
Split their interceptors from getpwent_r() and getgrent_r()
and disable for this OS.

Installation of supernumerary interceptors causes leaking of
errors to dlsym(3)-like operations.

No functional change for other OSes.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, fedor.sergeev, llvm-commits, #sanitizers, mgorny

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54041

llvm-svn: 346038
2018-11-02 20:28:10 +00:00
Lei Huang 78f7a6eaa6 [PowerPC]Disable randomized address space on Linux ppc64le
Recent versions of Ubuntu (17.04 and 18.04) on PowerPC have introduced changes
to Address Space Layout Randomization (ASLR) that is causing 500+ sanitizer
failures. This patch disables ASLR when running the sanitizers on PowerPC 64bit
LE.

Differential Revision: https://reviews.llvm.org/D52900

llvm-svn: 346030
2018-11-02 19:00:23 +00:00
Dan Liew a8eaaa8006 [NFC] Refactor initialisation of Darwin malloc zone fields into separate function called `InitMallocZoneFields()` which is now called from `ReplaceSystemMalloc()`.
Summary:
The motivation here is to be able support initialisation of the
malloc zone on other code paths.

rdar://problem/45284065

Reviewers: kubamracek, george.karpenkov

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D54037

llvm-svn: 345983
2018-11-02 15:28:11 +00:00
Dean Michael Berris 0c96ae3d6e [XRay] Update delta computations in runtime
Summary:
Fix some issues discovered from mostly manual inspection of outputs from
the `llvm-xray fdr-dump` tool.

It turns out we haven't been writing the deltas properly, and have been
writing down zeros for deltas of some records. This change fixes this
oversight born by the recent refactoring.

Reviewers: mboerger

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D54022

llvm-svn: 345954
2018-11-02 08:07:38 +00:00
Dean Michael Berris e8c650ab12 [XRay] Fix TSC and atomic custom/typed event accounting
Summary:
This is a follow-on change to D53858 which turns out to have had a TSC
accounting bug when writing out function exit records in FDR mode.

This change adds a number of tests to ensure that:

- We are handling the delta between the exit TSC and the last TSC we've
  seen.

- We are writing the custom event and typed event records as a single
  update to the buffer extents.

- We are able to catch boundary conditions when loading FDR logs.

We introduce a TSC matcher to the test helpers, which we use in the
testing/verification of the TSC accounting change.

Reviewers: mboerger

Subscribers: mgorny, hiraditya, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D53967

llvm-svn: 345905
2018-11-01 22:57:50 +00:00
Dean Michael Berris 6b67ff0300 [XRay] Add CPU ID in Custom Event FDR Records
Summary:
This change cuts across compiler-rt and llvm, to increment the FDR log
version number to 4, and include the CPU ID in the custom event records.

This is a step towards allowing us to change the `llvm::xray::Trace`
object to start representing both custom and typed events in the stream
of records. Follow-on changes will allow us to change the kinds of
records we're presenting in the stream of traces, to incorporate the
data in custom/typed events.

A follow-on change will handle the typed event case, where it may not
fit within the 15-byte buffer for metadata records.

This work is part of the larger effort to enable writing analysis and
processing tools using a common in-memory representation of the events
found in traces. The work will focus on porting existing tools in LLVM
to use the common representation and informing the design of a
library/framework for expressing trace event analysis as C++ programs.

Reviewers: mboerger, eizan

Subscribers: hiraditya, mgrang, llvm-commits

Differential Revision: https://reviews.llvm.org/D53920

llvm-svn: 345798
2018-11-01 00:18:52 +00:00
Reid Kleckner dffe5a3807 Compile and test i128 math builtins for Win64
Summary:
Windows has always been LLP64, not LP64, so the macros were incorrect.
Check for _WIN64, since AArch64 sets that too.

The tests have to be fixed up in two main ways:
1. Use 'ULL' suffixes to avoid sign extension when passing hex literals
   with the sign bit set to signed 128 arguments. Clang
   -fms-compatibility makes such hex literals signed, not unsigned.
2. Disable various tests for 80-bit long double interoperation with i128
   values.

Reviewers: joerg, mstorsjo

Subscribers: javed.absar, kristof.beyls, hiraditya, aheejin, llvm-commits

Differential Revision: https://reviews.llvm.org/D53918

llvm-svn: 345796
2018-11-01 00:00:03 +00:00
Petr Hosek 44eba12b74 [compiler-rt][Fuzzer] Use the new C++ ABI namespace CMake support
libc++ now supports customizing the ABI namespace directly from the
CMake build so we no longer need to rely on custom CFLAGS.

Differential Revision: https://reviews.llvm.org/D53939

llvm-svn: 345765
2018-10-31 19:15:48 +00:00
Kamil Rytarowski 8448460975 Update ioctl(2) operations for NetBSD 8.99.25
Eliminate dropped operations, add new operations.

Update included headers for newer need.

Add a fallback definition of nvlist_ref_t, becaue this type
is internal to libnpf and the kernel, not exported into public
headers.

llvm-svn: 345726
2018-10-31 15:04:20 +00:00
Reid Kleckner 3e27306565 [asan] Remove stale -fno-exceptions flag in sanitizer_common as well
llvm-svn: 345685
2018-10-31 00:35:46 +00:00
Reid Kleckner 1079d7ccfe [asan] Remove flags for clang-cl before it supported EH
Also remove -Wno-undefined-inline, which needed to work around PR19898,
which was fixed.

llvm-svn: 345677
2018-10-31 00:20:41 +00:00
Kamil Rytarowski 9fe6b4d522 Disable BufferOverflowAfterManyFrees for NetBSD
This test hangs in the i386 mode.

llvm-svn: 345666
2018-10-30 22:08:47 +00:00
Roman Lebedev 320e9af309 [compiler-rt][ubsan] Implicit Conversion Sanitizer - integer sign change - compiler-rt part
Summary:
This is a compiler-rt part.
The clang part is D50250.

See [[ https://bugs.llvm.org/show_bug.cgi?id=21530 | PR21530 ]], https://github.com/google/sanitizers/issues/940.

Reviewers: vsk, filcab, #sanitizers

Reviewed By: filcab, #sanitizers

Subscribers: mclow.lists, srhines, kubamracek, dberris, rjmccall, rsmith, llvm-commits, regehr

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D50251

llvm-svn: 345659
2018-10-30 21:58:54 +00:00
Reid Kleckner 96b823418d [builtins] Re-enable x86-only long double tests
Summary:
In r81552, the HAS_80_BIT_LONG_DOUBLE macro was added to the unit test
only version of int_lib.h. One month later in r85260 the duplicate
int_lib.h was removed, but the tests still passed because we don't build
with -Werror.

This is the minimal change to bring it back, and I decided to put the
configuration macro next to our 128-bit integer support macro.

Reviewers: joerg, compnerd, mstorsjo

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D53838

llvm-svn: 345645
2018-10-30 20:51:27 +00:00
Kamil Rytarowski df8802da7a Handle NetBSD alias for pthread_sigmask
Summary:
Add a new helper macro TSAN_INTERCEPTOR_NETBSD_ALIAS_THR2
that handles pthread(3)/libc aliases in scenarios when a name in both
libraries differs not just in prefix namespace.

Handle TSan pthread_sigmask mangling accordingly into __libc_thr_sigsetmask.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, jfb, llvm-commits, #sanitizers, mgorny

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D53863

llvm-svn: 345627
2018-10-30 18:49:19 +00:00
Kamil Rytarowski 98f1dd0bd5 Cast the return value of _Unwind_GetIP() to uptr
This is needed for NetBSD to match the expected type in
Unwind_GetIP().

llvm-svn: 345620
2018-10-30 18:25:12 +00:00
Kamil Rytarowski 8f545983a3 Cleanup includes in sanitizer_platform_limits_netbsd.cc
Sort the headers more correctly according to NetBSD style.
Prevent in this code part clang-format, as shuffling the order
will cause build failures.

llvm-svn: 345586
2018-10-30 11:34:36 +00:00
Kamil Rytarowski 4f343547ba Regenerate syscall hooks for NetBSD 8.99.25
Register new syscall getsockopt2.
Drop removed syscalls pmc_get_info and pmc_control.

While there address compiler warnings about potentially
unused variables.

llvm-svn: 345582
2018-10-30 11:10:36 +00:00
Kamil Rytarowski da78171643 Handle pthread_sigmask in DemangleFunctionName()
Translate __libc_thr_sigsetmask to pthread_sigmask on NetBSD.

llvm-svn: 345574
2018-10-30 08:51:34 +00:00
Kamil Rytarowski 4c5620f233 Handle NetBSD symbol mangling for tzset
Native code generated on NetBSD mangles tzet to __tzset50.

Handle this accordingly in MSan.

llvm-svn: 345571
2018-10-30 08:16:14 +00:00
Kamil Rytarowski 7b88204b39 Handle NetBSD symbol mangling for nanosleep and vfork
Summary:
Native code generated on NetBSD mangles:

 - vfork into __vfork14
 - nanosleep into __nanosleep50

Handle this accordingly in TSan.

Reviewers: vitalybuka, dvyukov, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers, mgorny

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D53806

llvm-svn: 345570
2018-10-30 08:06:42 +00:00
Dean Michael Berris 01aeb3221d [XRay] Migrate FDR runtime to use refactored controller
Summary:
This change completes the refactoring of the FDR runtime to support the
following:

- Generational buffer management.

- Centralised and well-tested controller implementation.

In this change we've had to:

- Greatly simplify the code in xray_fdr_logging.cc to only implement the
  glue code for calling into the controller.

- Implement the custom and typed event logging functions in the
  FDRLogWriter.

- Imbue the `XRAY_NEVER_INSTRUMENT` attribute onto all functions in the
  controller implementation.

Reviewers: mboerger, eizan, jfb

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D53858

llvm-svn: 345568
2018-10-30 04:35:48 +00:00
Vitaly Buka 7c180fa915 [ubsan-minimal] update comments in CMakeLists.txt
Summary:
Prepend minimal when UBsan is mentioned and delete a dead comment

Patch by Igor Sugak

Reviewers: eugenis, pcc

Reviewed By: eugenis

Subscribers: mgorny, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D53826

llvm-svn: 345557
2018-10-30 01:20:12 +00:00
Vitaly Buka 9817192691 [sanitizer] Move interceptor initialization check into real_clock_gettime
Reviewers: cryptoad, kubamracek

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D53777

llvm-svn: 345556
2018-10-30 01:20:01 +00:00
Erik Pilkington 81b89fd7bd [compiler-rt][builtins] Don't #include CoreFoundation in os_version_check.c
This breaks some configurations, so just forward declare everything that we need.

rdar://35943793

Differential revision: https://reviews.llvm.org/D50269

llvm-svn: 345551
2018-10-29 23:01:26 +00:00
Reid Kleckner 143f1a9e72 [builtins] Fix -Wunused-local-typedef warning in compile time asserts
Use the __attribute__ spellings when compiling with clang-cl so that
__attribute__((unused)) expands to something.

llvm-svn: 345550
2018-10-29 22:48:14 +00:00
Dean Michael Berris e01d25efbd [XRay] Guard call to postCurrentThreadFCT()
Summary:
Some cases where `postCurrentThreadFCT()` are not guarded by our
recursion guard. We've observed that sometimes these can lead to
deadlocks when some functions (like memcpy()) gets outlined and the
version of memcpy is XRay-instrumented, which can be materialised by the
compiler in the implementation of lower-level components used by the
profiling runtime.

This change ensures that all calls to `postCurrentThreadFCT` are guarded
by our thread-recursion guard, to prevent deadlocks.

Reviewers: mboerger, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D53805

llvm-svn: 345489
2018-10-29 05:39:43 +00:00
Dean Michael Berris 36d9746630 [XRay] Use more portable control block
Summary:
In D53560, we assumed a specific layout for memory without using an
explicit structure. This follow-up change uses more portable layout
control by using unions in a struct, and consolidating the memory
management code in the buffer queue.

We also take the opportunity to improve the documentation on the types
and operations, along with simplifying some of the logic in the buffer
queue implementation.

Reviewers: mboerger, eizan

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D53802

llvm-svn: 345485
2018-10-29 02:18:14 +00:00
Dean Michael Berris 299e9b8be0 [XRay] Refcount backing store for buffers
Summary:
This change implements the ref-counting for backing stores associated
with generational buffer management. We do this as an implementation
detail of the buffer queue, instead of exposing this to the interface.

This change allows us to keep the buffer queue interface and usage model
the same.

Depends on D53551.

Reviewers: mboerger, eizan

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D53560

llvm-svn: 345471
2018-10-28 09:12:43 +00:00
Dean Michael Berris 18024786b9 [XRay] Support generational buffers in FDR controller
Summary:
This is an intermediary step in the full support for generational buffer
management in the FDR runtime. This change makes the FDR controller
aware of the new generation number in the buffers handed out by the
BufferQueue type.

In the process of making this change, we've realised that the cleanest
way of ensuring that the backing store per generation is live while all
the threads that need access to it will need reference counting to tie
the backing store to the lifetime of all threads that have a handle on
buffers associated with the memory.

We also learn that we're missing the edge-case in the function exit
handler's implementation where the first record being written into the
buffer is a function exit, which is caught/fixed by the test for
generational buffer management.

We still haven't wired the controller into the FDR mode runtime, which
will need the reference counting on the backing store implemented to
ensure that we're being conservatively thread-safe with this approach.

Depends on D52974.

Reviewers: mboerger, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D53551

llvm-svn: 345445
2018-10-27 03:00:21 +00:00
Kuba Mracek 117360cc24 [sanitizer] Improve macOS version detection
Part of <https://reviews.llvm.org/D48445>.

llvm-svn: 345406
2018-10-26 17:19:41 +00:00
Evgeniy Stepanov ea857e8225 [sanitizer] Fix mallopt interceptor.
On error, mallopt is supposed to return 0, not -1.

llvm-svn: 345323
2018-10-25 22:15:44 +00:00
Kuba Mracek 6b6d9bbd48 [sanitizer] Avoid calling a nullptr in MonotonicNanoTime if interceptors are not yet initialized
There's a TSan startup crash on Linux when used in Swift programs, where MonotonicNanoTime will try to call real_clock_gettime and then jump to NULL because interceptors are not yet initialized. This is on Ubuntu 18.04. Looks like TSan's main Initialize() function is called at a point where __progname is already set, but interceptors aren't yet set up. Let's fix this by checking whether interceptors are initialized in MonotonicNanoTime.

Differential Revision: https://reviews.llvm.org/D53528

llvm-svn: 345174
2018-10-24 18:40:08 +00:00
Kostya Serebryany ad11526c30 [hwasan] when printing a stack-related bugs, also print stack frame descriptions provided by the compiler
llvm-svn: 345110
2018-10-24 01:35:50 +00:00
Kostya Serebryany af95597c3c [hwasan] add stack frame descriptions.
Summary:
At compile-time, create an array of {PC,HumanReadableStackFrameDescription}
for every function that has an instrumented frame, and pass this array
to the run-time at the module-init time.
Similar to how we handle pc-table in SanitizerCoverage.
The run-time is dummy, will add the actual logic in later commits.

Reviewers: morehouse, eugenis

Reviewed By: eugenis

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D53227

llvm-svn: 344985
2018-10-23 00:50:40 +00:00
Kuba Mracek 975352c0b1 [interception] Switch to _type suffix instead of _f, NFC.
Some Darwin functions have pairs like dispatch_apply and dispatch_apply_f so the added _f to interceptor types causes a clash. Let's add _type suffix instead.

Differential Revision: https://reviews.llvm.org/D53167

llvm-svn: 344954
2018-10-22 20:54:48 +00:00
Dean Michael Berris 2336dc3c51 [XRay] Account for virtual memory re-use
Change the assumption when releasing memory to a buffer queue that new
generations might not be able to re-use the memory mapped addresses.

llvm-svn: 344882
2018-10-22 06:11:50 +00:00
Dean Michael Berris 788b17ca78 [XRay][compiler-rt] Generational Buffer Management
Summary:
This change updates the buffer queue implementation to support using a
generation number to identify the lifetime of buffers. This first part
introduces the notion of the generation number, without changing the way
we handle the buffers yet.

What's missing here is the cleanup of the buffers. Ideally we'll keep
the two most recent generations. We need to ensure that before we do any
writes to the buffers, that we check the generation number(s) first.

Those changes will follow-on from this change.

Depends on D52588.

Reviewers: mboerger, eizan

Subscribers: llvm-commits, jfb

Differential Revision: https://reviews.llvm.org/D52974

llvm-svn: 344881
2018-10-22 04:53:58 +00:00
Dean Michael Berris ebfbf89000 [XRay] Handle allocator exhaustion in segmented array
Summary:
This change allows us to handle allocator exhaustion properly in the
segmented array implementation. Before this change, we relied on the
caller of the `trim` function to provide a valid number of elements to
trim. This change allows us to do the right thing in case the elements
to trim is greater than the size of the container.

Reviewers: mboerger, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D53484

llvm-svn: 344880
2018-10-22 02:11:27 +00:00
David Carlier cea5db8302 [Sanitizer] openbsd does not have sysctlbyname call
Enabling only for FreeBSD.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D53413

llvm-svn: 344876
2018-10-21 21:17:03 +00:00
Craig Topper 3fe6470951 [X86][compiler-rt] Add additional CPUs and features to the cpu detection to match libgcc
Summary: This patch adds additional features and cpus from libgcc. Unfortunately we've overflowed the existing 32-bits of features so we had to add a new __cpu_features2 variable to hold the additional bits. This matches libgcc as far as I can tell.

Reviewers: echristo

Reviewed By: echristo

Subscribers: dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D53461

llvm-svn: 344830
2018-10-20 03:49:04 +00:00
Douglas Yung e61c8eb98f Revert commit r344670 as the test fails on a bot http://lab.llvm.org:8011/builders/clang-cmake-armv7-full/builds/2683/.
llvm-svn: 344771
2018-10-19 04:09:32 +00:00
Dean Michael Berris e85af163bc [XRay][compiler-rt] Generational Buffer Management
Summary:
This change updates the buffer queue implementation to support using a
generation number to identify the lifetime of buffers. This first part
introduces the notion of the generation number, without changing the way
we handle the buffers yet.

What's missing here is the cleanup of the buffers. Ideally we'll keep
the two most recent generations. We need to ensure that before we do any
writes to the buffers, that we check the generation number(s) first.

Those changes will follow-on from this change.

Depends on D52588.

Reviewers: mboerger, eizan

Subscribers: llvm-commits, jfb

Differential Revision: https://reviews.llvm.org/D52974

llvm-svn: 344670
2018-10-17 06:57:50 +00:00
Petr Hosek 4886652773 [XRay] Use fully qualified name for LogWriter
This avoids the build error in newer versions of GCC.

llvm-svn: 344579
2018-10-16 02:06:08 +00:00
Petr Hosek 6a8cedef25 [XRay] Encapsulate all FD log related logic into a class
This abstracts away the file descriptor related logic which makes it
easier to port XRay to platform that don't use file descriptors or
file system for writing the log data, such as Fuchsia.

Differential Revision: https://reviews.llvm.org/D52161

llvm-svn: 344578
2018-10-16 01:24:46 +00:00
Dean Michael Berris 3c01508409 [XRay][compiler-rt] FDR Mode Controller
Summary:
This change implements a controller for abstracting away the details of
what happens when tracing with FDR mode. This controller type allows us
to test in isolation the various cases where we're encountering function
entry, exit, and other kinds of events we are handling when FDR mode is
enabled.

This change introduces a number of testing facilities we've needed to
better support expressing the conditions we need for the unit tests. We
leave some TODOs for moving those utilities into the LLVM project,
sitting in the `Testing` library, to make matching conditions on XRay
`Trace` instances through googlemock more manageable and declarative.

We don't wire in the controller right away, to allow us to incrementally
update the implementation(s) as we increase testing coverage of the
controller type. There's a need to re-think the way we're managing
buffers in a multi-threaded environment, which is more invasive than
this implementation.

This step in the process allows us to encode our assumptions in the
implementation of the controller, and then evolve the buffer queue
implementation to support generational buffer management to ensure we
can continue to support the cases we're already supporting with the
controller.

Reviewers: mboerger, eizan

Subscribers: mgorny, llvm-commits, jfb

Differential Revision: https://reviews.llvm.org/D52588

llvm-svn: 344488
2018-10-15 02:57:06 +00:00
Jonathan Metzman 9d0f3206ce [libfuzzer][Windows] Silence linker warning in unittest
Summary:
Silence warning when linking unittest binary by not passing
-lstdc++ to the linker since it is ignored.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: mgorny

Differential Revision: https://reviews.llvm.org/D53225

llvm-svn: 344480
2018-10-14 17:07:40 +00:00
Evgeniy Stepanov 9ab897dcb5 [sanitizer] Avoid extra newlines in syslog.
Fix line splitting logic to avoid sending empty lines to syslog, as
that adds extra newlines.

llvm-svn: 344426
2018-10-12 22:07:54 +00:00
Arnaud A. de Grandmaison b07b5a4115 [TSan] Cleanup TSan runtime support for Go on linux-aarch64. NFC.
This is a follow-up patch to r342541. After further investigations, only
48bits VMA size can be supported. As this is enforced in function
InitializePlatformEarly from lib/rt1/tsan_platform_linux.cc, the access
to the global variable vmaSize variable + switch can be removed. This
also addresses a comment from https://reviews.llvm.org/D52167.

vmaSize of 39 or 42bits are not compatible with a Go program memory
layout as the Go heap will not fit in the shadow memory area.

Patch by: Fangming Fang <Fangming.Fang@arm.com>

llvm-svn: 344329
2018-10-12 10:01:09 +00:00
Roman Lebedev d32c0d1466 [compiler-rt][ubsan] Split Implicit Integer Truncation Sanitizer into unsigned and signed checks
Summary:
This is compiler-rt part.
clang part is D50901.

Reviewers: rsmith, vsk, filcab, Sanitizers

Reviewed by: filcab

Differential Revision: https://reviews.llvm.org/D50902

llvm-svn: 344231
2018-10-11 09:09:52 +00:00
Kostya Serebryany dac7b2abaf [hwasan] more compact printing for 'Previosly allocated frames'
llvm-svn: 344210
2018-10-11 00:34:20 +00:00
Kostya Serebryany d7c60e42e3 [hwasan] when reporting a bug, print some very basic information about the heap chunk (in addition to the more detailed info that we may fail to show)
llvm-svn: 344193
2018-10-10 22:24:44 +00:00
Kostya Serebryany a393399741 [hwasan] print all threads in a bug report
llvm-svn: 344174
2018-10-10 18:56:31 +00:00
Kostya Serebryany 3fc15200e2 [hwasan] print more tags around the buggy address, and do it with a single Printf
llvm-svn: 344169
2018-10-10 18:32:31 +00:00
Martin Storsjo f0c286f8fb [sanitizers] [windows] Use a linker directive pragma for psapi
This allows users of static libraries (such as ubsan) to link without
knowing about this transitive dependency, if using the psapi functions
that require linking to a separate psapi library. Since Windows 7,
these functions (EnumProcessModules, GetModuleInformation,
GetProcessMemoryInfo) are remapped to K32- prefixed ones, available in
kernel32.dll.

Differential Revision: https://reviews.llvm.org/D53012

llvm-svn: 344126
2018-10-10 09:03:58 +00:00
George Karpenkov f28523bb3f [libFuzzer] Generalize the code for getting the previous offset for different architectures
Without this change, tests in coverage.test and dump_coverage.test are
failing on non-x86_64 platforms.
The diff is copied from sanitizer_common library, an alternative would
be to link it together with libFuzzer.

Differential Revision: https://reviews.llvm.org/D53040

llvm-svn: 344104
2018-10-10 00:57:44 +00:00
Matt Davis 839ec9d9a4 [compiler-rt] Remove unused formal parameters from allocateOneNode. NFC.
Summary: This is just a minor cleanup to the allocateOneNode interface.  The formals are no-longer used, so I just removed them.

Reviewers: davidxl, void

Reviewed By: davidxl

Subscribers: dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D52509

llvm-svn: 344073
2018-10-09 20:10:28 +00:00
Kamil Rytarowski d98081a783 Disable failing tests lib/asan/tests on NetBSD
These isses are not analyzed.

llvm-svn: 344045
2018-10-09 13:25:13 +00:00
Kamil Rytarowski daf662c492 Skip unsupported MSan tests on NetBSD
libm functions remquol and lgammal are missing on NetBSD.

llvm-svn: 344042
2018-10-09 12:58:35 +00:00
Kamil Rytarowski 88e545ec3d Remove remnant code of using indirect syscall on NetBSD
Summary:
The NetBSD version of internal routines no longer call
the indirect syscall interfaces, as these functions were
switched to lib calls.

Remove the remnant code complication that is no
longer needed after this change. Remove the variations
of internal_syscall, as they were NetBSD specific.

No functional change intended.

Reviewers: vitalybuka, joerg, javed.absar

Reviewed By: vitalybuka

Subscribers: kubamracek, fedor.sergeev, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D52955

llvm-svn: 343988
2018-10-08 18:04:46 +00:00
David Carlier b07407e6af [Sanitizer] fix internal_sysctlbyname build for FreeBSD.
llvm-svn: 343964
2018-10-08 12:18:19 +00:00
Reid Kleckner 11782759e3 Revert r343606/r342652 "[winasan] Unpoison the stack in NtTerminateThread""
This still seems to be causing pnacl + asan to crash.

llvm-svn: 343876
2018-10-05 18:48:53 +00:00
Kamil Rytarowski 57b0da0688 Unwind local macro DEFINE_INTERNAL()
No functional change intended.

This is a follow up of a suggestion from D52793.

llvm-svn: 343870
2018-10-05 18:07:34 +00:00
Kamil Rytarowski 980b424cbf Introduce internal_sysctlbyname in place of sysctlbyname
Summary:
This change will allow to install sysctlbyname() interceptors
more easily in sanitizers.

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D52793

llvm-svn: 343840
2018-10-05 06:58:02 +00:00
David Carlier 00ef67dc3f Esan for FreeBSD, forgotten file.
llvm-svn: 343815
2018-10-04 21:07:21 +00:00
David Carlier bbe5d55fea [Esan] Port cache frag to FreeBSD
Data involving struct accesses accounting work (plan to support only efficiency-cache-frag flag in the frontend side).

Reviewers: krytarowski, vitalybuka, jfb

Reviewed By : vitalybuka

Differential Revision: https://reviews.llvm.org/D52608

llvm-svn: 343812
2018-10-04 20:58:18 +00:00
Evgeniy Stepanov edede5daff [hwasan] Fix top PC in error stack trace being off by 1 instruction.
Summary:
GetStackTrace treats top PC as a return address from an error reporting
function, and adjusts it down by 1 instruction. This is not necessary in
a signal handler, so adjust PC up to compensate.

Reviewers: kcc, vitalybuka, jfb

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D52802

llvm-svn: 343638
2018-10-02 21:52:02 +00:00
Petr Hosek a70ece45eb [lib/fuzzer] Fix logging for Fuchsia
This change fixes two aspects of logging on Fuchsia:

1. When logging to stdout, the output file descriptor should not be
closed in ExecuteCommand, as it prevent fdio_spawn_etc from succeeding
in subsequent calls.
2. When logging to a file in anything other than standalone Zircon, the
log file needs to be created in mutable storage. The best indicator
where this is relative to the Fuchsia component will be given by
'-artifact_prefix=...', so save the log file relative to that path.

Patch by: aarongreen
Differential Revision: https://reviews.llvm.org/D52562

llvm-svn: 343607
2018-10-02 17:21:04 +00:00
David Major 6d6c9150f9 Reland r342652 "[winasan] Unpoison the stack in NtTerminateThread"
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.

Differential Revision: https://reviews.llvm.org/D52091

llvm-svn: 343606
2018-10-02 17:17:12 +00:00
Vitaly Buka 83e57e2123 [sanitizer] Include inlined frames into __sanitizer_symbolize_pc output
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.

New clients can read until they get empty string.

Reviewers: eugenis, morehouse

Subscribers: kubamracek, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D52743

llvm-svn: 343605
2018-10-02 17:01:18 +00:00
Kristina Brooks 59500f7a0b [Arm builtins] Remove non-necessary IS check
This patch removes the instruction set check to make the msr APSR_nzcvq,
ip instruction only execute if Thumb2 is used.
The APSR is a subset of the bits of the CPSR
(B.1.3.3 of the Arm v7 A and R ARM [1]) and is only available for A and
R profiles.
However in section B.9.3.11 of the same document we see that:

"In the A and R profiles, APSR_nzcvq is the same as CPSR_f"
"ARM recommends the APSR forms when only the N, Z, C, V, Q, and GE[3:0]
bits are being written."

This patch also make those files assemble for Armv8-M Mainline
architecture profile.

The builtins were cross-compiled for Arm, Aarch64 and Armv6-M, Armv7-M
and Armv7E-M targets.
Cross-compiled tests were executed for Arm target.

[1]: https://developer.arm.com/docs/ddi0406/latest/arm-architecture-reference-manual-armv7-a-and-armv7-r-edition

Patch by hug-dev (Hugues de Valon).

Differential Revision: https://reviews.llvm.org/D51854

llvm-svn: 343601
2018-10-02 16:32:32 +00:00
Jessica Paquette 9cfc8eca70 Revert "[sanitizer] Include inlined frames into __sanitizer_symbolize_pc output"
This reverts r343554.

It was breaking some bots:
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA/49997/

llvm-svn: 343600
2018-10-02 16:28:52 +00:00
Stephen Hines 80e7d65a42 [sanitizer] Use -Wl,-z,global on Android for sanitizers except UBsan
Summary:
This essentially reverts r337010 since it breaks UBSan, which is used
for a few platform libraries. The "-z global" flag is now added for
Scudo as well. The only other sanitizer shared libraries are for asan
and hwasan, which have also been reinstated to use the global flag.

Reviewers: cryptoad, eugenis

Reviewed By: cryptoad

Subscribers: kubamracek, mgorny, delcypher, #sanitizers, nickdesaulniers, chh, kongyi, pirama, llvm-commits

Differential Revision: https://reviews.llvm.org/D52770

llvm-svn: 343599
2018-10-02 16:19:42 +00:00
Vitaly Buka ae1bbea890 [sanitizer] Include inlined frames into __sanitizer_symbolize_pc output
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.

New clients can read until they get empty string.

Reviewers: eugenis, morehouse

Subscribers: kubamracek, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D52743

llvm-svn: 343554
2018-10-02 00:29:41 +00:00
Douglas Yung 22d4948a50 Remove a path-length limitation for the xray logfile.
Reviewers: MaskRay

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D52728

Patch by Jeremy Morse!

llvm-svn: 343524
2018-10-01 20:03:53 +00:00
Kamil Rytarowski 162aac511a Switch syscall(2)/__syscall(2) calls to libc calls on NetBSD
Summary:
When possible, switch syscall(2)/__syscall(2) calls
to direct calls of internal libc symbols.

Add a new function to detect address of a libc
symbol of a function that could be intercepted.
With the address detector in GetRealLibcAddress(),
an optional interceptor of libc call will be bypassed.

The original approach with syscall(2)/__syscall(2)
wasn't portable across supported ABIs and CPU
architectures. The indirect syscall interface is
also a candidate for removal in future revisions
of NetBSD, as the C language ABI is not a good
domain for serialization of arbitrary functions
arguments.

Switch the following functions to libc calls:
 - internal_mmap()
 - internal_munmap()
 - internal_mprotect()
 - internal_close()
 - internal_open()
 - internal_read()
 - internal_write()
 - internal_ftruncate()
 - internal_stat()
 - internal_lstat()
 - internal_fstat()
 - internal_dup2()
 - internal_readlink()
 - internal_unlink()
 - internal_rename()
 - internal_sched_yield()
 - internal__exit()
 - internal_sleep()
 - internal_execve()
 - NanoTime()
 - internal_clock_gettime()
 - internal_waitpid()
 - internal_getpid()
 - internal_getppid()
 - internal_getdents()
 - internal_lseek()
 - internal_sigaltstack()
 - internal_fork()
 - internal_sigprocmask()
 - internal_sysctl()
 - internal_sigemptyset()
 - internal_sigfillset()
 - GetTid()
 - TgKill()

This revision leaves room for refactoring in subsequent commits.

Reviewers: vitalybuka, kcc, joerg

Reviewed By: vitalybuka

Subscribers: mgorny, fedor.sergeev, jfb, loverszhaokai, devnexen, kubamracek, llvm-commits, ro, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D51419

llvm-svn: 343523
2018-10-01 19:50:12 +00:00
David Carlier 50a46a5961 Unbreak linux cfi build
llvm-svn: 343512
2018-10-01 18:14:02 +00:00
David Carlier dc0624844f [Cfi] Compiling cfi library on FreeBSD and NetBSD
Making the library slighty more portable.

Reviewers: vitalybuka, krytarowski

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D51682

llvm-svn: 343510
2018-10-01 18:01:55 +00:00
Dan Liew b6a07f5886 [NFC] Fix `-Wcast-qual` warnings in Darwin `internal_sysctl(...)` implementation.
Summary: The warnings were introduced in r341187 and r341192.

Reviewers: kubamracek, george.karpenkov, krytarowski

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52717

llvm-svn: 343496
2018-10-01 16:51:01 +00:00
Kamil Rytarowski a14b908381 Switch sanitizer_procmaps_bsd to internal_sysctl
Summary:
Stop using directly sysctl(3) routines in sanitizer_procmaps_bsd
and replace it with internal_sysctl().

This will allow to install interceptors for sysctl(3).

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D52646

llvm-svn: 343370
2018-09-29 07:45:03 +00:00
Hans Wennborg 83cd9b32dd Revert r342652 "[winasan] Unpoison the stack in NtTerminateThread"
This seems to cause the thread's exit code to be clobbered, breaking
Chromium tests.

Also revert follow-up r342654.

> In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
>
> Differential Revision: https://reviews.llvm.org/D52091

llvm-svn: 343322
2018-09-28 14:41:25 +00:00
Fangrui Song 58f209f5d7 [XRay] Guard local variables with `static` and struct with unnamed namespaces
Summary:
This is for coding standard conformance, and for fixing an ODR violation
issue: __xray::ThreadLocalData is defined twice and differently in
xray_fdr_logging.cc and xray_basic_logging.cc

Reviewers: dberris, mboerger, eizan

Reviewed By: dberris

Subscribers: delcypher, jfb, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D52639

llvm-svn: 343289
2018-09-28 02:05:52 +00:00
Dean Michael Berris c84c46fc0c [XRay] Add LD_LIBRARY_PATH to env variables for Unit Tests
Summary:
This change allows us to use the library path from which the LLVM
libraries are installed, in case the LLVM installation generates shared
libraries.

This should address llvm.org/PR39070.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52597

llvm-svn: 343280
2018-09-27 23:15:05 +00:00
Kostya Kortchinsky 67392feb49 [sanitizer] Disable failing Android test after D52371
Summary:
The default values used for Space/Size for the new SizeClassMap do not work
with Android. The Compact map appears to be in the same boat.
Disable the test on Android for now to turn the bots green, but there is no
reason Compact & Dense should not have an Android test.
Added a FIXME, I will revisit this soon.

Reviewers: eugenis

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52623

llvm-svn: 343252
2018-09-27 19:15:40 +00:00
Kostya Kortchinsky 7685301d79 [sanitizer] Introduce a new SizeClassMap with minimal amount of cached entries
Summary:
_Note_: I am not attached to the name `DenseSizeClassMap`, so if someone has a
better idea, feel free to suggest it.

The current pre-defined `SizeClassMap` hold a decent amount of cached entries,
either in cheer number of, or in amount of memory cached.

Empirical testing shows that more compact per-class arrays (whose sizes are
directly correlated to the number of cached entries) are beneficial to
performances, particularly in highly threaded environments.

The new proposed `SizeClassMap` has the following properties:
```
c00 => s: 0 diff: +0 00% l 0 cached: 0 0; id 0
c01 => s: 16 diff: +16 00% l 4 cached: 8 128; id 1
c02 => s: 32 diff: +16 100% l 5 cached: 8 256; id 2
c03 => s: 48 diff: +16 50% l 5 cached: 8 384; id 3
c04 => s: 64 diff: +16 33% l 6 cached: 8 512; id 4
c05 => s: 80 diff: +16 25% l 6 cached: 8 640; id 5
c06 => s: 96 diff: +16 20% l 6 cached: 8 768; id 6
c07 => s: 112 diff: +16 16% l 6 cached: 8 896; id 7

c08 => s: 128 diff: +16 14% l 7 cached: 8 1024; id 8
c09 => s: 144 diff: +16 12% l 7 cached: 7 1008; id 9
c10 => s: 160 diff: +16 11% l 7 cached: 6 960; id 10
c11 => s: 176 diff: +16 10% l 7 cached: 5 880; id 11
c12 => s: 192 diff: +16 09% l 7 cached: 5 960; id 12
c13 => s: 208 diff: +16 08% l 7 cached: 4 832; id 13
c14 => s: 224 diff: +16 07% l 7 cached: 4 896; id 14
c15 => s: 240 diff: +16 07% l 7 cached: 4 960; id 15

c16 => s: 256 diff: +16 06% l 8 cached: 4 1024; id 16
c17 => s: 320 diff: +64 25% l 8 cached: 3 960; id 49
c18 => s: 384 diff: +64 20% l 8 cached: 2 768; id 50
c19 => s: 448 diff: +64 16% l 8 cached: 2 896; id 51

c20 => s: 512 diff: +64 14% l 9 cached: 2 1024; id 48
c21 => s: 640 diff: +128 25% l 9 cached: 1 640; id 49
c22 => s: 768 diff: +128 20% l 9 cached: 1 768; id 50
c23 => s: 896 diff: +128 16% l 9 cached: 1 896; id 51

c24 => s: 1024 diff: +128 14% l 10 cached: 1 1024; id 48
c25 => s: 1280 diff: +256 25% l 10 cached: 1 1280; id 49
c26 => s: 1536 diff: +256 20% l 10 cached: 1 1536; id 50
c27 => s: 1792 diff: +256 16% l 10 cached: 1 1792; id 51

c28 => s: 2048 diff: +256 14% l 11 cached: 1 2048; id 48
c29 => s: 2560 diff: +512 25% l 11 cached: 1 2560; id 49
c30 => s: 3072 diff: +512 20% l 11 cached: 1 3072; id 50
c31 => s: 3584 diff: +512 16% l 11 cached: 1 3584; id 51

c32 => s: 4096 diff: +512 14% l 12 cached: 1 4096; id 48
c33 => s: 5120 diff: +1024 25% l 12 cached: 1 5120; id 49
c34 => s: 6144 diff: +1024 20% l 12 cached: 1 6144; id 50
c35 => s: 7168 diff: +1024 16% l 12 cached: 1 7168; id 51

c36 => s: 8192 diff: +1024 14% l 13 cached: 1 8192; id 48
c37 => s: 10240 diff: +2048 25% l 13 cached: 1 10240; id 49
c38 => s: 12288 diff: +2048 20% l 13 cached: 1 12288; id 50
c39 => s: 14336 diff: +2048 16% l 13 cached: 1 14336; id 51

c40 => s: 16384 diff: +2048 14% l 14 cached: 1 16384; id 48
c41 => s: 20480 diff: +4096 25% l 14 cached: 1 20480; id 49
c42 => s: 24576 diff: +4096 20% l 14 cached: 1 24576; id 50
c43 => s: 28672 diff: +4096 16% l 14 cached: 1 28672; id 51

c44 => s: 32768 diff: +4096 14% l 15 cached: 1 32768; id 48
c45 => s: 40960 diff: +8192 25% l 15 cached: 1 40960; id 49
c46 => s: 49152 diff: +8192 20% l 15 cached: 1 49152; id 50
c47 => s: 57344 diff: +8192 16% l 15 cached: 1 57344; id 51

c48 => s: 65536 diff: +8192 14% l 16 cached: 1 65536; id 48
c49 => s: 81920 diff: +16384 25% l 16 cached: 1 81920; id 49
c50 => s: 98304 diff: +16384 20% l 16 cached: 1 98304; id 50
c51 => s: 114688 diff: +16384 16% l 16 cached: 1 114688; id 51

c52 => s: 131072 diff: +16384 14% l 17 cached: 1 131072; id 48
c53 => s: 64 diff: +0 00% l 0 cached: 8 512; id 4
Total cached: 864928 (152/432)
```

It holds a bit less of 1MB of cached entries at most, and the cache fits in a 
page.

The plan is to use this map by default for Scudo once we make sure that there
is no unforeseen impact for any of current use case.

Benchmarks give the most increase in performance (with Scudo) when looking at
highly threaded/contentious environments. For example, rcp2-benchmark
experiences a 10K QPS increase (~3%), and a decrease of 50MB for the max RSS
(~10%). On platforms like Android where we only have a couple of caches,
performance remain similar.

Reviewers: eugenis, kcc

Reviewed By: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52371

llvm-svn: 343246
2018-09-27 18:20:42 +00:00
Evgeniy Stepanov f42aa29b68 [sanitizer] AndroidGetApiLevel for static executables.
A version of AndroidGetApiLevel for static executables that is
completely compile-time.

The generic version uses dl_iterate_phdr which, even though it works
in static executables, crashes if called before libc is initialized.

llvm-svn: 343153
2018-09-26 23:48:13 +00:00
Martin Storsjo a89731ca71 [asan] [windows] Don't use CheckFailed in dynamic runtime thunk mode
Since SVN r342651, CheckFailed isn't exported from asan-dynamic. See
comments in https://reviews.llvm.org/D52279#1246222 for a longer
discussion on this issue.

Differential Revision: https://reviews.llvm.org/D52566

llvm-svn: 343136
2018-09-26 19:49:35 +00:00
David Major e705624cff [winasan] Pin the ASan DLL to prevent unloading
Differential Revision: https://reviews.llvm.org/D52505

llvm-svn: 343123
2018-09-26 16:28:39 +00:00
Martin Storsjo 50de4518c5 [CMake] [MinGW] Build address sanitizer for MinGW if building with clang
Differential Revision: https://reviews.llvm.org/D51885

llvm-svn: 343074
2018-09-26 06:53:01 +00:00
Martin Storsjo fed729f28e [ASan] [MinGW] Only try to export MSVC specific C++ symbols if building with a MSVC like compiler
Differential Revision: https://reviews.llvm.org/D51878

llvm-svn: 343073
2018-09-26 06:52:55 +00:00
Martin Storsjo 32df11e3e3 Try to fix cosmetics to keep lines below 80 chars. NFC.
This should fix following buildbot errors:
http://lab.llvm.org:8011/builders/clang-ppc64le-linux/builds/20371

llvm-svn: 343031
2018-09-25 21:27:07 +00:00
Martin Storsjo 45205a2834 [Sanitizers] [MinGW] Produce undecorated symbols for /export: directives when in MinGW mode
In MinGW mode, the /export: directives in object files are interpreted
differently; the symbols are not assumed to be decorated.

Since the GNU linker doesn't support the /alternatename and /include
directives, there's no such distinction for them in lld. This assumes
that the resulting sanitizer libraries will be linked by lld.

Differential Revision: https://reviews.llvm.org/D51882

llvm-svn: 343015
2018-09-25 19:50:39 +00:00
Martin Storsjo 693f3a573c [ASan] [Windows] Avoid including windows.h in asan_malloc_win.cc
Instead provide manual declarations of the used types, to avoid
pulling in conflicting declarations of some of the functions that
are to be overridden.

Differential Revision: https://reviews.llvm.org/D51914

llvm-svn: 343014
2018-09-25 19:50:36 +00:00
Martin Storsjo fd9aa7e078 [Sanitizers] [MinGW] Check for __i386__ in addition to _M_IX86 for i386 specific details
The MinGW headers do define _M_IX86 (contrary to clang-cl and
MSVC where it is a compiler predefined macro), but the headers that
define it aren't included here.

Also check these defines for setting the symbol prefix, instead of
inconsistently using _WIN64 for that.

Differential Revision: https://reviews.llvm.org/D51883

llvm-svn: 343013
2018-09-25 19:50:32 +00:00
Evgeniy Stepanov 090f0f9504 [hwasan] Record and display stack history in stack-based reports.
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.

The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.

Developed in collaboration with Kostya Serebryany.

Reviewers: kcc

Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52249

llvm-svn: 342923
2018-09-24 23:03:34 +00:00
Evgeniy Stepanov 20c4999e8b Revert "[hwasan] Record and display stack history in stack-based reports."
This reverts commit r342921: test failures on clang-cmake-arm* bots.

llvm-svn: 342922
2018-09-24 22:50:32 +00:00
Evgeniy Stepanov 9043e17edd [hwasan] Record and display stack history in stack-based reports.
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.

The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.

Developed in collaboration with Kostya Serebryany.

Reviewers: kcc

Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52249

llvm-svn: 342921
2018-09-24 21:38:42 +00:00
Jordan Rupprecht dc48c4fff4 [compiler-rt] [builtins] Add logb/logbf/logbl methods to compiler-rt to avoid libm dependencies when possible.
Summary:
The complex division builtins (div?c3) use logb methods from libm to scale numbers during division and avoid rounding issues. However, these come from libm, meaning anyone that uses --rtlib=compiler-rt also has to include -lm. Implement logb* methods for standard ieee 754 floats so we can avoid -lm on those platforms, falling back to the old behavior (using either logb() or `__builtin_logb()`) when not supported.

These new methods are defined internally as `__compiler_rt_logb` so as not to conflict with the libm definitions in any way.

This fixes just the libm methods mentioned in PR32279 and PR28652. libc is still required, although that seems to not be an issue.

Note: this is proposed as an alternative to just adding -lm: D49330.

Reviewers: efriedma, compnerd, scanon, echristo

Reviewed By: echristo

Subscribers: jsji, echristo, nemanjai, dberris, mgorny, kbarton, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49514

llvm-svn: 342917
2018-09-24 20:39:19 +00:00
Dan Liew 071fdc1b01 Fix the configuration of the Primary allocator for Darwin ARM64 by
changing the value of `SANITIZER_MMAP_RANGE_SIZE` to something more
sensible. The available VMA is at most 64GiB and not 256TiB that
was previously being used.

This change gives us several wins:

* Drastically improves LeakSanitizer performance on
  Darwin ARM64 devices. On a simple synthentic benchmark
  this took leak detection time from ~30 seconds to 0.5 seconds
  due to the `ForEachChunk(...)` method enumerating a much smaller
  number of regions. Previously we would pointlessly iterate
  over a large portion of the SizeClassAllocator32's ByteMap
  that would could never be set due it being configured for a much
  larger VM space than is actually availble.

* Decreases the memory required for the Primary allocator.
  Previously the ByteMap inside the the allocator used
  an array of pointers that took 512KiB of space. Now the required
  space for the array is 128 bytes.

rdar://problem/43509428

Differential Revision: https://reviews.llvm.org/D51173

llvm-svn: 342868
2018-09-24 09:28:35 +00:00
Dean Michael Berris f578aaa058 [XRay] Clean up XRay build configuration
Summary:
This change spans both LLVM and compiler-rt, where we do the following:

- Add XRay to the LLVMBuild system, to allow for distributing the XRay
  trace loading library along with the LLVM distributions.

- Use `llvm-config` better in the compiler-rt XRay implementation, to
  depend on the potentially already-distributed LLVM XRay library.

While this is tested with the standalone compiler-rt build, it does
require that the LLVMXRay library (and LLVMSupport as well) are
available during the build. In case the static libraries are available,
the unit tests will build and work fine. We're still having issues with
attempting to use a shared library version of the LLVMXRay library since
the shared library might not be accessible from the standard shared
library lookup paths.

The larger change here is the inclusion of the LLVMXRay library in the
distribution, which allows for building tools around the XRay traces and
profiles that the XRay runtime already generates.

Reviewers: echristo, beanz

Subscribers: mgorny, hiraditya, mboerger, llvm-commits

Differential Revision: https://reviews.llvm.org/D52349

llvm-svn: 342859
2018-09-24 05:28:01 +00:00
Kostya Kortchinsky 7bdb0b9493 [sanitizer] Move __sanitizer_set_death_callback to a more appropriate file
Summary:
`__sanitizer_set_death_callback` is a public interface function wrapping
`SetUserDieCallback`. Said function is defined in `sanitizer_termination.cc`,
which is not included in all the RT. Moving the interface function to that
file allows to not have a spurious public fuinction in RT that don't use it.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52363

llvm-svn: 342747
2018-09-21 17:05:56 +00:00
Dean Michael Berris 0cb22386e0 [XRay][compiler-rt] Update use of internal_mmap
Summary:
The implementation of `internal_mmap(...)` deviates from the contract of
`mmap(...)` -- i.e. error returns are actually the equivalent of `errno`
results. We update how XRay uses `internal_mmap(...)` to better handle
these error conditions.

In the process, we change the default pointers we're using from `char*`
to `uint8_t*` to prevent potential usage of the pointers in the string
library functions that expect to operate on `char*`.

We also take the chance to "promote" sizes of individual `internal_mmap`
requests to at least page size bytes, consistent with the expectations
of calls to `mmap`.

Reviewers: cryptoad, mboerger

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D52361

llvm-svn: 342745
2018-09-21 16:34:42 +00:00
Jonathan Metzman 0744d3c5a1 [fuzzer] Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
Summary:
Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
to get externally defined functions (eg: LLVMFuzzerInitialize,
LLVMFuzzerCustomMutator, etc) working again.

Also enable tests that depended on these functions (on windows)

Reviewers: rnk, morehouse

Reviewed By: rnk, morehouse

Subscribers: rnk, morehouse, mgorny

Differential Revision: https://reviews.llvm.org/D51700

llvm-svn: 342698
2018-09-20 23:24:48 +00:00
David Carlier 362e6095a6 [Hwasan] interceptor macro / extra comma removal
gcc being pedantic, removing the unnecessary comma.

Reviewers: eugenis, kcc

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D52305

llvm-svn: 342680
2018-09-20 19:30:51 +00:00
David Major 468f53b58c [winasan] Unpoison the stack in NtTerminateThread
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.

Differential Revision: https://reviews.llvm.org/D52091

llvm-svn: 342652
2018-09-20 14:59:33 +00:00
Kostya Kortchinsky caa9619bce [sanitizer] Make __sanitizer::CheckFailed not public
Summary:
As far as I can tell, there is no reason why `__sanitizer::CheckFailed` should
be exported. Looking back in time, it was added with the FIXME with the
following by @timurrrr:

```
[*San/RTL] Fix minor breakage
Grumbling: this hasn't been caught by running 'make check-{a,l,t}san check-sanitizer'
```

I can't find any detail about the breakage, all tests seem to work for me, so
maybe Windows (@rnk?) or something I have no setup for.

The reason to make it private (past the FIXME) is that Scudo defines its own
(without callback) and I am trying to make the .so be loadable with the UBsan
one (that has its own public `CheckFailed`) with as little drama as possible.

Reviewers: eugenis, rnk

Reviewed By: eugenis, rnk

Subscribers: kubamracek, delcypher, #sanitizers, timurrrr, rnk, llvm-commits

Differential Revision: https://reviews.llvm.org/D52279

llvm-svn: 342651
2018-09-20 14:50:06 +00:00
David Major c4624d9e3f [winasan] Reduce hotpatch prefix check to 8 bytes
Same idea as r310419: The 8 byte nop is a suffix of the 9 byte nop, and we need at most 6 bytes.

Differential Revision: https://reviews.llvm.org/D51788

llvm-svn: 342649
2018-09-20 14:28:50 +00:00
Dean Michael Berris 1f60207984 [XRay][compiler-rt] FDRLogWriter Abstraction
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.

The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.

In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.

We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module. We also add
the terminfo library detection along with inclusion of the appropriate
compiler flags for header include lookup.

Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.

Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52220

llvm-svn: 342617
2018-09-20 05:22:37 +00:00
Yi Kong 0c4cf6436a [builtins] Add __emutls_unregister_key function
This is called by Bionic on dlclose to delete the emutls pthread key.

The return value of pthread_key_delete is unchecked and behaviour of
multiple calls to the method is dependent on the implementation of
pthread_key_delete.

Differential Revision: https://reviews.llvm.org/D52251

llvm-svn: 342608
2018-09-20 01:02:13 +00:00
Evgeniy Stepanov 09e7f243f1 Revert "[XRay][compiler-rt] FDRLogWriter Abstraction" and 1 more.
Revert the following 2 commits to fix standalone compiler-rt build:
* r342523 [XRay] Detect terminfo library
* r342518 [XRay][compiler-rt] FDRLogWriter Abstraction

llvm-svn: 342596
2018-09-19 22:29:56 +00:00
Kostya Kortchinsky 851a7c9b2b [sanitizer][fuchsia] Fix VMAR leak
Summary:
Destroy and close a range's vmar if all its memory was unmapped.

This addresses some performance regression due to the proliferation of vmars
when Secondary backed allocations are concerned with Scudo on Fuchsia.

When a Secondary backed allocation was freed, the associated
`ReservedAddressRange` was going away after unmapping the entirety of the
mapping, but without getting rid of the associated vmar properly (which
was created specifically for that mapping). This resulted in an increase of
defunct vmars, that in turn slowed down further new vmar allocations.

This appears to solve ZX-2560/ZX-2642, at least on QEMU.

Reviewers: flowerhack, mcgrathr, phosek, mseaborn

Reviewed By: mcgrathr

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52242

llvm-svn: 342584
2018-09-19 19:50:35 +00:00
Arnaud A. de Grandmaison 86a5e43656 [compiler-rt][TSan] Add TSan runtime support for Go on linux-aarch64.
Summary:
This patch adds TSan runtime support for Go on linux-aarch64
platforms. This enables people working on golang to implement their
platform/language part of the TSan support.

Basic testing is done with lib/tsan/go/buildgo.sh. Additional testing will be
done as part of the work done in the Go project.

It is intended to support other VMA sizes, except 39 which does not
have enough bits to support the Go heap requirements.

Patch by Fangming Fang <Fangming.Fang@arm.com>.

Reviewers: kubamracek, dvyukov, javed.absar

Subscribers: mcrosier, dberris, mgorny, kristof.beyls, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D52167

llvm-svn: 342541
2018-09-19 11:56:20 +00:00
Dean Michael Berris ff93d3a8aa [XRay] Detect terminfo library
Instead of assuming `-ltinfo` works, check whether there's terminfo
support on the host where LLVMSupport is compiled.

Follow-up to D52220.

llvm-svn: 342523
2018-09-19 01:35:52 +00:00
Dean Michael Berris b64f71b029 [XRay][compiler-rt] FDRLogWriter Abstraction
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.

The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.

In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.

We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module.

Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.

Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52220

llvm-svn: 342518
2018-09-18 23:59:32 +00:00
Kristina Brooks 22db696549 [builtins] Fix c?zdi2 on sparc64/Linux and ignore riscv32
On sparc64/Linux, sparc64 isn't defined; the canonical way of
checking for sparc64 is sparc && arch64, which also works on the
BSDs and Solaris. Since this problem does not occur on 32-bit
architectures, riscv32 can be ignored. This fixes and refines rL324593.

Patch by jrtc27 (James Clarke)

Differential Revision: https://reviews.llvm.org/D43146

llvm-svn: 342504
2018-09-18 18:56:52 +00:00
Matt Morehouse 1e1f3c8298 [libFuzzer] Avoid fuzzer symbols being hidden.
Fixes https://bugzilla.mozilla.org/show_bug.cgi?id=1490845#c5.

Patch By: Mike Hommey

llvm-svn: 342423
2018-09-17 23:08:15 +00:00
Petr Hosek 040ab65c53 [sanitizer_common] Fuchsia now supports .preinit_array
Support for .preinit_array has been implemented in Fuchsia's libc,
add Fuchsia to the list of platforms that support this feature.

Differential Revision: https://reviews.llvm.org/D52155

llvm-svn: 342357
2018-09-17 05:22:26 +00:00
Dean Michael Berris 1a23d3bbce [XRay] Simplify FDR buffer management
Summary:
This change makes XRay FDR mode use a single backing store for the
buffer queue, and have indexes into that backing store instead. We also
remove the reliance on the internal allocator implementation in the FDR
mode logging implementation.

In the process of making this change we found an inconsistency with the
way we're returning buffers to the queue, and how we're setting the
extents. We take the chance to simplify the way we're managing the
extents of each buffer. It turns out we do not need the indirection for
the extents, so we co-host the atomic 64-bit int with the buffer object.
It also seems that we've not been returning the buffers for the thread
running the flush functionality when writing out the files, so we can
run into a situation where we could be missing data.

We consolidate all the allocation routines now into xray_allocator.h,
where we used to have routines defined in xray_buffer_queue.cc.

Reviewers: mboerger, eizan

Subscribers: jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52077

llvm-svn: 342356
2018-09-17 03:09:01 +00:00
Dean Michael Berris d5577aea07 [XRay] Fix FDR initialization
Follow-up to D51606.

llvm-svn: 342355
2018-09-17 02:49:17 +00:00
Petr Hosek d197ebf3ed [XRay] Remove the deprecated __xray_log_init API
This API has been deprecated three months ago and shouldn't be used
anymore, all clients should migrate to the new string based API.

Differential Revision: https://reviews.llvm.org/D51606

llvm-svn: 342318
2018-09-15 02:55:42 +00:00
Kostya Serebryany c173a703b5 [hwasan] use a single Printf per line when printing a report (more friendly to android logging)
llvm-svn: 342164
2018-09-13 19:14:22 +00:00
Dean Michael Berris 90a46bdec2 [XRay] Bug fixes for FDR custom event and arg-logging
Summary:
This change has a number of fixes for FDR mode in compiler-rt along with
changes to the tooling handling the traces in llvm.

In the runtime, we do the following:

- Advance the "last record" pointer appropriately when writing the
  custom event data in the log.

- Add XRAY_NEVER_INSTRUMENT in the rewinding routine.

- When collecting the argument of functions appropriately marked, we
  should not attempt to rewind them (and reset the counts of functions
  that can be re-wound).

In the tooling, we do the following:

- Remove the state logic in BlockIndexer and instead rely on the
  presence/absence of records to indicate blocks.

- Move the verifier into a loop associated with each block.

Reviewers: mboerger, eizan

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D51965

llvm-svn: 342122
2018-09-13 09:25:42 +00:00
Vlad Tsyrklevich aadfe4661d [TSan] Update test values
Similarly to before, D51985 again reduced the number of registers
required for the read/write routines causing this test to fail on
sanitizer-x86_64-linux-autoconf.

llvm-svn: 342092
2018-09-12 22:16:14 +00:00
Kostya Serebryany ee9bb8761c [hwasan] On every use-after-free print a developer note: the index of this heap object in the thread's deallocation ring buffer. Mostly useful to hwasan developers, will hopefully let us know the good size of the deallocation ring buffer
llvm-svn: 342014
2018-09-12 00:58:15 +00:00
Martin Storsjo e199a2d434 [libFuzzer] [Windows] Include windows.h and psapi.h with lowercase
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase, and matches how these
headers are included elsewhere in compiler-rt.

Also include these headers with angle brackets, as they are system
headers.

Differential Revision: https://reviews.llvm.org/D51913

llvm-svn: 341983
2018-09-11 19:11:54 +00:00
Martin Storsjo b2b0f859d4 [ASan] [Windows] Remove const from _msize function declaration parameter
This function isn't declared with a const parameter anywhere; neither
in MSVC (neither in ucrt or in older msvcrt versions) nor in mingw-w64.

Differential Revision: https://reviews.llvm.org/D51876

llvm-svn: 341903
2018-09-11 06:07:32 +00:00
Martin Storsjo 6dc8968124 [Windows] Include BaseTsd.h with lowercase
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase. This header actually also
is named with a lowercase name in the Windows SDK as well.

Differential Revision: https://reviews.llvm.org/D51877

llvm-svn: 341857
2018-09-10 20:28:06 +00:00
Richard Smith 4357ca653a Switch to using a reserved identifier for this internal compiler-rt function.
llvm-svn: 341736
2018-09-08 00:17:37 +00:00
Evgeniy Stepanov d263cb8ea1 [hwasan] Export memory stats through /proc/$PID/maps.
Adds a line to /proc/$PID/maps with more or less up-to-date memory
stats of the process.

llvm-svn: 341735
2018-09-08 00:11:12 +00:00
Dean Michael Berris edf0f6a79b [XRay] XRAY_NEVER_INSTRUMENT more functions, consolidate allocators
Summary:
In this change we apply `XRAY_NEVER_INSTRUMENT` to more functions in the
profiling implementation to ensure that these never get instrumented if
the compiler used to build the library is capable of doing XRay
instrumentation.

We also consolidate all the allocators into a single header
(xray_allocator.h) which sidestep the use of the internal allocator
implementation in sanitizer_common.

This addresses more cases mentioned in llvm.org/PR38577.

Reviewers: mboerger, eizan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D51776

llvm-svn: 341647
2018-09-07 10:16:14 +00:00
Evgeniy Stepanov 5b332abd66 [hwasan] Fix malloc overflow detection.
Check size limit before rounding up, otherwise malloc((size_t)-1)
would happily allocate 0 bytes.

Steal a nice test case from scudo.

llvm-svn: 341612
2018-09-07 00:27:11 +00:00
Kostya Serebryany f9ec62cb3a [hwasan] change the thread list so that main_thread can also be removed
llvm-svn: 341610
2018-09-07 00:16:55 +00:00
Evgeniy Stepanov fe4808eba7 [hwasan] Remove allocator interceptors.
Summary:
When building without COMPILER_RT_HWASAN_WITH_INTERCEPTORS, skip
interceptors for malloc/free/etc and only export their versions with
__sanitizer_ prefix.

Also remove a hack in mallinfo() interceptor that does not apply to
hwasan.

Reviewers: kcc

Subscribers: kubamracek, krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D51711

llvm-svn: 341598
2018-09-06 22:53:08 +00:00
Kostya Serebryany f1f556c179 [hwasan] fix pthread_exit
llvm-svn: 341594
2018-09-06 22:13:43 +00:00
Kostya Serebryany 950a1a3936 [hwasan] introduce __hwasan_print_memory_usage
llvm-svn: 341592
2018-09-06 22:08:41 +00:00
Alexander Potapenko 7f270fcf0a [MSan] store origins for variadic function parameters in __msan_va_arg_origin_tls
Add the __msan_va_arg_origin_tls TLS array to keep the origins for variadic function parameters.
Change the instrumentation pass to store parameter origins in this array.

This is a reland of r341528.

test/msan/vararg.cc doesn't work on Mips, PPC and AArch64 (because this
patch doesn't touch them), XFAIL these arches.
Also turned out Clang crashed on i80 vararg arguments because of
incorrect origin type returned by getOriginPtrForVAArgument() - fixed it
and added a test.

llvm-svn: 341554
2018-09-06 15:14:36 +00:00
Alexander Potapenko ac6595bd53 [MSan] revert r341528 to unbreak the bots
llvm-svn: 341541
2018-09-06 12:19:27 +00:00
Tim Northover 8f141016ee ARM: wrap call to __clzsi2 so that the name is correct on MachO.
MachO symbols are prefixed with an extra '_' (that's 3 in total for this
function), so assembly calls have to go through a wrapper to insert any prefix
needed.

llvm-svn: 341540
2018-09-06 12:13:46 +00:00
Alexander Potapenko 1a10ae0def [MSan] store origins for variadic function parameters in __msan_va_arg_origin_tls
Add the __msan_va_arg_origin_tls TLS array to keep the origins for
variadic function parameters.
Change the instrumentation pass to store parameter origins in this array.

llvm-svn: 341528
2018-09-06 08:50:11 +00:00
Kostya Serebryany 68902c7050 [hwasan] when verbose_threads==1, print the memory usage per thread
llvm-svn: 341507
2018-09-05 23:52:31 +00:00
Kostya Serebryany b29d42ee31 [hwasan] simplify the code, NFC
llvm-svn: 341501
2018-09-05 23:22:38 +00:00
Evgeniy Stepanov 6afce6a438 [hwasan] Don't handle signals on Android.
Sigtrap is used for error reporting, but all other signals are better
left for the platform.

In particular, sanitizer signal handlers do not dump registers or
memory which makes debugging harder for no good reason.

llvm-svn: 341500
2018-09-05 22:46:19 +00:00
David Carlier c3d1d107ff [Xray] Darwin providing defined value for weak symbols to fix linkage issue
- Temporary hack to make the buildbot failure stop on Darwin.

llvm-svn: 341445
2018-09-05 05:18:34 +00:00
Brian Cain 416296b8f7 Add glibc_prereq to platform limits mmsghdr
sendmmsg requires glibc >= 2.14.

Fixes PR38589.

Review: https://reviews.llvm.org/D51538
llvm-svn: 341442
2018-09-05 02:15:54 +00:00
Kostya Serebryany ee0e2f8323 [hwasan] revert r341435 as it breaks the bot on aarch64
llvm-svn: 341441
2018-09-05 01:29:08 +00:00
Kostya Serebryany c12cc9b85c [hwasan] fix colored output
llvm-svn: 341440
2018-09-05 01:27:48 +00:00
Kostya Serebryany 5d05be84b7 [hwasan] print thread IDs when reporting a bug (also had to fix pthread_create on Linux)
llvm-svn: 341438
2018-09-05 01:16:50 +00:00
Kostya Serebryany 9fbedcad71 [hwasan] use real TLS on linux to store the current thread -- this way we can call t->Destroy in __hwasan_thread_exit, same as on Android
llvm-svn: 341435
2018-09-05 00:17:23 +00:00
Kostya Serebryany 2768b52117 [hwasan] simplify the code, NFC
llvm-svn: 341432
2018-09-05 00:01:45 +00:00
Kostya Serebryany c551220de8 [hwasan] remove stale Thread:: data members. While doing so noticed that GetThreadStackAndTls was always called with 'at_initialization=true', fixed that.
llvm-svn: 341431
2018-09-04 23:57:09 +00:00
Kostya Serebryany 3d8f49a6bf [hwasan] add a unique id to a thread and add debug prints for thread creation/destruction
llvm-svn: 341428
2018-09-04 23:26:08 +00:00
Kostya Serebryany 470db78115 [sanitizer] optimize internal_memset for the most performance critical case (16-byte-aligned)
llvm-svn: 341420
2018-09-04 22:43:30 +00:00
Matt Morehouse 24568789c4 [libFuzzer] Enable tests on Windows
Summary:
Enable tests on Windows and make check-fuzzer pass on it.  Make
check-fuzzer pass on Windows by fixing libFuzzer, fixing tests, and by
disabling tests on Windows. Most of these are disabled temporarily as
support for the tests and the features they test will be added
incrementally.  Other tests will not be enabled since they require
things that are not on Windows (eg: afl_driver tests).  Every test
that was explicitly disabled on Windows has a comment explaining why
(unless obvious like merge-posix.test).

The lit.cfg file was modified to support running tests on windows.
fuzzer-dirs.test was fixed by making the Windows implementation print
the same error message as the posix version.
merge-control-file.test was fixed by making the test binary end with
the ".exe" extension (on all platforms).

Patch By: metzman

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: srhines, mgorny

Differential Revision: https://reviews.llvm.org/D51549

llvm-svn: 341385
2018-09-04 17:08:47 +00:00