Summary:
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD.
Introduce a generic CheckASLR() routine, that implements a check for the
current process. This flag depends on the global or per-process settings.
There is no simple way to disable ASLR in the build process from the
level of a sanitizer or during the runtime execution.
With ASLR enabled sanitizers that operate over the process virtual address
space can misbehave usually breaking with cryptic messages.
This check is dummy for !NetBSD.
Sponsored by <The NetBSD Foundation>
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: cryptoad, kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D47442
llvm-svn: 333985
The problem is reported in:
https://github.com/google/sanitizers/issues/945
We already disable as much as possible after multithreaded fork,
trace switching is last place that can hang due to basic
operations (memory accesses, function calls).
Disable it too.
llvm-svn: 331163
1. Allow to suppress by current stack.
We generally allow to suppress by all main stacks.
Current is probably the stack one wants to use to
suppress such reports.
2. Fix last lock stack restoration.
We trimmed shadow value by storing it in u32.
This magically worked for the test that provoked
the report on the main thread. But this breaks
for locks in any other threads.
llvm-svn: 331023
Summary:
Host symbolizer & stacktraces related code in their own RT:
`RTSanitizerCommonSymbolizer`, which is "libcdep" by nature. Symbolizer &
stacktraces specific code that used to live in common files is moved to a new
file `sanitizer_symbolizer_report.cc` as is.
The purpose of this is the enforce a separation between code that relies on
symbolization and code that doesn't. This saves the inclusion of spurious code
due to the interface functions with default visibility, and the extra data
associated.
The following sanitizers makefiles were modified & tested locally:
- dfsan: doesn't require the new symbolizer RT
- esan: requires it
- hwasan: requires it
- lsan: requires it
- msan: requires it
- safestack: doesn't require it
- xray: doesn't require it
- tsan: requires it
- ubsan: requires it
- ubsan_minimal: doesn't require it
- scudo: requires it (but not for Fuchsia that has a minimal runtime)
This was tested locally on Linux, Android, Fuchsia.
Reviewers: alekseyshl, eugenis, dberris, kubamracek, vitalybuka, dvyukov, mcgrathr
Reviewed By: alekseyshl, vitalybuka
Subscribers: srhines, kubamracek, mgorny, krytarowski, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D45457
llvm-svn: 330131
The current implementation of the Go sanitizer only works on x86_64.
Added some modifications to the buildgo.sh script and the Tsan code
to make it work on powerpc64/linux.
Author: cseo (Carlos Eduardo Seo)
Reviewed in: https://reviews.llvm.org/D43025
llvm-svn: 330122
Objective-C's @synchronize synchronization primitive uses calls to objc_sync_enter and objc_sync_exit runtime functions. In most cases, they end up just calling pthread_mutex_lock/pthread_mutex_unlock, but there are some cases where the synchronization from pthread_mutex_lock/pthread_mutex_unlock interceptors isn't enough. Let's add explicit interceptors for objc_sync_enter and objc_sync_exit to handle all cases.
Differential Revision: https://reviews.llvm.org/D45487
llvm-svn: 329982
Summary:
`Symbolizer::PrepareForSandboxing` is empty for all platforms and apparently
has been for a while (D10213). Remove it, and shuffle things around so that the
platform specific code is now in `PlatformPrepareForSandboxing`.
This allows to have one less symbolizer dependency in a common file, which
helps for the upcoming split.
Also remove `SymbolizerPrepareForSandboxing` in tsan_go which appears to not
be used anywhere.
Reviewers: alekseyshl, eugenis, dvyukov, mcgrathr
Reviewed By: alekseyshl
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D44953
llvm-svn: 329094
Summary:
`sanitizer_common`'s coverage support is fairly well separated, and libcdep by
default. Several sanitizers don't make use of coverage, and as far as I can
tell do no benefit from the extra dependencies pulled in by the coverage public
interface functions.
The following sanitizers call `InitializeCoverage` explicitely: MSan, ASan,
LSan, HWAsan, UBSan. On top of this, any sanitizer bundling RTUBSan should
add the coverage RT as well: ASan, Scudo, UBSan, CFI (diag), TSan, MSan, HWAsan.
So in the end the following have no need: DFSan, ESan, CFI, SafeStack (nolibc
anyway), XRay, and the upcoming Scudo minimal runtime.
I tested this with all the sanitizers check-* with gcc & clang, and in
standalone on Linux & Android, and there was no issue. I couldn't test this on
Mac, Fuchsia, BSDs, & Windows for lack of an environment, so adding a bunch of
people for additional scrunity. I couldn't test HWAsan either.
Reviewers: eugenis, vitalybuka, alekseyshl, flowerhack, kubamracek, dberris, rnk, krytarowski
Reviewed By: vitalybuka, alekseyshl, flowerhack, dberris
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44701
llvm-svn: 328204
New API passes a callback function to the external symbolizer,
allowing it to add multiple frames to the traceback. Note that
the old interface API will be still supported until the clients
migrate to the new one.
Author: asmundak (Alexander Smundak)
Reviewed in: https://reviews.llvm.org/D44714
llvm-svn: 328079
This commit breaks actual Go runtime build on gomote builders (10.12) with:
xcode-select: error: tool 'xcodebuild' requires Xcode,
but active developer directory '/Library/Developer/CommandLineTools'
is a command line tools instance
Without this part build works fine.
The original commit does not include any explanation as to why
it is needed.
llvm-svn: 327700
Summary:
Without this diff, the test segfaults. Examining the generated executable
(which gets auto-deleted likely by cmake/ninja) yields this error message:
ThreadSanitizer failed to allocate 0x4000 (16384) bytes at address 1755558480000 (errno: 12)
Note that the address has more than 47 bits, which on amd64 means special
treatment and therefore points out an overflow. The allocation came from
__tsan_map_shadow on a .data pointer, which (on my work Debian-based box)
means the 0x550000000000 range. This doesn't correspond to the constants
mentioned in tsan_platform.h for Go binaries on Linux/amd64.
The diff therefore allocates memory in the sort of area Go programs would,
and prevents the test from crashing. It would be nice if reviewers kindly
considered other setups and architectures :-)
Reviewers: kcc, dvyukov
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44071
llvm-svn: 327621
Summary:
Add more standard compliant posix_memalign implementation for LSan and
use corresponding sanitizer's posix_memalign implenetations in allocation
wrappers on Mac.
Reviewers: eugenis, fjricci
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44335
llvm-svn: 327338
This changes the add_custom_libcxx macro to resemble the
llvm_ExternalProject_Add. The primary motivation is to avoid
unnecessary libFuzzer rebuilds that are being done on every
Ninja/Make invocation. The libc++ should be only rebuilt whenever
the libc++ source itself changes.
Differential Revision: https://reviews.llvm.org/D43213
llvm-svn: 326921
NetBSD ships now with netbsd_syscall_hooks.h and requires support
for TSan specific features to be enabled.
This is follow up of:
D42048: Add NetBSD syscall hooks skeleton in sanitizers
Sponsored by <The NetBSD Foundation>
llvm-svn: 325245
Summary:
Implement the skeleton of NetBSD syscall hooks for use with sanitizers.
Add a script that generates the rules to handle syscalls
on NetBSD: generate_netbsd_syscalls.awk. It has been written
in NetBSD awk(1) (patched nawk) and is compatible with gawk.
Generate lib/sanitizer_common/sanitizer_platform_limits_netbsd.h
that is a public header for applications, and included as:
<sanitizer_common/sanitizer_platform_limits_netbsd.h>.
Generate sanitizer_syscalls_netbsd.inc that defines all the
syscall rules for NetBSD. This file is modeled after the Linux
specific file: sanitizer_common_syscalls.inc.
Start recognizing NetBSD syscalls with existing sanitizers:
ASan, ESan, HWASan, TSan, MSan.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, vitalybuka, kcc, dvyukov, eugenis
Reviewed By: vitalybuka
Subscribers: hintonda, kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D42048
llvm-svn: 325206
Summary:
Allow for options to be defined at compile time, like is already the case for
other sanitizers, via `SCUDO_DEFAULT_OPTIONS`.
Reviewers: alekseyshl, dberris
Reviewed By: alekseyshl, dberris
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D42980
llvm-svn: 324620
add_custom_libcxx uses the just built compiler and installs the
built libc++, e.g. for testing, neither of which is desirable in
case of Fuzzer where the libc++ should be built using the host
compiler and it's only linked into the libFuzzer and should never
be installed. This change introduces additional arguments to
add_custom_libcxx to allow parametrizing its behavior.
Differential Revision: https://reviews.llvm.org/D42330
llvm-svn: 323054
add_custom_libcxx uses the just built compiler and installs the
built libc++, e.g. for testing, neither of which is desirable in
case of Fuzzer where the libc++ should be built using the host
compiler and it's only linked into the libFuzzer and should never
be installed. This change introduces additional arguments to
add_custom_libcxx to allow parametrizing its behavior.
Differential Revision: https://reviews.llvm.org/D42330
llvm-svn: 323032
MemToShadowImpl() maps lower addresses to a memory space out of sanitizers
range. The simplest example is address 0 which is mapped to 0x2000000000
static const uptr kShadowBeg = 0x2400000000ull;
but accessing the address during tsan execution will lead to a segmentation
fault.
This patch expands the range used by the sanitizer and ensures that 1/8 of
the maximum valid address in the virtual address spaces is used for shadow
memory.
Patch by Milos Stojanovic.
Differential Revision: https://reviews.llvm.org/D41777
llvm-svn: 323013
Summary:
Make common allocator agnostic to failure handling modes and move the
decision up to the particular sanitizer's allocator, where the context
is available (call stack, parameters, return nullptr/crash mode etc.)
It simplifies the common allocator and allows the particular sanitizer's
allocator to generate more specific and detailed error reports (which
will be implemented later).
The behavior is largely the same, except one case, the violation of the
common allocator's check for "size + alignment" overflow is now reportied
as OOM instead of "bad request". It feels like a worthy tradeoff and
"size + alignment" is huge in this case anyway (thus, can be interpreted
as not enough memory to satisfy the request). There's also a Report()
statement added there.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D42198
llvm-svn: 322784
This macro is only defined after XCode 8, causing build breakage for
build systems with prior versions. Ignore DISPATCH_NOESCAPE if not
defined.
Differential Revision: https://reviews.llvm.org/D41601
llvm-svn: 321543
Summary:
Providing aligned new/delete implementations to match ASan.
Unlike ASan, MSan and TSan do not perform any additional checks
on overaligned memory, hence no sanitizer specific tests.
Reviewers: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D41532
llvm-svn: 321365
This also slightly refactors the code that's checking the directory
presence which allows eliminating one unnecessary variable.
Differential Revision: https://reviews.llvm.org/D40637
llvm-svn: 320446
Summary:
Switching the rest of intercepted allocs to InternalAlloc (well, except
__libc_memalign) when current thread is 'in_symbolizer'. Symbolizer
might (and does) use allocation functions other than malloc/calloc/realloc.
posix_memalign is the one actually used, others switched just in case
(since the failure is obscure and not obvious to diagnose).
Reviewers: dvyukov
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D40877
llvm-svn: 319929
Summary:
The low-fat STL-like vector container will be reused in MSan.
It is needed to implement an atexit(3) interceptor on NetBSD/amd64 in MSan.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc
Reviewed By: dvyukov
Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40726
llvm-svn: 319650
This renames ASM_TSAN_SYMBOL and ASM_TSAN_SYMBOL_INTERCEPTOR to just ASM_SYMBOL and ASM_SYMBOL_INTERCEPTOR, because they can be useful in more places than just TSan. Also introduce a CMake function to add ASM sources to a target.
Differential Revision: https://reviews.llvm.org/D40143
llvm-svn: 319339
Summary:
This change adds support for the setjmp(3)/longjmp(3)
family of functions on NetBSD.
There are three types of them on NetBSD:
- setjmp(3) / longjmp(3)
- sigsetjmp(3) / sigsetjmp(3)
- _setjmp(3) / _longjmp(3)
Due to historical and compat reasons the symbol
names are mangled:
- setjmp -> __setjmp14
- longjmp -> __longjmp14
- sigsetjmp -> __sigsetjmp14
- siglongjmp -> __siglongjmp14
- _setjmp -> _setjmp
- _longjmp -> _longjmp
This leads to symbol renaming in the existing codebase.
There is no such symbol as __sigsetjmp/__longsetjmp
on NetBSD
Add a comment that GNU-style executable stack
note is not needed on NetBSD. The stack is not
executable without it.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, dvyukov, vitalybuka
Reviewed By: dvyukov
Subscribers: llvm-commits, kubamracek, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40337
llvm-svn: 319189
In more recent Linux kernels with 47 bit VMAs the layout of virtual memory
for powerpc64 changed causing the thread sanitizer to not work properly. This
patch adds support for 47 bit VMA kernels for powerpc64.
(second part)
Tested on several 4.x and 3.x kernel releases.
llvm-svn: 319180
The proper index is 6, not 2.
Patch extracted from https://reviews.llvm.org/D40337
Reviewed and accepted by <dvyukov>.
Sponsored by <The NetBSD Foundation>
llvm-svn: 319163
Summary:
NetBSD uses the __sigaction14 symbol name for historical and compat
reasons for the sigaction(2) function name.
Rename the interceptors and users of sigaction to sigaction_symname
and reuse it in the code base.
This change fixes 4 failing tests in TSan/NetBSD:
- ThreadSanitizer-x86_64 :: signal_errno.cc
- ThreadSanitizer-x86_64 :: signal_malloc.cc
- ThreadSanitizer-x86_64 :: signal_sync2.cc
- ThreadSanitizer-x86_64 :: signal_thread.cc
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, vitalybuka, eugenis, dvyukov, kcc
Reviewed By: dvyukov
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40341
llvm-svn: 319160
Summary:
The pthread_once(3)/NetBSD type is built with the following structure:
struct __pthread_once_st {
pthread_mutex_t pto_mutex;
int pto_done;
};
Set the pto_done position as shifted by __sanitizer::pthread_mutex_t_sz
from the beginning of the pthread_once struct.
This corrects deadlocks when the pthread_once(3) function
is used.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, dvyukov, vitalybuka
Reviewed By: dvyukov
Subscribers: llvm-commits, kubamracek, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40262
llvm-svn: 318742
Summary:
Correct handling of libpthread(3) functions in TSan/NetBSD:
- pthread_cond_init(3),
- pthread_cond_signal(3),
- pthread_cond_broadcast(3),
- pthread_cond_wait(3),
- pthread_cond_destroy(3),
- pthread_mutex_init(3),
- pthread_mutex_destroy(3),
- pthread_mutex_trylock(3),
- pthread_rwlock_init(3),
- pthread_rwlock_destroy(3),
- pthread_rwlock_rdlock(3),
- pthread_rwlock_tryrdlock(3),
- pthread_rwlock_wrlock(3),
- pthread_rwlock_trywrlock(3),
- pthread_rwlock_unlock(3),
- pthread_once(3).
Code out of the libpthread(3) context uses the libc symbols
that are prefixed with __libc_, for example: __libc_cond_init.
This caused that these functions were invisible to sanitizers on NetBSD.
Intercept the libc-specific ones and add them as NetBSD-specific aliases
for the common pthread(3) ones.
NetBSD needs to intercept both functions, as the regularly named ones
are used internally in libpthread(3).
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, dvyukov, vitalybuka
Reviewed By: dvyukov
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40243
llvm-svn: 318673
In more recent Linux kernels with 47 bit VMAs the layout of virtual memory
for powerpc64 changed causing the thread sanitizer to not work properly. This
patch adds support for 47 bit VMA kernels for powerpc64.
Tested on several 4.x and 3.x kernel releases.
llvm-svn: 318044
Building with a new clang produces a bunch of warnings about dropped 'const' and 'volatile' qualifiers on pointers. Let's fix them.
Differential Revision: https://reviews.llvm.org/D39861
llvm-svn: 317929
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.
Changes:
- Split paths of handling intercepted __cxa_atexit() and atexit(3).
This affects all supported Operating Systems.
- Add a local stack-like structure to hold the __cxa_atexit() context.
atexit(3) is documented in the C standard as calling callback from the
earliest to the oldest entry. This path also fixes potential ABI
problem of passing an argument to a function from the atexit(3)
callback mechanism.
- Add new test to ensure LIFO style of atexit(3) callbacks: atexit3.cc
Proposal to change the behavior of __cxa_atexit() in NetBSD has been rejected.
With the above changes TSan/NetBSD with the current tsan_interceptors.cc
can bootstrap into operation.
Sponsored by <The NetBSD Foundation>
Reviewers: vitalybuka, dvyukov, joerg, kcc, eugenis
Reviewed By: dvyukov
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D39619
llvm-svn: 317735
We allow usage of global/per-thread data with non-trivial ctors/dtors
throughout tsan code base by placing all global/per-thread data into
Context/ThreadState and then explicitly constructing them with
placement new. This greatly simplifies code by restricting the
"linker initialized plague" to only these 2 objects.
Do the same for interceptors data.
This allows to use Vector instead of bunch of hand-written code in:
https://reviews.llvm.org/D39619
Reviewed in: https://reviews.llvm.org/D39721
llvm-svn: 317587
Summary: The PARENT_TARGET was correctly set under APPLE but not under linux.
Reviewers: kubamracek, samsonov
Subscribers: dberris, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D39621
llvm-svn: 317391
Summary:
Stop using the Linux solution with pthread_key_create(3).
This approach does not work on NetBSD, because calling
the thread destructor is not the latest operation on a POSIX
thread entity. NetBSD's libpthread still calls at least
pthread_mutex_lock and pthread_mutex_unlock.
Detect _lwp_exit(2) call as it is really the latest operation
called from a detaching POSIX thread.
This resolves one set of crashes observed in
the Thread Sanitizer execution.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, kcc, vitalybuka, dvyukov, eugenis
Reviewed By: vitalybuka
Subscribers: llvm-commits, kubamracek, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D39618
llvm-svn: 317363
Summary:
Changes:
* Add initial msan stub support.
* Handle NetBSD specific pthread_setname_np(3).
* NetBSD supports __attribute__((tls_model("initial-exec"))),
define it in SANITIZER_TLS_INITIAL_EXEC_ATTRIBUTE.
* Add ReExec() specific bits for NetBSD.
* Simplify code and add syscall64 and syscall_ptr for !NetBSD.
* Correct bunch of syscall wrappers for NetBSD.
* Disable test/tsan/map32bit on NetBSD as not applicable.
* Port test/tsan/strerror_r to a POSIX-compliant OSes.
* Disable __libc_stack_end on NetBSD.
* Disable ReadNullSepFileToArray() on NetBSD.
* Define struct_ElfW_Phdr_sz, detected missing symbol by msan.
* Change type of __sanitizer_FILE from void to char. This helps
to reuse this type as an array. Long term it will be properly
implemented along with SANITIZER_HAS_STRUCT_FILE setting to 1.
* Add initial NetBSD support in lib/tsan/go/buildgo.sh.
* Correct referencing stdout and stderr in tsan_interceptors.cc
on NetBSD.
* Document NetBSD x86_64 specific virtual memory layout in
tsan_platform.h.
* Port tests/rtl/tsan_test_util_posix.cc to NetBSD.
* Enable NetBSD tests in test/msan/lit.cfg.
* Enable NetBSD tests in test/tsan/lit.cfg.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, vitalybuka, eugenis, kcc, dvyukov
Reviewed By: dvyukov
Subscribers: #sanitizers, llvm-commits, kubamracek
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D39124
llvm-svn: 316591
C99 technically requires the rest arguments to be used in C variadic macros.
This presents a problem with the macro SCOPED_TSAN_INTERCEPTOR when func
takes no arguments. This happens with the function pause. Like other void
argument functions, we pass in a fake argument to avoid this warning.
Author: Alex Langford (xiaobai)
Reviewed in: https://reviews.llvm.org/D39151
llvm-svn: 316558
Add a new flag, __tsan_mutex_not_static, which has the opposite sense
of __tsan_mutex_linker_init. When the new __tsan_mutex_not_static flag
is passed to __tsan_mutex_destroy, tsan ignores the destruction unless
the mutex was also created with the __tsan_mutex_not_static flag.
This is useful for constructors that otherwise woud set
__tsan_mutex_linker_init but cannot, because they are declared constexpr.
Google has a custom mutex with two constructors, a "linker initialized"
constructor that relies on zero-initialization and sets
__tsan_mutex_linker_init, and a normal one which sets no tsan flags.
The "linker initialized" constructor is morally constexpr, but we can't
declare it constexpr because of the need to call into tsan as a side effect.
With this new flag, the normal c'tor can set __tsan_mutex_not_static,
the "linker initialized" constructor can rely on tsan's lazy initialization,
and __tsan_mutex_destroy can still handle both cases correctly.
Author: Greg Falcon (gfalcon)
Reviewed in: https://reviews.llvm.org/D39095
llvm-svn: 316209
Summary:
This is a new attempt at D38706, which had 2 issues.
The first one was that it broke TSan, because `sanitizer_errno.h` was not
directly included in `tsan_mman.cc`. This fixes the include.
The second one was that it broke the nolibc build, because `__errno_location`
couldn't be found. This adds the new .cc to the libcdep list instead of the
base one.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: kubamracek, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D38743
llvm-svn: 315509
Summary:
This change moves cxx-abi library in asan/ubsan/dd link command line
ahead of other libraries, such as pthread/rt/dl/c/gcc. Given that
cxx-abi may be the full libstdc++/libc++, it makes sense for it to be
ahead of libc and libgcc, at least.
The real motivation is Android, where in the arm32 NDK toolchain
libstdc++.a is actually a linker script that tries to sneak LLVM's
libunwind ahead of libgcc's. Wrong library order breaks unwinding.
Reviewers: srhines, danalbert
Subscribers: aemerson, kubamracek, mgorny, kristof.beyls, llvm-commits
Differential Revision: https://reviews.llvm.org/D38520
llvm-svn: 314948
This commit annotates the block parameters of the following functions
declared in compiler-rt with 'noescape':
- dispatch_sync
- dispatch_barrier_sync
- dispatch_once
- dispatch_apply
This is needed to commit the patch that adds support for 'noescape' in
clang (see https://reviews.llvm.org/D32210) since these functions are
annotated with 'noescape' in the SDK header files.
Differential Revision: https://reviews.llvm.org/D32210
llvm-svn: 313929
Summary:
This test was broken by the tail duplication logic being changed in
r311139, update the test values and add a note about how to properly run
a benchmark to verify that the values are safe to update.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Subscribers: dvyukov, kubamracek
Differential Revision: https://reviews.llvm.org/D36889
llvm-svn: 311189
into a function.
Most CMake configuration under compiler-rt/lib/*/tests have
almost-the-same-but-not-quite functions of the form add_X_[unit]tests
for compiling and running the tests.
Much of the logic is duplicated with minor variations across different
sub-folders.
This can harm productivity for multiple reasons:
For newcomers, resulting CMake files are very large, hard to understand,
and hide the intention of the code.
Changes for enabling certain architectures end up being unnecessarily
large, as they get duplicated across multiple folders.
Adding new sub-projects requires more effort than it should, as a
developer has to again copy-n-paste the configuration, and it's not even
clear from which sub-project it should be copy-n-pasted.
With this change the logic of compile-and-generate-a-set-of-tests is
extracted into a function, which hopefully makes writing and reading
CMake much easier.
Differential Revision: https://reviews.llvm.org/D36116
llvm-svn: 310971
Summary:
`CheckForPvallocOverflow` was introduced with D35818 to detect when pvalloc
would wrap when rounding up to the next multiple of the page size.
Add this check to TSan's pvalloc implementation.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D36245
llvm-svn: 309897
TSan tests on Darwin first link all libraries into a static archive file.
With this change, the linking is done once per all architecture,
and previously the linking step was repeated per each architecture per
each add_tsan_test call.
Furthermore, the code is cleared up.
Differential Revision: https://reviews.llvm.org/D35913
llvm-svn: 309406
Currently there's a large amount of CMake logic duplication for
compiling sanitizer tests.
If we add more sanitizers, the duplication will get even worse.
This change factors out common compilation commands into a macro
available to all sanitizers.
llvm-svn: 309405
This change adds sanitizer support for LLVM's libunwind and libc++abi
as an alternative to libstdc++. This allows using the in tree version
of libunwind and libc++abi which is useful when building a toolchain
for different target.
Differential Revision: https://reviews.llvm.org/D34501
llvm-svn: 309362
This change adds support for compiler-rt builtins as an alternative
compiler runtime to libgcc.
Differential Revision: https://reviews.llvm.org/D35165
llvm-svn: 309361
This change adds sanitizer support for LLVM's libunwind and libc++abi
as an alternative to libstdc++. This allows using the in tree version
of libunwind and libc++abi which is useful when building a toolchain
for different target.
Differential Revision: https://reviews.llvm.org/D34501
llvm-svn: 309074
This change adds support for compiler-rt builtins as an alternative
compiler runtime to libgcc.
Differential Revision: https://reviews.llvm.org/D35165
llvm-svn: 309060
Summary:
Set proper errno code on allocation failures and change realloc, pvalloc,
aligned_alloc, memalign and posix_memalign implementation to satisfy
their man-specified requirements.
Modify allocator API implementation to bring it closer to other
sanitizers allocators.
Reviewers: dvyukov
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D35690
llvm-svn: 308929
Summary:
This is a pure refactoring change. It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}. This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.
Submitted on behalf of Roland McGrath.
Reviewers: kcc, eugenis, alekseyshl
Reviewed By: alekseyshl
Subscribers: vitalybuka, llvm-commits, kubamracek, mgorny, phosek
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D35591
llvm-svn: 308819
This is a pure refactoring change. It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}. This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.
Commiting for mcgrathr.
Reviewers: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35591
llvm-svn: 308640
Summary:
ASan/MSan/LSan allocators set errno on allocation failures according to
malloc/calloc/etc. expected behavior.
MSan allocator was refactored a bit to make its structure more similar
with other allocators.
Also switch Scudo allocator to the internal errno definitions.
TSan allocator changes will follow.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D35275
llvm-svn: 308344
This change implements 2 optimizations of sync clocks that reduce memory consumption:
Use previously unused first level block space to store clock elements.
Currently a clock for 100 threads consumes 3 512-byte blocks:
2 64-bit second level blocks to store clock elements
+1 32-bit first level block to store indices to second level blocks
Only 8 bytes of the first level block are actually used.
With this change such clock consumes only 2 blocks.
Share similar clocks differing only by a single clock entry for the current thread.
When a thread does several release operations on fresh sync objects without intervening
acquire operations in between (e.g. initialization of several fields in ctor),
the resulting clocks differ only by a single entry for the current thread.
This change reuses a single clock for such release operations. The current thread time
(which is different for different clocks) is stored in dirty entries.
We are experiencing issues with a large program that eats all 64M clock blocks
(32GB of non-flushable memory) and crashes with dense allocator overflow.
Max number of threads in the program is ~170 which is currently quite unfortunate
(consume 4 blocks per clock). Currently it crashes after consuming 60+ GB of memory.
The first optimization brings clock block consumption down to ~40M and
allows the program to work. The second optimization further reduces block consumption
to "modest" 16M blocks (~8GB of RAM) and reduces overall RAM consumption to ~30GB.
Measurements on another real world C++ RPC benchmark show RSS reduction
from 3.491G to 3.186G and a modest speedup of ~5%.
Go parallel client/server HTTP benchmark:
https://github.com/golang/benchmarks/blob/master/http/http.go
shows RSS reduction from 320MB to 240MB and a few percent speedup.
Reviewed in https://reviews.llvm.org/D35323
llvm-svn: 308018
Summary:
libsanitizer doesn't build against latest glibc anymore, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81066 for details.
One of the changes is that stack_t changed from typedef struct sigaltstack { ... } stack_t; to typedef struct { ... } stack_t; for conformance reasons.
And the other change is that the glibc internal __need_res_state macro is now ignored, so when doing
```
#define __need_res_state
#include <resolv.h>
```
the effect is now the same as just
```
#include <resolv.h>
```
and thus one doesn't get just the
```
struct __res_state { ... };
```
definition, but newly also the
```
extern struct __res_state *__res_state(void) __attribute__ ((__const__));
```
prototype. So __res_state is no longer a type, but a function.
Reviewers: kcc, ygribov
Reviewed By: kcc
Subscribers: kubamracek
Differential Revision: https://reviews.llvm.org/D35246
llvm-svn: 307969
1. Add SyncClock::ResetImpl which removes code
duplication between ctor and Reset.
2. Move SyncClock::Resize to SyncClock methods,
currently it's defined between ThreadClock methods.
llvm-svn: 307785
Don't create sync object if it does not exist yet. For example, an atomic
pointer is initialized to nullptr and then periodically acquire-loaded.
llvm-svn: 307778
r307338 enabled new optimization reducing number of operation in tested functions.
There is no any performance regression detectable with TsanRtlTest DISABLED_BENCH.Mop* tests.
llvm-svn: 307739
Summary:
This is the first in a series of patches to refactor sanitizer_procmaps
to allow MachO section information to be exposed on darwin.
In addition, grouping all segment information in a single struct is
cleaner than passing it through a large set of output parameters, and
avoids the need for annotations of NULL parameters for unneeded
information.
The filename string is optional and must be managed and supplied by the
calling function. This is to allow the MemoryMappedSegment struct to be
stored on the stack without causing overly large stack sizes.
Reviewers: alekseyshl, kubamracek, glider
Subscribers: emaste, llvm-commits
Differential Revision: https://reviews.llvm.org/D35135
llvm-svn: 307688
This patch ports the assembly file implementing TSan's setjmp support to AArch64 on Darwin.
Differential Revision: https://reviews.llvm.org/D35143
llvm-svn: 307541
Summary:
Operator new interceptors behavior is now controlled by their nothrow
property as well as by allocator_may_return_null flag value:
- allocator_may_return_null=* + new() - die on allocation error
- allocator_may_return_null=0 + new(nothrow) - die on allocation error
- allocator_may_return_null=1 + new(nothrow) - return null
Ideally new() should throw std::bad_alloc exception, but that is not
trivial to achieve, hence TODO.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34731
llvm-svn: 306604
Summary:
Move cached allocator_may_return_null flag to sanitizer_allocator.cc and
provide API to consolidate and unify the behavior of all specific allocators.
Make all sanitizers using CombinedAllocator to follow
AllocatorReturnNullOrDieOnOOM() rules to behave the same way when OOM
happens.
When OOM happens, turn allocator_out_of_memory flag on regardless of
allocator_may_return_null flag value (it used to not to be set when
allocator_may_return_null == true).
release_to_os_interval_ms and rss_limit_exceeded will likely be moved to
sanitizer_allocator.cc too (later).
Reviewers: eugenis
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34310
llvm-svn: 305858
GNU version of strerror_r returns a result pointer that doesn't match the input
buffer. The result pointer is in fact a pointer to some internal storage.
TSAN was recording a write to this location, which was incorrect.
Fixed https://github.com/google/sanitizers/issues/696
llvm-svn: 304858
The existing implementation ran CHECKs to assert that the thread state
was stored inside the tls. However, the mac implementation of tsan doesn't
store the thread state in tls, so these checks fail once darwin tls support
is added to the sanitizers. Only run these checks on platforms where
the thread state is expected to be contained in the tls.
llvm-svn: 303886
Summary:
With rL279771, SizeClassAllocator64 was changed to accept only one template
instead of 5, for the following reasons: "First, this will make the mangled
names shorter. Second, this will make adding more parameters simpler". This
patch mirrors that work for SizeClassAllocator32.
This is in preparation for introducing the randomization of chunks in the
32-bit SizeClassAllocator in a later patch.
Reviewers: kcc, alekseyshl, dvyukov
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D33141
llvm-svn: 303071
This patch allows the Swift compiler to emit calls to `__tsan_external_write` before starting any modifying access, which will cause TSan to detect races on arrays, dictionaries and other classes defined in non-instrumented modules. Races on collections from the Swift standard library and user-defined structs and a frequent cause of subtle bugs and it's important that TSan detects those on top of existing LLVM IR instrumentation, which already detects races in direct memory accesses.
Differential Revision: https://reviews.llvm.org/D31630
llvm-svn: 302050
The fast reset for large memory regions is not working
only on windows. So enable it for Go/linux/darwin/freebsd.
See https://github.com/golang/go/issues/20139
for background and motivation.
Based on idea by Josh Bleecher Snyder.
llvm-svn: 301927
Summary:
TSan's Android `__get_tls()` and `TLS_SLOT_TSAN` can be used by other sanitizers as well (see D32649), this change moves them to sanitizer_common.
I picked sanitizer_linux.h as their new home.
In the process, add the 32-bit versions for ARM, i386 & MIPS.
Can the address of `__get_tls()[TLS_SLOT_TSAN]` change in between the calls?
I am not sure if there is a need to repeat the construct as opposed to using a variable. So I left things as they were.
Testing on my side was restricted to a successful cross-compilation.
Reviewers: dvyukov, kubamracek
Reviewed By: dvyukov
Subscribers: aemerson, rengolin, srhines, dberris, arichardson, llvm-commits
Differential Revision: https://reviews.llvm.org/D32705
llvm-svn: 301926
These test cases occassionally fail when run on powerpc64le:
ignore_lib1.cc
ignore_lib5.cc
TestCases/Posix/current_allocated_bytes.cc
rtl/TsanRtlTest/Posix.ThreadLocalAccesses
TestCases/Posix/coverage-fork-direct.cc
The failures cause false problem reports to be sent to developers whose
code had nothing to do with the failures. Reactivate them when the real
problems are fixed.
This could also be related to the same problems as with the tests
ThreadedOneSizeMallocStressTest, ThreadedMallocStressTest, ManyThreadsTest,
and several others that do not run reliably on powerpc.
llvm-svn: 301798
For a linker init mutex with lazy flag setup
(no __tsan_mutex_create call), it is possible that
no lock/unlock happened before the destroy call.
Then when destroy runs we still don't know that
it is a linker init mutex and will emulate a memory write.
This in turn can lead to false positives as the mutex
is in fact linker initialized.
Support linker init flag in destroy annotation to resolve this.
llvm-svn: 301795
To make the TSan external API work with Swift and other use cases, we need to track "tags" for individual memory accesses. Since there is no space to store this information in shadow cells, let's use the thread traces for that. This patch stores the tag as an extra frame in the stack traces (by calling FuncEntry and FuncExit with the address of a registered tag), this extra frame is then stripped before printing the backtrace to stderr.
Differential Revision: https://reviews.llvm.org/D32382
llvm-svn: 301777
On Darwin, the setting ignore_noninstrumented_modules is used to suppress false positives in code that users don't have control of. The recently added "external" API (which can be used to detect races on objects provided by system libraries, but the race is actually user's fault) ignores this flag and it can report issues in non-instrumented modules. This patch fixes that.
Differential Revision: https://reviews.llvm.org/D31553
llvm-svn: 301000
This patch make sure we don't report deadlocks and other bug types when we're inside an interceptor that was called from a noninstrumented module (when ignore_noninstrumented_modules=1 is set). Adding a testcase that shows that deadlock detection still works on Darwin (to make sure we're not silencing too many reports).
Differential Revision: https://reviews.llvm.org/D31449
llvm-svn: 300998
We seem to assume that OS-provided thread IDs are either uptr or int, neither of which is true on Darwin. This introduces a tid_t type, which holds a OS-provided thread ID (gettid on Linux, pthread_threadid_np on Darwin, pthread_self on FreeBSD).
Differential Revision: https://reviews.llvm.org/D31774
llvm-svn: 300473
Summary:
Lsan was using PTHREAD_CREATE_JOINABLE/PTHREAD_CREATE_DETACHED
as truthy values, which works on Linux, where the values are 0 and 1,
but this fails on OS X, where the values are 1 and 2.
Set PTHREAD_CREATE_DETACHED to the correct value for a given system.
Reviewers: kcc, glider, kubamracek, alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31883
llvm-svn: 300221
r299658 fixed a case where InstCombine was replicating instructions instead of combining. Fixing this reduced the number of pushes and pops in the __tsan_read and __tsan_write functions.
Adjust the expectations to account for this after talking to Dmitry Vyukov.
llvm-svn: 299661
TSan reports a false positive when using xpc_connection_cancel. We're missing a happens-before edge from xpc_connection_cancel to the event handler on the same connection.
Differential Revision: https://reviews.llvm.org/D31475
llvm-svn: 299086
{M, T, E}San have fread and fwrite interceptors, let's move them to sanitizer_common to enable ASan checks as well.
Differential Revision: https://reviews.llvm.org/D31456
llvm-svn: 299061
While it's usually a bug to call GCD APIs, such as dispatch_after, with NULL as a queue, this often "somehow" works and TSan should maintain binary compatibility with existing code. This patch makes sure we don't try to call Acquire and Release on NULL queues, and add one such testcase for dispatch_after.
Differential Revision: https://reviews.llvm.org/D31355
llvm-svn: 298820
There are several problems with the current annotations (AnnotateRWLockCreate and friends):
- they don't fully support deadlock detection (we need a hook _before_ mutex lock)
- they don't support insertion of random artificial delays to perturb execution (again we need a hook _before_ mutex lock)
- they don't support setting extended mutex attributes like read/write reentrancy (only "linker init" was bolted on)
- they don't support setting mutex attributes if a mutex don't have a "constructor" (e.g. static, Java, Go mutexes)
- they don't ignore synchronization inside of lock/unlock operations which leads to slowdown and false negatives
The new annotations solve of the above problems. See tsan_interface.h for the interface specification and comments.
Reviewed in https://reviews.llvm.org/D31093
llvm-svn: 298809
HLE flags can be combined with memory order in atomic operations.
Currently tsan runtime crashes on e.g. IsStoreOrder(mo) in atomic store
if any of these additional flags are specified.
Filter these flags out.
See the comment as to why it is safe.
llvm-svn: 298378
Kamil Rytarowski