When we delay signals we can deliver them when the signal
is blocked. This can be surprising to the program.
Intercept signal blocking functions merely to process
pending signals. As the result, at worst we will delay
a signal till return from the signal blocking function.
llvm-svn: 276876
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.
Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.
This patch doesn't contain behavior changes.
Reviewers: kubabrecka, rnk
Subscribers: wang0109, llvm-commits, kubabrecka, chrisha
Differential Revision: http://reviews.llvm.org/D21952
llvm-svn: 275111
This patch adds interceptors for dispatch_io_*, dispatch_read and dispatch_write functions. This avoids false positives when using GCD IO. Adding several test cases.
Differential Revision: http://reviews.llvm.org/D21889
llvm-svn: 275071
These test in this change are objc++, but are built using %clang, not %clangxx.
The reason this works is the driver has been adding -lc++ for sanitizer enabled
builds. By making these tests use %clangxx, they no longer depend on the driver
linking to c++. Doing so will allow us to prevent overlinking of libc++ for
applications.
llvm-svn: 274989
This patch adds synchronization between the creation of the GCD data object and destructor’s execution. It’s far from perfect, because ideally we’d want to synchronize the destruction of the last reference (via dispatch_release) and the destructor’s execution, but intercepting objc_release is problematic.
Differential Revision: http://reviews.llvm.org/D21990
llvm-svn: 274749
We already have interceptors for dispatch_source API (e.g. dispatch_source_set_event_handler), but they currently only handle submission synchronization. We also need to synchronize based on the target queue (serial, concurrent), in other words, we need to use dispatch_callback_wrap. This patch implements that.
Differential Revision: http://reviews.llvm.org/D21999
llvm-svn: 274619
In the patch that introduced support for GCD barrier blocks, I removed releasing a group when leaving it (in dispatch_group_leave). However, this is necessary to synchronize leaving a group and a notification callback (dispatch_group_notify). Adding this back, simplifying dispatch_group_notify_f and adding a test case.
Differential Revision: http://reviews.llvm.org/D21927
llvm-svn: 274549
Because we use SCOPED_TSAN_INTERCEPTOR in the dispatch_once interceptor, the original dispatch_once can also be sometimes called (when ignores are enabled or when thr->is_inited is false). However the original dispatch_once function doesn’t expect to find “2” in the storage and it will spin forever (but we use “2” to indicate that the initialization is already done, so no waiting is necessary). This patch makes sure we never call the original dispatch_once.
Differential Revision: http://reviews.llvm.org/D21976
llvm-svn: 274548
The dispatch_group_async interceptor actually extends the lifetime of the executed block. This means the destructor of the block (and captured variables) is called *after* dispatch_group_leave, which changes the semantics of dispatch_group_async. This patch fixes that.
Differential Revision: http://reviews.llvm.org/D21816
llvm-svn: 274117
Adding support for GCD barrier blocks in concurrent queues. This uses two sync object in the same way as read-write locks do. This also simplifies the use of dispatch groups (the notifications act as barrier blocks).
Differential Revision: http://reviews.llvm.org/D21604
llvm-svn: 273893
There is a "well-known" TSan false positive when using C++ weak_ptr/shared_ptr and code in destructors, e.g. described at <https://llvm.org/bugs/show_bug.cgi?id=22324>. The "standard" solution is to build and use a TSan-instrumented version of libcxx, which is not trivial for end-users. This patch tries a different approach (on OS X): It adds an interceptor for the specific function in libc++.dylib, which implements the atomic operation that needs to be visible to TSan.
Differential Revision: http://reviews.llvm.org/D21609
llvm-svn: 273806
The new annotation was added a while ago, but was not actually used.
Use the annotation to detect linker-initialized mutexes instead
of the broken IsGlobalVar which has both false positives and false
negatives. Remove IsGlobalVar mess.
llvm-svn: 271663
Currently the added test produces false race reports with glibc 2.19,
because DLTS memory is reused by pthread under the hood.
Use the DTLS machinery to intercept new DTLS ranges.
__tls_get_addr known to cause issues for tsan in the past,
so write the interceptor more carefully.
Reviewed in http://reviews.llvm.org/D20927
llvm-svn: 271568
In one of the already existing apps that I'm testing TSan on, I really see a mutex path that is longer than 10 (but not by much, something like 11-13 actually). Let's raise this to 20 and weaken the assertion so we don't crash.
Differential Revision: http://reviews.llvm.org/D20427
llvm-svn: 270319
We're missing interceptors for dispatch_after and dispatch_after_f. Let's add them to avoid false positives. Added a test case.
Differential Revision: http://reviews.llvm.org/D20426
llvm-svn: 270071
The ignore_interceptors_accesses setting did not have an effect on mmap, so
let's change that. It helps in cases user code is accessing the memory
written to by mmap when the synchronization is ensured by the code that
does not get rebuilt.
(This effects Swift interoperability since it's runtime is mapping memory
which gets accessed by the code emitted into the Swift application by the
compiler.)
Differential Revision: http://reviews.llvm.org/D20294
llvm-svn: 269855
Another stack where we try to free sync objects,
but don't have a processors is:
// ResetRange
// __interceptor_munmap
// __deallocate_stack
// start_thread
// clone
Again, it is a latent bug that lead to memory leaks.
Also, increase amount of memory we scan in MetaMap::ResetRange.
Without that the test does not fail, as we fail to free
the sync objects on stack.
llvm-svn: 269041
Fixes crash reported in:
https://bugs.chromium.org/p/v8/issues/detail?id=4995
The problem is that we don't have a processor in a free interceptor
during thread exit.
The crash was introduced by introduction of Processors.
However, previously we silently leaked memory which wasn't any better.
llvm-svn: 268782
In http://reviews.llvm.org/D19100, I introduced a bug: On OS X, existing programs rely on malloc_size() to detect whether a pointer comes from heap memory (malloc_size returns non-zero) or not. We have to distinguish between a zero-sized allocation (where we need to return 1 from malloc_size, due to other binary compatibility reasons, see http://reviews.llvm.org/D19100), and pointers that are not returned from malloc at all.
Differential Revision: http://reviews.llvm.org/D19653
llvm-svn: 268157
The field "pid" in ReportThread is used to store the OS-provided thread ID (pthread_self or gettid). The name "pid" suggests it's a process ID, which it isn't. Let's rename it.
Differential Revision: http://reviews.llvm.org/D19365
llvm-svn: 266994
At the moment almost every lit.site.cfg.in contains two lines comment:
## Autogenerated by LLVM/Clang configuration.
# Do not edit!
The patch adds variable LIT_SITE_CFG_IN_HEADER, that is replaced from
configure_lit_site_cfg with the note and some useful information.
llvm-svn: 266520
The custom zone implementation for OS X must not return 0 (even for 0-sized allocations). Returning 0 indicates that the pointer doesn't belong to the zone. This can break existing applications. The underlaying allocator allocates 1 byte for 0-sized allocations anyway, so returning 1 in this case is okay.
Differential Revision: http://reviews.llvm.org/D19100
llvm-svn: 266283
On one of our testing machines, we're running the tests under heavy load, and especially in the fork-based TSan tests, we're seeing timeouts when a test uses sleep(10), assuming that calling fork() on another thread will finish sooner than that. This patch removes a timeout and makes another one longer.
Differential Revision: http://reviews.llvm.org/D18476
llvm-svn: 265666
OS X provides atomic functions in libkern/OSAtomic.h. These provide atomic guarantees and they have alternatives which have barrier semantics. This patch adds proper TSan support for the functions from libkern/OSAtomic.h.
Differential Revision: http://reviews.llvm.org/D18500
llvm-svn: 265665
Adding an interceptor with two more release+acquire pairs to avoid false positives with dispatch_apply.
Differential Revision: http://reviews.llvm.org/D18722
llvm-svn: 265662
XPC APIs have async callbacks, and we need some more happen-before edges to avoid false positives. This patch add them, plus a test case (sorry for the long boilerplate code, but XPC just needs all that).
Differential Revision: http://reviews.llvm.org/D18493
llvm-svn: 265661
GCD has APIs for event sources, we need some more release-acquire pairs to avoid false positives in TSan.
Differential Revision: http://reviews.llvm.org/D18515
llvm-svn: 265660
In the interceptor for dispatch_sync, we're currently missing synchronization between the callback and the code *after* the call to dispatch_sync. This patch fixes this by adding an extra release+acquire pair to dispatch_sync() and similar APIs. Added a testcase.
Differential Revision: http://reviews.llvm.org/D18502
llvm-svn: 265659
A little embarrassing, but we're missing the call to FileCheck in several Darwin tests. Let's fix this.
Differential Revision: http://reviews.llvm.org/D18503
llvm-svn: 265658
This patch fixes the custom ThreadState destruction on OS X to avoid crashing when dispatch_main calls pthread_exit which quits the main thread.
Differential Revision: http://reviews.llvm.org/D18496
llvm-svn: 264627
On OS X, fork() under TSan asserts (in debug builds only) because REAL(fork) calls some intercepted functions, which check that no internal locks are held via CheckNoLocks(). But the wrapper of fork intentionally holds some locks. This patch fixes that by using ScopedIgnoreInterceptors during the call to REAL(fork). After that, all the fork-based tests seem to pass on OS X, so let's just remove all the UNSUPPORTED: darwin annotations we have.
Differential Revision: http://reviews.llvm.org/D18409
llvm-svn: 264261
On OS X, internal_mmap just uses mmap, which can invoke callbacks into libmalloc (e.g. when MallocStackLogging is enabled). This can subsequently call other intercepted functions, and this breaks our Darwin-specific ThreadState initialization. Let's use direct syscalls in internal_mmap and internal_munmap. Added a testcase.
Differential Revision: http://reviews.llvm.org/D18431
llvm-svn: 264259
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.
TSan unit tests run with a statically linked runtime, where interceptors don't work. To avoid aborting the process in this case, the patch replaces `DisableReexec()` with a weak `ReexecDisabled()` function which is defined to return true in unit tests.
Differential Revision: http://reviews.llvm.org/D18212
llvm-svn: 263695
This patch adds a new TSan report type, ReportTypeMutexInvalidAccess, which is triggered when pthread_mutex_lock or pthread_mutex_unlock returns EINVAL (this means the mutex is invalid, uninitialized or already destroyed).
Differential Revision: http://reviews.llvm.org/D18132
llvm-svn: 263641
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.
Differential Revision: http://reviews.llvm.org/D18121
llvm-svn: 263551
Currently, TSan only reports everything in a formatted textual form. The idea behind this patch is to provide a consistent API that can be used to query information contained in a TSan-produced report. User can use these APIs either in a debugger (via a script or directly), or they can use it directly from the process (e.g. in the __tsan_on_report callback). ASan already has a similar API, see http://reviews.llvm.org/D4466.
Differential Revision: http://reviews.llvm.org/D16191
llvm-svn: 263126
Incremented the pc for each architecture in accordance with StackTrace:GetPreviousInstructionPC
Reviewers: samsonov, dvyukov
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential: http://reviews.llvm.org/D17802
llvm-svn: 262483
This test expects pthread_mutex_init in the frame #0 of thread T1 but we
get memset at frame #0 because memset that is called from pthread_init_mutex
is being intercepted by TSan
llvm-svn: 261986
The first issue is that we longjmp from ScopedInterceptor scope
when called from an ignored lib. This leaves thr->in_ignored_lib set.
This, in turn, disables handling of sigaction. This, in turn,
corrupts tsan state since signals delivered asynchronously.
Another issue is that we can ignore synchronization in asignal
handler, if the signal is delivered into an IgnoreSync region.
Since signals are generally asynchronous, they should ignore
memory access/synchronization/interceptor ignores.
This could lead to false positives in signal handlers.
llvm-svn: 261658
1. Add two explicit -stdlib=libstdc++ in conjunction with -static-libstdc++
2. Pass -nostdinc++ when adding include paths for libc++ built for tsan. This
prevents clang finding the headers twice which would confuse #include_next
Differential Revision: http://reviews.llvm.org/D17189
llvm-svn: 260883
There's no obvious reason it should fail in this way but it's the only change
on the blamelist. I suspect stale lit*.cfg's from previous builds.
llvm-svn: 260672
The lit test-suite containing the unit tests needs to be explicitly specified
as an argument to lit.py since it is no longer discovered when the other tests
are run (because they are one directory deeper).
dfsan, lsan, and sanitizer_common don't show the same problem.
llvm-svn: 260669
Summary:
This is a workaround to a problem in the 3.8 release that affects MIPS and
possibly other targets where the default is not supported but a sibling is
supported.
When TSAN_SUPPORTED_ARCH is not empty, cmake currently attempts to build a
tsan'd libcxx as well as test tsan for the default target regardless of whether
the default target is supported or not. This causes problems on MIPS32 since
tsan is supported for MIPS64 but not MIPS32.
This patch causes cmake to only build the libcxx and run the lit test-suite for
archictures in ${TSAN_SUPPORTED_ARCH}
This re-commit fixes an issue where 'check-tsan' continued to look for the
tsan'd libc++ in the directory it used to be built in.
Reviewers: hans, samsonov
Subscribers: tberghammer, llvm-commits, danalbert, srhines, dvyukov
Differential Revision: http://reviews.llvm.org/D16685
llvm-svn: 259542
check-tsan does not pick up the correct libc++.so. It succeeded on my machine
by picking up the libc++.so that was built before making this change.
llvm-svn: 259519
Summary:
This is a workaround to a problem in the 3.8 release that affects MIPS and
possibly other targets where the default is not supported but a sibling is
supported.
When TSAN_SUPPORTED_ARCH is not empty, cmake currently attempts to build a
tsan'd libcxx as well as test tsan for the default target regardless of whether
the default target is supported or not. This causes problems on MIPS32 since
tsan is supported for MIPS64 but not MIPS32.
This patch causes cmake to only build the libcxx and run the lit test-suite for
archictures in ${TSAN_SUPPORTED_ARCH}
Reviewers: hans, samsonov
Subscribers: tberghammer, llvm-commits, danalbert, srhines, dvyukov
Differential Revision: http://reviews.llvm.org/D16685
llvm-svn: 259512
On OS X, TSan already passes all unit and lit tests, but for real-world applications (even very simple ones), we currently produce a lot of false positive reports about data races. This makes TSan useless at this point, because the noise dominates real bugs. This introduces a runtime flag, "ignore_interceptors_accesses", off by default, which makes TSan ignore all memory accesses that happen from interceptors. This will significantly lower the coverage and miss a lot of bugs, but it eliminates most of the current false positives on OS X.
Differential Revision: http://reviews.llvm.org/D15189
llvm-svn: 257760
This patch adds PIE executable support for aarch64-linux. It adds
two more segments:
- 0x05500000000-0x05600000000: 39-bits PIE program segments
- 0x2aa00000000-0x2ab00000000: 42-bits PIE program segments
Fortunately it is possible to use the same transformation formula for
the new segments range with some adjustments in shadow to memory
formula (it adds a constant offset based on the VMA size).
A simple testcase is also added, however it is disabled on x86 due the
fact it might fail on newer kernels [1].
[1] https://git.kernel.org/linus/d1fd836dcf00d2028c700c7e44d2c23404062c90
llvm-svn: 256184
We're using the dispatch group itself to synchronize (to call Release() and Acquire() on it), but in dispatch group notifications, the group can already be disposed/deallocated. This causes a later assertion failure at `DCHECK_EQ(*meta, 0);` in `MetaMap::AllocBlock` when the same memory is reused (note that the failure only happens in debug builds).
Fixing this by retaining the group and releasing it in the notification. Adding a stress test case that reproduces this.
Differential Revision: http://reviews.llvm.org/D15380
llvm-svn: 255494
This patch is by Simone Atzeni with portions by Adhemerval Zanella.
This contains the LLVM patches to enable the thread sanitizer for
PPC64, both big- and little-endian. Two different virtual memory
sizes are supported: Old kernels use a 44-bit address space, while
newer kernels require a 46-bit address space.
There are two companion patches that will be added shortly. There is
a Clang patch to actually turn on the use of the thread sanitizer for
PPC64. There is also a patch that I wrote to provide interceptor
support for setjmp/longjmp on PPC64.
Patch discussion at reviews.llvm.org/D12841.
llvm-svn: 255057
Another attempt at fixing tsan_invisible_barrier.
Current implementation causes:
https://llvm.org/bugs/show_bug.cgi?id=25643
There were several unsuccessful iterations for this functionality:
Initially it was implemented in user code using REAL(pthread_barrier_wait). But pthread_barrier_wait is not supported on MacOS. Futexes are linux-specific for this matter.
Then we switched to atomics+usleep(10). But usleep produced parasitic "as-if synchronized via sleep" messages in reports which failed some output tests.
Then we switched to atomics+sched_yield. But this produced tons of tsan- visible events, which lead to "failed to restore stack trace" failures.
Move implementation into runtime and use internal_sched_yield in the wait loop.
This way tsan should see no events from the barrier, so not trace overflows and
no "as-if synchronized via sleep" messages.
llvm-svn: 255030
This patch adds release and acquire semantics for dispatch groups, plus a test case.
Differential Revision: http://reviews.llvm.org/D15048
llvm-svn: 255020
Check that TSan runtime doesn't contain compiler-inserted calls
to memset/memmove functions.
In future, we may consider moving this test to test/sanitizer_common,
as we don't want to have compiler-inserted memcpy/memmove calls in
any sanitizer runtime.
llvm-svn: 254955
This script is superseded by lit test suite integrated into CMake
for quite a while now. It doesn't support many tests, and require
custom hacks for a few other.
llvm-svn: 254932
On OS X, there are other-than-pthread locking APIs that are used quite extensively - OSSpinLock and os_lock_lock. Let's add interceptors for those.
Differential Revision: http://reviews.llvm.org/D14987
llvm-svn: 254611
In mmap_large.cc, let's use MAP_ANON instead of MAP_ANONYMOUS, because MAP_ANONYMOUS is only available on OS X 10.11 and later.
Differential Revision: http://reviews.llvm.org/D15180
llvm-svn: 254601
This patch adds release and acquire semantics for libdispatch semaphores and a test case.
Differential Revision: http://reviews.llvm.org/D14992
llvm-svn: 254412
This patch complete removed SANITIZER_AARCH64_VMA definition and usage.
AArch64 ports now supports runtime VMA detection and instrumentation
for 39 and 42-bit VMA.
It also Rewrite print_address to take a variadic argument list
(the addresses to print) and adjust the tests which uses it to the new
signature.
llvm-svn: 254319
Changing comments that have references to code.google.com to point to GitHub instead, because the current links are not redirected properly (they instead redirect to different issues, mostly ASan). NFC.
Differential Revision: http://reviews.llvm.org/D15053
llvm-svn: 254300
1) There's a few wrongly defined things in tsan_interceptors.cc,
2) a typo in tsan_rtl_amd64.S which calls setjmp instead of sigsetjmp in the interceptor, and
3) on OS X, accessing an mprotected page results in a SIGBUS (and not SIGSEGV).
Differential Revision: http://reviews.llvm.org/D15052
llvm-svn: 254299
Serial queues need extra happens-before between individual tasks executed in the same queue. This patch adds `Acquire(queue)` before the executed task and `Release(queue)` just after it (for serial queues only). Added a test case.
Differential Revision: http://reviews.llvm.org/D15011
llvm-svn: 254229
There's a few more lit tests that require features not available on OS X (MAP_32BIT, pthread_setname_np), let's mark them with "UNSUPPORTED: darwin".
Differential Revision: http://reviews.llvm.org/D14923
llvm-svn: 254225
Pthread spinlocks are not available on OS X and this test doesn't really require a spinlock.
Differential Revision: http://reviews.llvm.org/D14949
llvm-svn: 254224
When a race on file descriptors is detected, `FindThreadByUidLocked()` is called to retrieve ThreadContext with a specific unique_id. However, this ThreadContext might not exist in the thread registry anymore (it may have been recycled), in which case `FindThreadByUidLocked` will cause an assertion failure in `GetThreadLocked`. Adding a test case that reproduces this, producing:
FATAL: ThreadSanitizer CHECK failed: sanitizer_common/sanitizer_thread_registry.h:92 "((tid)) < ((n_contexts_))" (0x34, 0x34)
This patch fixes this by replacing the loop with `FindThreadContextLocked`.
Differential Revision: http://reviews.llvm.org/D14984
llvm-svn: 254223
This patch unify the 39 and 42-bit support for AArch64 by using an external
memory read to check the runtime detected VMA and select the better mapping
and transformation. Although slower, this leads to same instrumented binary
to be independent of the kernel.
Along with this change this patch also fix some 42-bit failures with
ALSR disable by increasing the upper high app memory threshold and also
the 42-bit madvise value for non large page set.
llvm-svn: 254151
We need to intercept libdispatch APIs (dispatch_sync, dispatch_async, etc.) to add synchronization between the code that submits the task and the code that gets executed (possibly on a different thread). This patch adds release+acquire semantics for dispatch_sync, and dispatch_async (plus their "_f" and barrier variants). The synchronization is done on malloc'd contexts (separate for each submitted block/callback). Added tests to show usage of dispatch_sync and dispatch_async, for cases where we expect no warnings and for cases where TSan finds races.
Differential Revision: http://reviews.llvm.org/D14745
llvm-svn: 253982
The test relies on two variables in different frames to end up being on the same address. For some reason, this isn't true on OS X. This patch adds `__attribute__((aligned(64)))` to the variables, which actually makes the variables occupy the same address. This is still not a guarantee, but it's more likely to work (the test looks very fragile already).
Differential Revision: http://reviews.llvm.org/D14925
llvm-svn: 253981
Pthread semaphores are not available on OS X. Let's replace sem_wait/sem_post with barrier_wait, which makes the test pass on OS X. I know that sem_wait/sem_post is intercepted by TSan, whereas barrier_wait is TSan-invisible, but the purpose of the test is not affected by this.
Also, let's properly initialize the mutex and cond variables.
Differential Revision: http://reviews.llvm.org/D14924
llvm-svn: 253980
On OS X, __thread variables are lazily heap-allocated (with malloc). Therefore, they're recognized as heap blocks (which is what they are) and not as TLS variables in TSan reports. Figuring out if a heap block is a TLS or not is difficult (in malloc interceptor we could analyze the caller and then mark the object), so let's instead modify the tests so that we expect the report to say "Location is heap block" instead of "Location is TLS".
Differential Revision: http://reviews.llvm.org/D14873
llvm-svn: 253858
OS X doesn't support POSIX semaphores (but it does have the API for it, which returns ENOSYS - "Function not implemented").
Differential Revision: http://reviews.llvm.org/D14865
llvm-svn: 253665
On OS X, we don't have pthread spinlocks, let's just use a regular mutex instead. Secondly, pthread_rwlock_t is much larger (200 bytes), so `char padding_[64 - sizeof(pthread_rwlock_t)]` actually underflows.
Differential Revision: http://reviews.llvm.org/D14862
llvm-svn: 253659
Several tests rely on CLOCK_MONOTONIC, which doesn't exist on OS X. This patch fixes these tests by either disabling them (in case of cond_version.c which doesn't make sense on OS X), or by porting the test to also work on OS X.
Differential Revision: http://reviews.llvm.org/D14861
llvm-svn: 253658
Several testcases need pthread barriers (e.g. all bench_*.cc which use test/tsan/bench.h) which are not available on OS X. Let's mark them with "UNSUPPORTED: darwin".
Differential Revision: http://reviews.llvm.org/D14636
llvm-svn: 253558
Reimplement dispatch_once in an interceptor to solve these issues that may produce false positives with TSan on OS X:
1) there is a racy load inside an inlined part of dispatch_once,
2) the fast path in dispatch_once doesn't perform an acquire load, so we don't properly synchronize the initialization and subsequent uses of whatever is initialized,
3) dispatch_once is already used in a lot of already-compiled code, so TSan doesn't see the inlined fast-path.
This patch uses a trick to avoid ever taking the fast path (by never storing ~0 into the predicate), which means the interceptor will always be called even from already-compiled code. Within the interceptor, our own atomic reads and writes are not written into shadow cells, so the race in the inlined part is not reported (because the accesses are only loads).
Differential Revision: http://reviews.llvm.org/D14811
llvm-svn: 253552
This patch adds assembly routines to enable setjmp/longjmp for aarch64
on linux. It fixes:
* test/tsan/longjmp2.cc
* test/tsan/longjmp3.cc
* test/tsan/longjmp4.cc
* test/tsan/signal_longjmp.cc
I also checked with perlbench from specpu2006 (it fails to run
with missing setjmp/longjmp intrumentation).
llvm-svn: 253205
I noticed that when a symbol is named just "x", it gets demangled to "long long". On POSIX, AFAIK, mangled names always start with "_Z", so lets just require that.
Differential Revision: http://reviews.llvm.org/D14637
llvm-svn: 253080
On OS X, memcpy and memmove are actually aliases of the same function, so the memmove interceptor can be invoked on a call to memcpy. This patch updates the tests to expect either memmove or memcpy on a stack trace.
Differential Revision: http://reviews.llvm.org/D14638
llvm-svn: 253077
The OS X symbolizers (namely AtosSymbolizer) don't return full file paths, only file names. This patch modifies `mutexset*.cc` tests not to require a path to be present in the symbol on the stack trace.
Differential Revision: http://reviews.llvm.org/D14642
llvm-svn: 253075
This patch adds support for symbolication of globals (implements `SymbolizeData`) for `AtosSymbolizer` on OS X.
Differential Revision: http://reviews.llvm.org/D14618
llvm-svn: 253015
`DlAddrSymbolizer` is used on OS X when we're running inside a sandbox that prevents us from spawning an external symbolizer. This patch adds support for symbolication of globals (implements `SymbolizeData`) for `DlAddrSymbolizer`.
Differential Revision: http://reviews.llvm.org/D14613
llvm-svn: 252899
The default symbolizer, `llvm-symbolizer` provides sizes for global symbols. On OS X, we want to also allow using `atos` (because it's available everywhere and users don't need to copy/install it) and `dladdr` (it's the only available option when running in a sandbox). However, these symbolizers do not supply the symbol sizes, only names and starting addresses. This patch changes the reporting functions to hide the size of the symbol when this value is unavailable, and modifies tests to make this part of the report "optional".
Differential Revision: http://reviews.llvm.org/D14608
llvm-svn: 252896
The TSan-instrumented version of libcxx doesn't even build on OS X at this point. Let's skip it from the OS X build for now, since most of TSan functionality doesn't depend on it. This will enable `check-tsan` to be run.
Differential Revision: http://reviews.llvm.org/D14486
llvm-svn: 252455
Several tests currently deadlock when the lit test suite is run on OS X. Let's mark them as unsupported.
Differential Revision: http://reviews.llvm.org/D14443
llvm-svn: 252402
This patch enables running lit tests on OS X:
1) Simply enable tests for Darwin (they were restricted to Linux and FreeBSD).
2) Disable using instrumented libcxx (libcxx_tsan) on Darwin.
3) On Darwin, override abort_on_error=0, otherwise all tests would generate crash logs and take much longer to process.
Differential Revision: http://reviews.llvm.org/D14439
llvm-svn: 252309
The current implementation does not work on darwin and can have issues with other OSes in future.
See http://reviews.llvm.org/D14427
Make it portable once and for all (minus usleep call).
Reviewed in:
http://reviews.llvm.org/D14434
llvm-svn: 252292
Hi, this patch adds a CMake flag called `COMPILER_RT_ENABLE_TSAN_OSX`, which is off by default. If enabled, the build system will be building the OS X version of the TSan runtime library (called `libclang_rt.tsan_osx_dynamic.dylib`). I'll submit patches that fix OS X build errors shortly.
This is part of an effort to port TSan to OS X, and it's one the very first steps. Don't expect TSan on OS X to actually work or pass tests at this point.
llvm-svn: 251915
Instead, assume we're going to target triple specified by
COMPILER_RT_DEFAULT_TARGET_TRIPLE and build runtimes for this triple
(and hope that the host compiler can target them).
This will help users that use cross-compiler on their host to build
Clang that would work on a different architecture. This will also come in
handy if one would want to configure several compiler-rt build trees on
the same host, using just-built Clang that can target many
architectures.
This doesn't change the behavior in the default build configuration.
llvm-svn: 247099
Race deduplication code proved to be a performance bottleneck in the past if suppressions/annotations are used, or just some races left unaddressed. And we still get user complaints about this:
https://groups.google.com/forum/#!topic/thread-sanitizer/hB0WyiTI4e4
ReportRace already has several layers of caching for racy pcs/addresses to make deduplication faster. However, ReportRace still takes a global mutex (ThreadRegistry and ReportMutex) during deduplication and also calls mmap/munmap (which take process-wide semaphore in kernel), this makes deduplication non-scalable.
This patch moves race deduplication outside of global mutexes and also removes all mmap/munmap calls.
As the result, race_stress.cc with 100 threads and 10000 iterations become 30x faster:
before:
real 0m21.673s
user 0m5.932s
sys 0m34.885s
after:
real 0m0.720s
user 0m23.646s
sys 0m1.254s
http://reviews.llvm.org/D12554
llvm-svn: 246758
This patch adds support for tsan on aarch64-linux with 42-bit VMA
(current default config for 64K pagesize kernels). The support is
enabled by defining the SANITIZER_AARCH64_VMA to 42 at build time
for both clang/llvm and compiler-rt. The default VMA is 39 bits.
It also enabled tsan for previous supported VMA (39).
llvm-svn: 246330
This patch enabled TSAN for aarch64 with 39-bit VMA layout. As defined by
tsan_platform.h the layout used is:
0000 4000 00 - 0200 0000 00: main binary
2000 0000 00 - 4000 0000 00: shadow memory
4000 0000 00 - 5000 0000 00: metainfo
5000 0000 00 - 6000 0000 00: -
6000 0000 00 - 6200 0000 00: traces
6200 0000 00 - 7d00 0000 00: -
7d00 0000 00 - 7e00 0000 00: heap
7e00 0000 00 - 7fff ffff ff: modules and main thread stack
Which gives it about 8GB for main binary, 4GB for heap and 8GB for
modules and main thread stack.
Most of tests are passing, with the exception of:
* ignore_lib0, ignore_lib1, ignore_lib3 due a kernel limitation for
no support to make mmap page non-executable.
* longjmp tests due missing specialized assembly routines.
These tests are xfail for now.
The only tsan issue still showing is:
rtl/TsanRtlTest/Posix.ThreadLocalAccesses
Which still required further investigation. The test is disable for
aarch64 for now.
llvm-svn: 244055
POSIX states that "It shall be safe to destroy an initialized condition
variable upon which no threads are currently blocked", and later clarifies
"A condition variable can be destroyed immediately after all the threads
that are blocked on it are awakened) (in examples section). Tsan reported
such destruction as a data race.
Fixes https://llvm.org/bugs/show_bug.cgi?id=23616
Reviewed in http://reviews.llvm.org/D10693
llvm-svn: 241082
The new suppression type is called "race_top" and is matched only against top frame in report stacks.
This is required for situations when we want to suppress a race in a "thread pool" or "event loop" implementation.
If we simply use "race:ThreadPool::Execute" suppression, that can suppress everything in the program.
Reviewed in http://reviews.llvm.org/D10686
llvm-svn: 240949
Previously tsan modelled dup2(oldfd, newfd) as write on newfd.
We hit several cases where the write lead to false positives:
1. Some software dups a closed pipe in place of a socket before closing
the socket (to prevent races actually).
2. Some daemons dup /dev/null in place of stdin/stdout.
On the other hand we have not seen cases when write here catches real bugs.
So model dup2 as read on newfd instead.
llvm-svn: 240687
We see false reports between dlopen and dl_iterate_phdr.
This happens because tsan does not see dynamic linker
internal synchronization. Unpoison module names
in dl_iterate_phdr callback.
llvm-svn: 240576
This happens only in corner cases, but we observed this on a real app.
See the test for description of the exact scenario that lead to unbounded memory consumption.
llvm-svn: 240535
Mark longjmp tests as XFAIL because longjmp assembly for mips is not yet implemented.
Reviewers: dsanders, dvyukov, samsonov
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential Revision: http://reviews.llvm.org/D9526
llvm-svn: 236847
We incorrectly replaced shadow slots
when the new value is not stronger than the old one.
The bug can lead to false negatives.
The bug was detected by Go race test suite:
https://github.com/golang/go/issues/10589
llvm-svn: 236008
Fixes https://llvm.org/bugs/show_bug.cgi?id=23235
If pthread_create is followed by pthread_detach,
the new thread may not acquire synchronize with
the parent thread.
llvm-svn: 235293
Munmap interceptor did not reset meta shadow for the range,
and __tsan_java_move crashed because it encountered
non-zero meta shadow for the destination.
llvm-svn: 232029
If the thread receives a signal concurrently with PTRACE_ATTACH,
we can get notification about the signal before notification about stop.
In such case we need to forward the signal to the thread, otherwise
the signal will be missed (as we do PTRACE_DETACH with arg=0) and
any logic relying on signals will break. After forwarding we need to
continue to wait for stopping, because the thread is not stopped yet.
We do ignore delivery of SIGSTOP, because we want to make stop-the-world
as invisible as possible.
http://reviews.llvm.org/D7723
--This line, and those below, will be ignored--
M lib/sanitizer_common/sanitizer_stoptheworld_linux_libcdep.cc
M test/tsan/signal_segv_handler.cc
llvm-svn: 229832
Long story short: stop-the-world briefly resets SIGSEGV handler to SIG_DFL.
This breaks programs that handle and continue after SIGSEGV (namely JVM).
See the test and comments for details.
http://reviews.llvm.org/D7722
llvm-svn: 229678
If a memory access is unaligned, emit __tsan_unaligned_read/write
callbacks instead of __tsan_read/write.
Required to change semantics of __tsan_unaligned_read/write to not do the user memory.
But since they were unused (other than through __sanitizer_unaligned_load/store) this is fine.
Fixes long standing issue 17:
https://code.google.com/p/thread-sanitizer/issues/detail?id=17
llvm-svn: 227230
Even sleep(1) lead to episodical flakes on some machines.
Use an invisible by tsan barrier to enforce required execution order instead.
This makes the tests deterministic and faster.
llvm-svn: 226659
signal handler reads sa_sigaction when a concurrent sigaction call can modify it
as the result in could try to call SIG_DFL or a partially overwritten function pointer
llvm-svn: 224530
Summary:
Always quote suppressions files given to *_OPTIONS.
This will make it not break when given full Windows paths (otherwise,
parsing would stop after the drive's letter + ':').
Also fix one or two cases where the suppression files' extensions were
not *.supp.
Reviewers: samsonov, kcc
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6680
llvm-svn: 224529
currently it fails in cmake build with weird errors:
/tmp/real_deadlock_detector_stress_test-68a5ae.o: In function `__clang_call_terminate':
/ssd/src/llvm/projects/compiler-rt/test/tsan/real_deadlock_detector_stress_test.cc:(.text.__clang_call_terminate[__clang_call_terminate]+0x12): undefined reference to `__cxa_begin_catch'
/ssd/src/llvm/projects/compiler-rt/test/tsan/real_deadlock_detector_stress_test.cc:(.text.__clang_call_terminate[__clang_call_terminate]+0x17): undefined reference to `std::terminate()'
/tmp/real_deadlock_detector_stress_test-68a5ae.o: In function `std::vector<int, std::allocator<int> >::_M_check_len(unsigned long, char const*) const':
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:1339: undefined reference to `std::__throw_length_error(char const*)'
/tmp/real_deadlock_detector_stress_test-68a5ae.o: In function `__gnu_cxx::new_allocator<int>::allocate(unsigned long, void const*)':
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:102: undefined reference to `std::__throw_bad_alloc()'
/tmp/real_deadlock_detector_stress_test-68a5ae.o:(.eh_frame+0x63): undefined reference to `__gxx_personality_v0'
clang-3.5: error: linker command failed with exit code 1 (use -v to see invocation)
llvm-svn: 224511
this test is flaky because of ASLR
app memory is 7e8000000000-800000000000,
there may or may not be a 1TB hole depending on
where ASLR will choose to map libraries
llvm-svn: 223469
This commit changes the place where TSan runtime turns full path
to binary or shared library into its basename
(/usr/foo/mybinary -> mybinary). Instead of doing it as early as possible
(when we obtained the full path from the symbolizer), we now do it as
late as possible (right before printing the error report).
This seems like a right thing to do - stripping to basename is a detail
of report formatting implementation, and should belong there. Also, we
might need the full path at some point - for example, to match the
suppressions.
llvm-svn: 221225
The current handling (manual execution of atexit callbacks)
is overly complex and leads to constant problems due to mutual ordering of callbacks.
Instead simply wrap callbacks into our wrapper to establish
the necessary synchronization.
Fixes issue https://code.google.com/p/thread-sanitizer/issues/detail?id=80
llvm-svn: 219675
In debug mode tsan checks that user accesses
access user memory. NULL is not user memory.
So the test fails. Allocate real inaccessible
memory for the test.
llvm-svn: 218069
I don't remember that crash on mmap in internal allocator
ever yielded anything useful, only crashes in rare wierd untested situations.
One of the reasons for crash was to catch if tsan starts allocating
clocks using mmap. Tsan does not allocate clocks using internal_alloc anymore.
Solve it once and for all by allowing mmaps.
llvm-svn: 217929
We are interested in verifying that -gline-tables-only provides enough
debug information for verbose error reports and symbolized stack traces.
llvm-svn: 217284
Vector clocks is the most actively allocated object in tsan runtime.
Current internal allocator is not scalable enough to handle allocation
of clocks in scalable way (too small caches). This changes transforms
clocks to 2-level array with 512-byte blocks. Since all blocks are of
the same size, it's possible to cache them more efficiently in per-thread caches.
llvm-svn: 214912
This is a minor fix to two tsan tests that were expecting the wrong
column information. Now that clang emits column information by default
in its debugging output, the tests had started failing.
llvm-svn: 212779
The tsan's deadlock detector has been used in Chromium for a while;
it found a few real bugs and reported no false positives.
So, it's time to give it a bit more exposure.
llvm-svn: 212533
Introduce new public header <sanitizer/allocator_interface.h> and a set
of functions __sanitizer_get_ownership(), __sanitizer_malloc_hook() etc.
that will eventually replace their tool-specific equivalents
(__asan_get_ownership(), __msan_get_ownership() etc.). Tool-specific
functions are now deprecated and implemented as stubs redirecting
to __sanitizer_ versions (which are implemented differently in each tool).
Replace all uses of __xsan_ versions with __sanitizer_ versions in unit
and lit tests.
llvm-svn: 212469
Add a script that is used to deflake inherently flaky tsan tests.
It is invoked from lit tests as:
%deflake %run %t
The script runs the target program up to 10 times,
until it produces a tsan warning.
llvm-svn: 209898
The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists).
This solves a number of problems:
- eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26)
- eliminates contention in SyncTab
- eliminates contention in internal allocator during allocation of sync objects
- removes a bunch of ad-hoc code in java interface
- reduces java shadow from 2x to 1/2x
- allows to memorize heap block meta info for Java and Go
- allows to cleanup sync object meta info for Go
- which in turn enabled deadlock detector for Go
llvm-svn: 209810
TSan can produce false positives in code that uses C++11 threading,
as it doesn't see synchronization inside standard library. See
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-February/035408.html
for an example of such case.
We may build custom TSan-instrumented version libcxx to fight with that.
This change adds build rules for libcxx_tsan and integrates it into
testing infrastructure.
llvm-svn: 208737
ocasionally it fails with a slightly different report:
WARNING: ThreadSanitizer: signal handler spoils errno (pid=3674)
#0 MyHandler(int, siginfo*, void*)
...
#4 __tsan_free_hook
#5 main signal_errno.cc:40
llvm-svn: 206754
if the thread is cancelled in pthread_cond_wait, it locks the mutex before
processing pthread_cleanup stack
but tsan was missing that, thus reporting false double-lock/wrong-unlock errors
see the test for details
llvm-svn: 203648
Dmitry Vyukov