Summary:
Operator new interceptors behavior is now controlled by their nothrow
property as well as by allocator_may_return_null flag value:
- allocator_may_return_null=* + new() - die on allocation error
- allocator_may_return_null=0 + new(nothrow) - die on allocation error
- allocator_may_return_null=1 + new(nothrow) - return null
Ideally new() should throw std::bad_alloc exception, but that is not
trivial to achieve, hence TODO.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34731
llvm-svn: 306604
This reverts commit r304941. Vitaly Buka writes:
"Actually it depends on return value.
Test is for char* version of function. It will probably fail for int
version."
llvm-svn: 304943
On Darwin, strerror_r returns an int, not a char*. I don't think this
test really depends on what strerror_r returns, so I've used something
else in place of the result of the call to strerror_r.
llvm-svn: 304941
GNU version of strerror_r returns a result pointer that doesn't match the input
buffer. The result pointer is in fact a pointer to some internal storage.
TSAN was recording a write to this location, which was incorrect.
Fixed https://github.com/google/sanitizers/issues/696
llvm-svn: 304858
This patch allows the Swift compiler to emit calls to `__tsan_external_write` before starting any modifying access, which will cause TSan to detect races on arrays, dictionaries and other classes defined in non-instrumented modules. Races on collections from the Swift standard library and user-defined structs and a frequent cause of subtle bugs and it's important that TSan detects those on top of existing LLVM IR instrumentation, which already detects races in direct memory accesses.
Differential Revision: https://reviews.llvm.org/D31630
llvm-svn: 302050
These test cases occassionally fail when run on powerpc64le:
ignore_lib1.cc
ignore_lib5.cc
TestCases/Posix/current_allocated_bytes.cc
rtl/TsanRtlTest/Posix.ThreadLocalAccesses
TestCases/Posix/coverage-fork-direct.cc
The failures cause false problem reports to be sent to developers whose
code had nothing to do with the failures. Reactivate them when the real
problems are fixed.
This could also be related to the same problems as with the tests
ThreadedOneSizeMallocStressTest, ThreadedMallocStressTest, ManyThreadsTest,
and several others that do not run reliably on powerpc.
llvm-svn: 301798
We need to make sure that the "external" API isn't dup'ing all data races into a single one (because the stack might look the same) and suppressing all external races. This works now, so just adding a test for that.
Differential Revision: https://reviews.llvm.org/D31734
llvm-svn: 301011
On Darwin, the setting ignore_noninstrumented_modules is used to suppress false positives in code that users don't have control of. The recently added "external" API (which can be used to detect races on objects provided by system libraries, but the race is actually user's fault) ignores this flag and it can report issues in non-instrumented modules. This patch fixes that.
Differential Revision: https://reviews.llvm.org/D31553
llvm-svn: 301000
This patch make sure we don't report deadlocks and other bug types when we're inside an interceptor that was called from a noninstrumented module (when ignore_noninstrumented_modules=1 is set). Adding a testcase that shows that deadlock detection still works on Darwin (to make sure we're not silencing too many reports).
Differential Revision: https://reviews.llvm.org/D31449
llvm-svn: 300998
We seem to assume that OS-provided thread IDs are either uptr or int, neither of which is true on Darwin. This introduces a tid_t type, which holds a OS-provided thread ID (gettid on Linux, pthread_threadid_np on Darwin, pthread_self on FreeBSD).
Differential Revision: https://reviews.llvm.org/D31774
llvm-svn: 300473
TSan reports a false positive when using xpc_connection_cancel. We're missing a happens-before edge from xpc_connection_cancel to the event handler on the same connection.
Differential Revision: https://reviews.llvm.org/D31475
llvm-svn: 299086
While it's usually a bug to call GCD APIs, such as dispatch_after, with NULL as a queue, this often "somehow" works and TSan should maintain binary compatibility with existing code. This patch makes sure we don't try to call Acquire and Release on NULL queues, and add one such testcase for dispatch_after.
Differential Revision: https://reviews.llvm.org/D31355
llvm-svn: 298820
There are several problems with the current annotations (AnnotateRWLockCreate and friends):
- they don't fully support deadlock detection (we need a hook _before_ mutex lock)
- they don't support insertion of random artificial delays to perturb execution (again we need a hook _before_ mutex lock)
- they don't support setting extended mutex attributes like read/write reentrancy (only "linker init" was bolted on)
- they don't support setting mutex attributes if a mutex don't have a "constructor" (e.g. static, Java, Go mutexes)
- they don't ignore synchronization inside of lock/unlock operations which leads to slowdown and false negatives
The new annotations solve of the above problems. See tsan_interface.h for the interface specification and comments.
Reviewed in https://reviews.llvm.org/D31093
llvm-svn: 298809
HLE flags can be combined with memory order in atomic operations.
Currently tsan runtime crashes on e.g. IsStoreOrder(mo) in atomic store
if any of these additional flags are specified.
Filter these flags out.
See the comment as to why it is safe.
llvm-svn: 298378
In D28836, we added a way to tag heap objects and thus provide object types into report. This patch exposes this information into the debugging API.
Differential Revision: https://reviews.llvm.org/D30023
llvm-svn: 295318
This test fails consistently on Ubuntu 16.xx powerpc64 LE systems.
The cause is being investigated and in the meantime disable it so
the buildbots can run cleanly.
llvm-svn: 293939
This patch allows a non-instrumented library to call into TSan runtime, and tell us about "readonly" and "modifying" accesses to an arbitrary "object" and provide the caller and tag (type of object). This allows TSan to detect violations of API threading contracts where "read-only" methods can be called simulatenously from multiple threads, while modifying methods must be exclusive.
Differential Revision: https://reviews.llvm.org/D28836
llvm-svn: 293885
When dealing with GCD worker threads, TSan currently prints weird things like "created by thread T-1" and "[failed to restore the stack]" in reports. This patch avoids that and instead prints "Thread T3 (...) is a GCD worker thread".
Differential Revision: https://reviews.llvm.org/D29103
llvm-svn: 293882
Currently, os_id of the main thread contains the PID instead of a thread ID. Let's fix this.
Differential Revision: https://reviews.llvm.org/D29106
llvm-svn: 293201
TSan recently got the "ignore_noninstrumented_modules" flag, which disables tracking of read and writes that come from noninstrumented modules (via interceptors). This is a way of suppressing false positives coming from system libraries and other noninstrumented code. This patch turns this on by default on Darwin, where it's supposed to replace the previous solution, "ignore_interceptors_accesses", which disables tracking in *all* interceptors. The new approach should re-enable TSan's ability to find races via interceptors on Darwin.
Differential Revision: https://reviews.llvm.org/D29041
llvm-svn: 292981
Running lit tests and unit tests of ASan and TSan on macOS has very bad performance when running with a high number of threads. This is caused by xnu (the macOS kernel), which currently doesn't handle mapping and unmapping of sanitizer shadow regions (reserved VM which are several terabytes large) very well. The situation is so bad that increasing the number of threads actually makes the total testing time larger. The macOS buildbots are affected by this. Note that we can't easily limit the number of sanitizer testing threads without affecting the rest of the tests.
This patch adds a special "group" into lit, and limits the number of concurrently running tests in this group. This helps solve the contention problem, while still allowing other tests to run in full, that means running lit with -j8 will still with 8 threads, and parallelism is only limited in sanitizer tests.
Differential Revision: https://reviews.llvm.org/D28420
llvm-svn: 292549
Running lit tests and unit tests of ASan and TSan on macOS has very bad performance when running with a high number of threads. This is caused by xnu (the macOS kernel), which currently doesn't handle mapping and unmapping of sanitizer shadow regions (reserved VM which are several terabytes large) very well. The situation is so bad that increasing the number of threads actually makes the total testing time larger. The macOS buildbots are affected by this. Note that we can't easily limit the number of sanitizer testing threads without affecting the rest of the tests.
This patch adds a special "group" into lit, and limits the number of concurrently running tests in this group. This helps solve the contention problem, while still allowing other tests to run in full, that means running lit with -j8 will still with 8 threads, and parallelism is only limited in sanitizer tests.
Differential Revision: https://reviews.llvm.org/D28420
llvm-svn: 292232
On Darwin, we currently use 'ignore_interceptors_accesses', which is a heavy-weight solution that simply turns of race detection in all interceptors. This was done to suppress false positives coming from system libraries (non-instrumented code), but it also silences a lot of real races. This patch implements an alternative approach that should allow us to enable interceptors and report races coming from them, but only if they are called directly from instrumented code.
The patch matches the caller PC in each interceptors. For non-instrumented code, we call ThreadIgnoreBegin.
The assumption here is that the number of instrumented modules is low. Most likely there's only one (the instrumented main executable) and all the other modules are system libraries (non-instrumented).
Differential Revision: https://reviews.llvm.org/D28264
llvm-svn: 291631
This patch starts passing architecture information about a module to llvm-symbolizer and into text reports. This fixes the longstanding x86_64/x86_64h mismatch issue on Darwin.
Differential Revision: https://reviews.llvm.org/D27390
llvm-svn: 291287
In ASan, we have __asan_locate_address and __asan_get_alloc_stack, which is used in LLDB/Xcode to show the allocation backtrace for a heap memory object. This patch implements the same for TSan.
Differential Revision: https://reviews.llvm.org/D27656
llvm-svn: 290119
We already have an interceptor for __shared_weak_count::__release_shared, this patch handles __shared_count::__release_shared in the same way. This should get rid of TSan false positives when using std::future.
Differential Revision: https://reviews.llvm.org/D27797
llvm-svn: 289831
Objects may move during the garbage collection, and JVM needs
to notify ThreadAnalyzer about that. The new function
__tsan_java_find eliminates the need to maintain these
objects both in ThreadAnalyzer and JVM.
Author: Alexander Smundak (asmundak)
Reviewed in https://reviews.llvm.org/D27720
llvm-svn: 289682
The Clang driver on macOS decides the deployment target based on various things, like your host OS version, the SDK version and some environment variables, which makes lit tests pass or fail based on your environment. Let's make sure we run all lit tests with `-mmacosx-version-min=${SANITIZER_MIN_OSX_VERSION}` (10.9 unless overriden).
Differential Revision: https://reviews.llvm.org/D26929
llvm-svn: 288186
The lit expansion of "%deflake " (notice the space after) expands in a way that the space is removed, this fixes that.
Differential Revision: https://reviews.llvm.org/D27139
llvm-svn: 287989
GCD queues can be suspended and resumed with dispatch_suspend and dispatch_resume. We need to add synchronization between the call to dispatch_resume and any subsequent executions of blocks in the queue that was resumed. We already have an Acquire(q) before the block executes, so this patch just adds the Release(q) in an interceptor of dispatch_resume.
Differential Revision: https://reviews.llvm.org/D27112
llvm-svn: 287902
/proc/self/maps can't be read atomically, this leads to episodic
crashes in libignore as it thinks that a module is loaded twice.
See the new test for an example.
dl_iterate_phdr does not have this problem.
Switch libignore to dl_iterate_phdr.
llvm-svn: 287632
This adds support for TSan C++ exception handling, where we need to add extra calls to __tsan_func_exit when a function is exitted via exception mechanisms. Otherwise the shadow stack gets corrupted (leaked). This patch moves and enhances the existing implementation of EscapeEnumerator that finds all possible function exit points, and adds extra EH cleanup blocks where needed.
Differential Revision: https://reviews.llvm.org/D26177
llvm-svn: 286894
This patch replaces fprintf with print_address function in LSAN
tests. This is necessary because of different printing of pointers
in fprintf and sanitizer's print function.
Differential Revision: https://reviews.llvm.org/D26084.
llvm-svn: 286816
Summary: Unit tests for the new clang flags.
Reviewers: eugenis, dvyukov
Subscribers: kubabrecka, llvm-commits
Patch by Alex Shlyapnikov.
Differential Revision: https://reviews.llvm.org/D26462
llvm-svn: 286670
Atomic stores terminate release sequences on the atomic variable,
and must use ReleaseStore primitive instead of Release.
This was broken in r192355 during a refactoring.
Restore correct behavior and add a test.
llvm-svn: 286211
Although rare, atomic accesses to floating-point types seem to be valid, i.e. `%a = load atomic float ...`. The TSan instrumentation pass however tries to emit inttoptr, which is incorrect, we should use a bitcast here. Anyway, IRBuilder already has a convenient helper function for this.
Differential Revision: https://reviews.llvm.org/D26266
llvm-svn: 286136
Apparently, the std_shared_ptr.cc testcase works fine on Darwin, even without the instrumented libcxx. Let's enable it.
Differential Revision: https://reviews.llvm.org/D26162
llvm-svn: 285634
GCD (libdispatch) has a concept of “target queues”: Each queue has either an implicit or explicit target queue, where the task is handed over to when it’s time to execute it. For example, a concurrent queue can have a serial target queue (effectively making the first queue serial), or multiple queues can have the same serial target queue (which means tasks in all the queues are mutually excluded). Thus we need to acquire-release semantics on the full “chain” of target queues.
This patch changes the way we Acquire() and Release() when executing tasks in queues. Now we’ll walk the chain of target queues and synchronize on each queue that is serial (or when dealing with a barrier block). This should avoid false positives when using dispatch_set_target_queue().
Differential Revision: https://reviews.llvm.org/D25835
llvm-svn: 285613
4.1+ Linux kernels map pie binaries at 0x55:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d1fd836dcf00d2028c700c7e44d2c23404062c90
Currently tsan does not support app memory at 0x55 (https://github.com/google/sanitizers/issues/503).
Older kernels also map pie binaries at 0x55 when ASLR is disables (most notably under gdb).
This change extends tsan mapping for linux/x86_64 to cover 0x554-0x568 app range and fixes both 4.1+ kernels and gdb.
This required to slightly shrink low and high app ranges and move heap. The mapping become even more non-linear, since now we xor lower bits. Now even a continuous app range maps to split, intermixed shadow ranges. This breaks ShadowToMemImpl as it assumes linear mapping at least within a continuous app range (however it turned out to be already broken at least on arm64/42-bit vma as uncovered by r281970). So also change ShadowToMemImpl to hopefully a more robust implementation that does not assume a linear mapping.
llvm-svn: 282152
This patch adds a wrapper for call_once, which uses an already-compiled helper __call_once with an atomic release which is invisible to TSan. To avoid false positives, the interceptor performs an explicit atomic release in the callback wrapper.
Differential Revision: https://reviews.llvm.org/D24188
llvm-svn: 280920
The system implementation of OSAtomicTestAndClear returns the original bit, but the TSan interceptor has a bug which always returns zero from the function. This patch fixes this and adds a test.
Differential Revision: https://reviews.llvm.org/D23061
llvm-svn: 277461
On Darwin, there are some apps that rely on realloc(nullptr, 0) returning a valid pointer. TSan currently returns nullptr in this case, let's fix it to avoid breaking binary compatibility.
Differential Revision: https://reviews.llvm.org/D22800
llvm-svn: 277458
When we delay signals we can deliver them when the signal
is blocked. This can be surprising to the program.
Intercept signal blocking functions merely to process
pending signals. As the result, at worst we will delay
a signal till return from the signal blocking function.
llvm-svn: 276876
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.
Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.
This patch doesn't contain behavior changes.
Reviewers: kubabrecka, rnk
Subscribers: wang0109, llvm-commits, kubabrecka, chrisha
Differential Revision: http://reviews.llvm.org/D21952
llvm-svn: 275111
This patch adds interceptors for dispatch_io_*, dispatch_read and dispatch_write functions. This avoids false positives when using GCD IO. Adding several test cases.
Differential Revision: http://reviews.llvm.org/D21889
llvm-svn: 275071
These test in this change are objc++, but are built using %clang, not %clangxx.
The reason this works is the driver has been adding -lc++ for sanitizer enabled
builds. By making these tests use %clangxx, they no longer depend on the driver
linking to c++. Doing so will allow us to prevent overlinking of libc++ for
applications.
llvm-svn: 274989
This patch adds synchronization between the creation of the GCD data object and destructor’s execution. It’s far from perfect, because ideally we’d want to synchronize the destruction of the last reference (via dispatch_release) and the destructor’s execution, but intercepting objc_release is problematic.
Differential Revision: http://reviews.llvm.org/D21990
llvm-svn: 274749
We already have interceptors for dispatch_source API (e.g. dispatch_source_set_event_handler), but they currently only handle submission synchronization. We also need to synchronize based on the target queue (serial, concurrent), in other words, we need to use dispatch_callback_wrap. This patch implements that.
Differential Revision: http://reviews.llvm.org/D21999
llvm-svn: 274619
In the patch that introduced support for GCD barrier blocks, I removed releasing a group when leaving it (in dispatch_group_leave). However, this is necessary to synchronize leaving a group and a notification callback (dispatch_group_notify). Adding this back, simplifying dispatch_group_notify_f and adding a test case.
Differential Revision: http://reviews.llvm.org/D21927
llvm-svn: 274549
Because we use SCOPED_TSAN_INTERCEPTOR in the dispatch_once interceptor, the original dispatch_once can also be sometimes called (when ignores are enabled or when thr->is_inited is false). However the original dispatch_once function doesn’t expect to find “2” in the storage and it will spin forever (but we use “2” to indicate that the initialization is already done, so no waiting is necessary). This patch makes sure we never call the original dispatch_once.
Differential Revision: http://reviews.llvm.org/D21976
llvm-svn: 274548
The dispatch_group_async interceptor actually extends the lifetime of the executed block. This means the destructor of the block (and captured variables) is called *after* dispatch_group_leave, which changes the semantics of dispatch_group_async. This patch fixes that.
Differential Revision: http://reviews.llvm.org/D21816
llvm-svn: 274117
Adding support for GCD barrier blocks in concurrent queues. This uses two sync object in the same way as read-write locks do. This also simplifies the use of dispatch groups (the notifications act as barrier blocks).
Differential Revision: http://reviews.llvm.org/D21604
llvm-svn: 273893
There is a "well-known" TSan false positive when using C++ weak_ptr/shared_ptr and code in destructors, e.g. described at <https://llvm.org/bugs/show_bug.cgi?id=22324>. The "standard" solution is to build and use a TSan-instrumented version of libcxx, which is not trivial for end-users. This patch tries a different approach (on OS X): It adds an interceptor for the specific function in libc++.dylib, which implements the atomic operation that needs to be visible to TSan.
Differential Revision: http://reviews.llvm.org/D21609
llvm-svn: 273806
The new annotation was added a while ago, but was not actually used.
Use the annotation to detect linker-initialized mutexes instead
of the broken IsGlobalVar which has both false positives and false
negatives. Remove IsGlobalVar mess.
llvm-svn: 271663
Currently the added test produces false race reports with glibc 2.19,
because DLTS memory is reused by pthread under the hood.
Use the DTLS machinery to intercept new DTLS ranges.
__tls_get_addr known to cause issues for tsan in the past,
so write the interceptor more carefully.
Reviewed in http://reviews.llvm.org/D20927
llvm-svn: 271568
In one of the already existing apps that I'm testing TSan on, I really see a mutex path that is longer than 10 (but not by much, something like 11-13 actually). Let's raise this to 20 and weaken the assertion so we don't crash.
Differential Revision: http://reviews.llvm.org/D20427
llvm-svn: 270319
We're missing interceptors for dispatch_after and dispatch_after_f. Let's add them to avoid false positives. Added a test case.
Differential Revision: http://reviews.llvm.org/D20426
llvm-svn: 270071
The ignore_interceptors_accesses setting did not have an effect on mmap, so
let's change that. It helps in cases user code is accessing the memory
written to by mmap when the synchronization is ensured by the code that
does not get rebuilt.
(This effects Swift interoperability since it's runtime is mapping memory
which gets accessed by the code emitted into the Swift application by the
compiler.)
Differential Revision: http://reviews.llvm.org/D20294
llvm-svn: 269855
Another stack where we try to free sync objects,
but don't have a processors is:
// ResetRange
// __interceptor_munmap
// __deallocate_stack
// start_thread
// clone
Again, it is a latent bug that lead to memory leaks.
Also, increase amount of memory we scan in MetaMap::ResetRange.
Without that the test does not fail, as we fail to free
the sync objects on stack.
llvm-svn: 269041
Fixes crash reported in:
https://bugs.chromium.org/p/v8/issues/detail?id=4995
The problem is that we don't have a processor in a free interceptor
during thread exit.
The crash was introduced by introduction of Processors.
However, previously we silently leaked memory which wasn't any better.
llvm-svn: 268782
In http://reviews.llvm.org/D19100, I introduced a bug: On OS X, existing programs rely on malloc_size() to detect whether a pointer comes from heap memory (malloc_size returns non-zero) or not. We have to distinguish between a zero-sized allocation (where we need to return 1 from malloc_size, due to other binary compatibility reasons, see http://reviews.llvm.org/D19100), and pointers that are not returned from malloc at all.
Differential Revision: http://reviews.llvm.org/D19653
llvm-svn: 268157
The field "pid" in ReportThread is used to store the OS-provided thread ID (pthread_self or gettid). The name "pid" suggests it's a process ID, which it isn't. Let's rename it.
Differential Revision: http://reviews.llvm.org/D19365
llvm-svn: 266994
At the moment almost every lit.site.cfg.in contains two lines comment:
## Autogenerated by LLVM/Clang configuration.
# Do not edit!
The patch adds variable LIT_SITE_CFG_IN_HEADER, that is replaced from
configure_lit_site_cfg with the note and some useful information.
llvm-svn: 266520
The custom zone implementation for OS X must not return 0 (even for 0-sized allocations). Returning 0 indicates that the pointer doesn't belong to the zone. This can break existing applications. The underlaying allocator allocates 1 byte for 0-sized allocations anyway, so returning 1 in this case is okay.
Differential Revision: http://reviews.llvm.org/D19100
llvm-svn: 266283
On one of our testing machines, we're running the tests under heavy load, and especially in the fork-based TSan tests, we're seeing timeouts when a test uses sleep(10), assuming that calling fork() on another thread will finish sooner than that. This patch removes a timeout and makes another one longer.
Differential Revision: http://reviews.llvm.org/D18476
llvm-svn: 265666
OS X provides atomic functions in libkern/OSAtomic.h. These provide atomic guarantees and they have alternatives which have barrier semantics. This patch adds proper TSan support for the functions from libkern/OSAtomic.h.
Differential Revision: http://reviews.llvm.org/D18500
llvm-svn: 265665
Adding an interceptor with two more release+acquire pairs to avoid false positives with dispatch_apply.
Differential Revision: http://reviews.llvm.org/D18722
llvm-svn: 265662
XPC APIs have async callbacks, and we need some more happen-before edges to avoid false positives. This patch add them, plus a test case (sorry for the long boilerplate code, but XPC just needs all that).
Differential Revision: http://reviews.llvm.org/D18493
llvm-svn: 265661
GCD has APIs for event sources, we need some more release-acquire pairs to avoid false positives in TSan.
Differential Revision: http://reviews.llvm.org/D18515
llvm-svn: 265660
In the interceptor for dispatch_sync, we're currently missing synchronization between the callback and the code *after* the call to dispatch_sync. This patch fixes this by adding an extra release+acquire pair to dispatch_sync() and similar APIs. Added a testcase.
Differential Revision: http://reviews.llvm.org/D18502
llvm-svn: 265659
A little embarrassing, but we're missing the call to FileCheck in several Darwin tests. Let's fix this.
Differential Revision: http://reviews.llvm.org/D18503
llvm-svn: 265658
This patch fixes the custom ThreadState destruction on OS X to avoid crashing when dispatch_main calls pthread_exit which quits the main thread.
Differential Revision: http://reviews.llvm.org/D18496
llvm-svn: 264627
On OS X, fork() under TSan asserts (in debug builds only) because REAL(fork) calls some intercepted functions, which check that no internal locks are held via CheckNoLocks(). But the wrapper of fork intentionally holds some locks. This patch fixes that by using ScopedIgnoreInterceptors during the call to REAL(fork). After that, all the fork-based tests seem to pass on OS X, so let's just remove all the UNSUPPORTED: darwin annotations we have.
Differential Revision: http://reviews.llvm.org/D18409
llvm-svn: 264261
On OS X, internal_mmap just uses mmap, which can invoke callbacks into libmalloc (e.g. when MallocStackLogging is enabled). This can subsequently call other intercepted functions, and this breaks our Darwin-specific ThreadState initialization. Let's use direct syscalls in internal_mmap and internal_munmap. Added a testcase.
Differential Revision: http://reviews.llvm.org/D18431
llvm-svn: 264259
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.
TSan unit tests run with a statically linked runtime, where interceptors don't work. To avoid aborting the process in this case, the patch replaces `DisableReexec()` with a weak `ReexecDisabled()` function which is defined to return true in unit tests.
Differential Revision: http://reviews.llvm.org/D18212
llvm-svn: 263695
This patch adds a new TSan report type, ReportTypeMutexInvalidAccess, which is triggered when pthread_mutex_lock or pthread_mutex_unlock returns EINVAL (this means the mutex is invalid, uninitialized or already destroyed).
Differential Revision: http://reviews.llvm.org/D18132
llvm-svn: 263641
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.
Differential Revision: http://reviews.llvm.org/D18121
llvm-svn: 263551
Currently, TSan only reports everything in a formatted textual form. The idea behind this patch is to provide a consistent API that can be used to query information contained in a TSan-produced report. User can use these APIs either in a debugger (via a script or directly), or they can use it directly from the process (e.g. in the __tsan_on_report callback). ASan already has a similar API, see http://reviews.llvm.org/D4466.
Differential Revision: http://reviews.llvm.org/D16191
llvm-svn: 263126
Incremented the pc for each architecture in accordance with StackTrace:GetPreviousInstructionPC
Reviewers: samsonov, dvyukov
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential: http://reviews.llvm.org/D17802
llvm-svn: 262483
This test expects pthread_mutex_init in the frame #0 of thread T1 but we
get memset at frame #0 because memset that is called from pthread_init_mutex
is being intercepted by TSan
llvm-svn: 261986
The first issue is that we longjmp from ScopedInterceptor scope
when called from an ignored lib. This leaves thr->in_ignored_lib set.
This, in turn, disables handling of sigaction. This, in turn,
corrupts tsan state since signals delivered asynchronously.
Another issue is that we can ignore synchronization in asignal
handler, if the signal is delivered into an IgnoreSync region.
Since signals are generally asynchronous, they should ignore
memory access/synchronization/interceptor ignores.
This could lead to false positives in signal handlers.
llvm-svn: 261658
1. Add two explicit -stdlib=libstdc++ in conjunction with -static-libstdc++
2. Pass -nostdinc++ when adding include paths for libc++ built for tsan. This
prevents clang finding the headers twice which would confuse #include_next
Differential Revision: http://reviews.llvm.org/D17189
llvm-svn: 260883
There's no obvious reason it should fail in this way but it's the only change
on the blamelist. I suspect stale lit*.cfg's from previous builds.
llvm-svn: 260672
The lit test-suite containing the unit tests needs to be explicitly specified
as an argument to lit.py since it is no longer discovered when the other tests
are run (because they are one directory deeper).
dfsan, lsan, and sanitizer_common don't show the same problem.
llvm-svn: 260669
Summary:
This is a workaround to a problem in the 3.8 release that affects MIPS and
possibly other targets where the default is not supported but a sibling is
supported.
When TSAN_SUPPORTED_ARCH is not empty, cmake currently attempts to build a
tsan'd libcxx as well as test tsan for the default target regardless of whether
the default target is supported or not. This causes problems on MIPS32 since
tsan is supported for MIPS64 but not MIPS32.
This patch causes cmake to only build the libcxx and run the lit test-suite for
archictures in ${TSAN_SUPPORTED_ARCH}
This re-commit fixes an issue where 'check-tsan' continued to look for the
tsan'd libc++ in the directory it used to be built in.
Reviewers: hans, samsonov
Subscribers: tberghammer, llvm-commits, danalbert, srhines, dvyukov
Differential Revision: http://reviews.llvm.org/D16685
llvm-svn: 259542
check-tsan does not pick up the correct libc++.so. It succeeded on my machine
by picking up the libc++.so that was built before making this change.
llvm-svn: 259519
Summary:
This is a workaround to a problem in the 3.8 release that affects MIPS and
possibly other targets where the default is not supported but a sibling is
supported.
When TSAN_SUPPORTED_ARCH is not empty, cmake currently attempts to build a
tsan'd libcxx as well as test tsan for the default target regardless of whether
the default target is supported or not. This causes problems on MIPS32 since
tsan is supported for MIPS64 but not MIPS32.
This patch causes cmake to only build the libcxx and run the lit test-suite for
archictures in ${TSAN_SUPPORTED_ARCH}
Reviewers: hans, samsonov
Subscribers: tberghammer, llvm-commits, danalbert, srhines, dvyukov
Differential Revision: http://reviews.llvm.org/D16685
llvm-svn: 259512
On OS X, TSan already passes all unit and lit tests, but for real-world applications (even very simple ones), we currently produce a lot of false positive reports about data races. This makes TSan useless at this point, because the noise dominates real bugs. This introduces a runtime flag, "ignore_interceptors_accesses", off by default, which makes TSan ignore all memory accesses that happen from interceptors. This will significantly lower the coverage and miss a lot of bugs, but it eliminates most of the current false positives on OS X.
Differential Revision: http://reviews.llvm.org/D15189
llvm-svn: 257760
This patch adds PIE executable support for aarch64-linux. It adds
two more segments:
- 0x05500000000-0x05600000000: 39-bits PIE program segments
- 0x2aa00000000-0x2ab00000000: 42-bits PIE program segments
Fortunately it is possible to use the same transformation formula for
the new segments range with some adjustments in shadow to memory
formula (it adds a constant offset based on the VMA size).
A simple testcase is also added, however it is disabled on x86 due the
fact it might fail on newer kernels [1].
[1] https://git.kernel.org/linus/d1fd836dcf00d2028c700c7e44d2c23404062c90
llvm-svn: 256184
We're using the dispatch group itself to synchronize (to call Release() and Acquire() on it), but in dispatch group notifications, the group can already be disposed/deallocated. This causes a later assertion failure at `DCHECK_EQ(*meta, 0);` in `MetaMap::AllocBlock` when the same memory is reused (note that the failure only happens in debug builds).
Fixing this by retaining the group and releasing it in the notification. Adding a stress test case that reproduces this.
Differential Revision: http://reviews.llvm.org/D15380
llvm-svn: 255494
This patch is by Simone Atzeni with portions by Adhemerval Zanella.
This contains the LLVM patches to enable the thread sanitizer for
PPC64, both big- and little-endian. Two different virtual memory
sizes are supported: Old kernels use a 44-bit address space, while
newer kernels require a 46-bit address space.
There are two companion patches that will be added shortly. There is
a Clang patch to actually turn on the use of the thread sanitizer for
PPC64. There is also a patch that I wrote to provide interceptor
support for setjmp/longjmp on PPC64.
Patch discussion at reviews.llvm.org/D12841.
llvm-svn: 255057
Another attempt at fixing tsan_invisible_barrier.
Current implementation causes:
https://llvm.org/bugs/show_bug.cgi?id=25643
There were several unsuccessful iterations for this functionality:
Initially it was implemented in user code using REAL(pthread_barrier_wait). But pthread_barrier_wait is not supported on MacOS. Futexes are linux-specific for this matter.
Then we switched to atomics+usleep(10). But usleep produced parasitic "as-if synchronized via sleep" messages in reports which failed some output tests.
Then we switched to atomics+sched_yield. But this produced tons of tsan- visible events, which lead to "failed to restore stack trace" failures.
Move implementation into runtime and use internal_sched_yield in the wait loop.
This way tsan should see no events from the barrier, so not trace overflows and
no "as-if synchronized via sleep" messages.
llvm-svn: 255030
This patch adds release and acquire semantics for dispatch groups, plus a test case.
Differential Revision: http://reviews.llvm.org/D15048
llvm-svn: 255020
Check that TSan runtime doesn't contain compiler-inserted calls
to memset/memmove functions.
In future, we may consider moving this test to test/sanitizer_common,
as we don't want to have compiler-inserted memcpy/memmove calls in
any sanitizer runtime.
llvm-svn: 254955
This script is superseded by lit test suite integrated into CMake
for quite a while now. It doesn't support many tests, and require
custom hacks for a few other.
llvm-svn: 254932
On OS X, there are other-than-pthread locking APIs that are used quite extensively - OSSpinLock and os_lock_lock. Let's add interceptors for those.
Differential Revision: http://reviews.llvm.org/D14987
llvm-svn: 254611
In mmap_large.cc, let's use MAP_ANON instead of MAP_ANONYMOUS, because MAP_ANONYMOUS is only available on OS X 10.11 and later.
Differential Revision: http://reviews.llvm.org/D15180
llvm-svn: 254601
This patch adds release and acquire semantics for libdispatch semaphores and a test case.
Differential Revision: http://reviews.llvm.org/D14992
llvm-svn: 254412
This patch complete removed SANITIZER_AARCH64_VMA definition and usage.
AArch64 ports now supports runtime VMA detection and instrumentation
for 39 and 42-bit VMA.
It also Rewrite print_address to take a variadic argument list
(the addresses to print) and adjust the tests which uses it to the new
signature.
llvm-svn: 254319
Changing comments that have references to code.google.com to point to GitHub instead, because the current links are not redirected properly (they instead redirect to different issues, mostly ASan). NFC.
Differential Revision: http://reviews.llvm.org/D15053
llvm-svn: 254300
1) There's a few wrongly defined things in tsan_interceptors.cc,
2) a typo in tsan_rtl_amd64.S which calls setjmp instead of sigsetjmp in the interceptor, and
3) on OS X, accessing an mprotected page results in a SIGBUS (and not SIGSEGV).
Differential Revision: http://reviews.llvm.org/D15052
llvm-svn: 254299
Serial queues need extra happens-before between individual tasks executed in the same queue. This patch adds `Acquire(queue)` before the executed task and `Release(queue)` just after it (for serial queues only). Added a test case.
Differential Revision: http://reviews.llvm.org/D15011
llvm-svn: 254229
There's a few more lit tests that require features not available on OS X (MAP_32BIT, pthread_setname_np), let's mark them with "UNSUPPORTED: darwin".
Differential Revision: http://reviews.llvm.org/D14923
llvm-svn: 254225
Pthread spinlocks are not available on OS X and this test doesn't really require a spinlock.
Differential Revision: http://reviews.llvm.org/D14949
llvm-svn: 254224
When a race on file descriptors is detected, `FindThreadByUidLocked()` is called to retrieve ThreadContext with a specific unique_id. However, this ThreadContext might not exist in the thread registry anymore (it may have been recycled), in which case `FindThreadByUidLocked` will cause an assertion failure in `GetThreadLocked`. Adding a test case that reproduces this, producing:
FATAL: ThreadSanitizer CHECK failed: sanitizer_common/sanitizer_thread_registry.h:92 "((tid)) < ((n_contexts_))" (0x34, 0x34)
This patch fixes this by replacing the loop with `FindThreadContextLocked`.
Differential Revision: http://reviews.llvm.org/D14984
llvm-svn: 254223
This patch unify the 39 and 42-bit support for AArch64 by using an external
memory read to check the runtime detected VMA and select the better mapping
and transformation. Although slower, this leads to same instrumented binary
to be independent of the kernel.
Along with this change this patch also fix some 42-bit failures with
ALSR disable by increasing the upper high app memory threshold and also
the 42-bit madvise value for non large page set.
llvm-svn: 254151
We need to intercept libdispatch APIs (dispatch_sync, dispatch_async, etc.) to add synchronization between the code that submits the task and the code that gets executed (possibly on a different thread). This patch adds release+acquire semantics for dispatch_sync, and dispatch_async (plus their "_f" and barrier variants). The synchronization is done on malloc'd contexts (separate for each submitted block/callback). Added tests to show usage of dispatch_sync and dispatch_async, for cases where we expect no warnings and for cases where TSan finds races.
Differential Revision: http://reviews.llvm.org/D14745
llvm-svn: 253982
The test relies on two variables in different frames to end up being on the same address. For some reason, this isn't true on OS X. This patch adds `__attribute__((aligned(64)))` to the variables, which actually makes the variables occupy the same address. This is still not a guarantee, but it's more likely to work (the test looks very fragile already).
Differential Revision: http://reviews.llvm.org/D14925
llvm-svn: 253981
Pthread semaphores are not available on OS X. Let's replace sem_wait/sem_post with barrier_wait, which makes the test pass on OS X. I know that sem_wait/sem_post is intercepted by TSan, whereas barrier_wait is TSan-invisible, but the purpose of the test is not affected by this.
Also, let's properly initialize the mutex and cond variables.
Differential Revision: http://reviews.llvm.org/D14924
llvm-svn: 253980
On OS X, __thread variables are lazily heap-allocated (with malloc). Therefore, they're recognized as heap blocks (which is what they are) and not as TLS variables in TSan reports. Figuring out if a heap block is a TLS or not is difficult (in malloc interceptor we could analyze the caller and then mark the object), so let's instead modify the tests so that we expect the report to say "Location is heap block" instead of "Location is TLS".
Differential Revision: http://reviews.llvm.org/D14873
llvm-svn: 253858
OS X doesn't support POSIX semaphores (but it does have the API for it, which returns ENOSYS - "Function not implemented").
Differential Revision: http://reviews.llvm.org/D14865
llvm-svn: 253665
On OS X, we don't have pthread spinlocks, let's just use a regular mutex instead. Secondly, pthread_rwlock_t is much larger (200 bytes), so `char padding_[64 - sizeof(pthread_rwlock_t)]` actually underflows.
Differential Revision: http://reviews.llvm.org/D14862
llvm-svn: 253659
Several tests rely on CLOCK_MONOTONIC, which doesn't exist on OS X. This patch fixes these tests by either disabling them (in case of cond_version.c which doesn't make sense on OS X), or by porting the test to also work on OS X.
Differential Revision: http://reviews.llvm.org/D14861
llvm-svn: 253658
Several testcases need pthread barriers (e.g. all bench_*.cc which use test/tsan/bench.h) which are not available on OS X. Let's mark them with "UNSUPPORTED: darwin".
Differential Revision: http://reviews.llvm.org/D14636
llvm-svn: 253558
Reimplement dispatch_once in an interceptor to solve these issues that may produce false positives with TSan on OS X:
1) there is a racy load inside an inlined part of dispatch_once,
2) the fast path in dispatch_once doesn't perform an acquire load, so we don't properly synchronize the initialization and subsequent uses of whatever is initialized,
3) dispatch_once is already used in a lot of already-compiled code, so TSan doesn't see the inlined fast-path.
This patch uses a trick to avoid ever taking the fast path (by never storing ~0 into the predicate), which means the interceptor will always be called even from already-compiled code. Within the interceptor, our own atomic reads and writes are not written into shadow cells, so the race in the inlined part is not reported (because the accesses are only loads).
Differential Revision: http://reviews.llvm.org/D14811
llvm-svn: 253552
This patch adds assembly routines to enable setjmp/longjmp for aarch64
on linux. It fixes:
* test/tsan/longjmp2.cc
* test/tsan/longjmp3.cc
* test/tsan/longjmp4.cc
* test/tsan/signal_longjmp.cc
I also checked with perlbench from specpu2006 (it fails to run
with missing setjmp/longjmp intrumentation).
llvm-svn: 253205
Hiroshi Inoue