This patch complete removed SANITIZER_AARCH64_VMA definition and usage.
AArch64 ports now supports runtime VMA detection and instrumentation
for 39 and 42-bit VMA.
It also Rewrite print_address to take a variadic argument list
(the addresses to print) and adjust the tests which uses it to the new
signature.
llvm-svn: 254319
Changing comments that have references to code.google.com to point to GitHub instead, because the current links are not redirected properly (they instead redirect to different issues, mostly ASan). NFC.
Differential Revision: http://reviews.llvm.org/D15053
llvm-svn: 254300
1) There's a few wrongly defined things in tsan_interceptors.cc,
2) a typo in tsan_rtl_amd64.S which calls setjmp instead of sigsetjmp in the interceptor, and
3) on OS X, accessing an mprotected page results in a SIGBUS (and not SIGSEGV).
Differential Revision: http://reviews.llvm.org/D15052
llvm-svn: 254299
Serial queues need extra happens-before between individual tasks executed in the same queue. This patch adds `Acquire(queue)` before the executed task and `Release(queue)` just after it (for serial queues only). Added a test case.
Differential Revision: http://reviews.llvm.org/D15011
llvm-svn: 254229
There's a few more lit tests that require features not available on OS X (MAP_32BIT, pthread_setname_np), let's mark them with "UNSUPPORTED: darwin".
Differential Revision: http://reviews.llvm.org/D14923
llvm-svn: 254225
Pthread spinlocks are not available on OS X and this test doesn't really require a spinlock.
Differential Revision: http://reviews.llvm.org/D14949
llvm-svn: 254224
When a race on file descriptors is detected, `FindThreadByUidLocked()` is called to retrieve ThreadContext with a specific unique_id. However, this ThreadContext might not exist in the thread registry anymore (it may have been recycled), in which case `FindThreadByUidLocked` will cause an assertion failure in `GetThreadLocked`. Adding a test case that reproduces this, producing:
FATAL: ThreadSanitizer CHECK failed: sanitizer_common/sanitizer_thread_registry.h:92 "((tid)) < ((n_contexts_))" (0x34, 0x34)
This patch fixes this by replacing the loop with `FindThreadContextLocked`.
Differential Revision: http://reviews.llvm.org/D14984
llvm-svn: 254223
This patch unify the 39 and 42-bit support for AArch64 by using an external
memory read to check the runtime detected VMA and select the better mapping
and transformation. Although slower, this leads to same instrumented binary
to be independent of the kernel.
Along with this change this patch also fix some 42-bit failures with
ALSR disable by increasing the upper high app memory threshold and also
the 42-bit madvise value for non large page set.
llvm-svn: 254151
We need to intercept libdispatch APIs (dispatch_sync, dispatch_async, etc.) to add synchronization between the code that submits the task and the code that gets executed (possibly on a different thread). This patch adds release+acquire semantics for dispatch_sync, and dispatch_async (plus their "_f" and barrier variants). The synchronization is done on malloc'd contexts (separate for each submitted block/callback). Added tests to show usage of dispatch_sync and dispatch_async, for cases where we expect no warnings and for cases where TSan finds races.
Differential Revision: http://reviews.llvm.org/D14745
llvm-svn: 253982
The test relies on two variables in different frames to end up being on the same address. For some reason, this isn't true on OS X. This patch adds `__attribute__((aligned(64)))` to the variables, which actually makes the variables occupy the same address. This is still not a guarantee, but it's more likely to work (the test looks very fragile already).
Differential Revision: http://reviews.llvm.org/D14925
llvm-svn: 253981
Pthread semaphores are not available on OS X. Let's replace sem_wait/sem_post with barrier_wait, which makes the test pass on OS X. I know that sem_wait/sem_post is intercepted by TSan, whereas barrier_wait is TSan-invisible, but the purpose of the test is not affected by this.
Also, let's properly initialize the mutex and cond variables.
Differential Revision: http://reviews.llvm.org/D14924
llvm-svn: 253980
On OS X, __thread variables are lazily heap-allocated (with malloc). Therefore, they're recognized as heap blocks (which is what they are) and not as TLS variables in TSan reports. Figuring out if a heap block is a TLS or not is difficult (in malloc interceptor we could analyze the caller and then mark the object), so let's instead modify the tests so that we expect the report to say "Location is heap block" instead of "Location is TLS".
Differential Revision: http://reviews.llvm.org/D14873
llvm-svn: 253858
OS X doesn't support POSIX semaphores (but it does have the API for it, which returns ENOSYS - "Function not implemented").
Differential Revision: http://reviews.llvm.org/D14865
llvm-svn: 253665
On OS X, we don't have pthread spinlocks, let's just use a regular mutex instead. Secondly, pthread_rwlock_t is much larger (200 bytes), so `char padding_[64 - sizeof(pthread_rwlock_t)]` actually underflows.
Differential Revision: http://reviews.llvm.org/D14862
llvm-svn: 253659
Several tests rely on CLOCK_MONOTONIC, which doesn't exist on OS X. This patch fixes these tests by either disabling them (in case of cond_version.c which doesn't make sense on OS X), or by porting the test to also work on OS X.
Differential Revision: http://reviews.llvm.org/D14861
llvm-svn: 253658
Several testcases need pthread barriers (e.g. all bench_*.cc which use test/tsan/bench.h) which are not available on OS X. Let's mark them with "UNSUPPORTED: darwin".
Differential Revision: http://reviews.llvm.org/D14636
llvm-svn: 253558
Reimplement dispatch_once in an interceptor to solve these issues that may produce false positives with TSan on OS X:
1) there is a racy load inside an inlined part of dispatch_once,
2) the fast path in dispatch_once doesn't perform an acquire load, so we don't properly synchronize the initialization and subsequent uses of whatever is initialized,
3) dispatch_once is already used in a lot of already-compiled code, so TSan doesn't see the inlined fast-path.
This patch uses a trick to avoid ever taking the fast path (by never storing ~0 into the predicate), which means the interceptor will always be called even from already-compiled code. Within the interceptor, our own atomic reads and writes are not written into shadow cells, so the race in the inlined part is not reported (because the accesses are only loads).
Differential Revision: http://reviews.llvm.org/D14811
llvm-svn: 253552
This patch adds assembly routines to enable setjmp/longjmp for aarch64
on linux. It fixes:
* test/tsan/longjmp2.cc
* test/tsan/longjmp3.cc
* test/tsan/longjmp4.cc
* test/tsan/signal_longjmp.cc
I also checked with perlbench from specpu2006 (it fails to run
with missing setjmp/longjmp intrumentation).
llvm-svn: 253205
I noticed that when a symbol is named just "x", it gets demangled to "long long". On POSIX, AFAIK, mangled names always start with "_Z", so lets just require that.
Differential Revision: http://reviews.llvm.org/D14637
llvm-svn: 253080
On OS X, memcpy and memmove are actually aliases of the same function, so the memmove interceptor can be invoked on a call to memcpy. This patch updates the tests to expect either memmove or memcpy on a stack trace.
Differential Revision: http://reviews.llvm.org/D14638
llvm-svn: 253077
The OS X symbolizers (namely AtosSymbolizer) don't return full file paths, only file names. This patch modifies `mutexset*.cc` tests not to require a path to be present in the symbol on the stack trace.
Differential Revision: http://reviews.llvm.org/D14642
llvm-svn: 253075
This patch adds support for symbolication of globals (implements `SymbolizeData`) for `AtosSymbolizer` on OS X.
Differential Revision: http://reviews.llvm.org/D14618
llvm-svn: 253015
`DlAddrSymbolizer` is used on OS X when we're running inside a sandbox that prevents us from spawning an external symbolizer. This patch adds support for symbolication of globals (implements `SymbolizeData`) for `DlAddrSymbolizer`.
Differential Revision: http://reviews.llvm.org/D14613
llvm-svn: 252899
The default symbolizer, `llvm-symbolizer` provides sizes for global symbols. On OS X, we want to also allow using `atos` (because it's available everywhere and users don't need to copy/install it) and `dladdr` (it's the only available option when running in a sandbox). However, these symbolizers do not supply the symbol sizes, only names and starting addresses. This patch changes the reporting functions to hide the size of the symbol when this value is unavailable, and modifies tests to make this part of the report "optional".
Differential Revision: http://reviews.llvm.org/D14608
llvm-svn: 252896
The TSan-instrumented version of libcxx doesn't even build on OS X at this point. Let's skip it from the OS X build for now, since most of TSan functionality doesn't depend on it. This will enable `check-tsan` to be run.
Differential Revision: http://reviews.llvm.org/D14486
llvm-svn: 252455
Several tests currently deadlock when the lit test suite is run on OS X. Let's mark them as unsupported.
Differential Revision: http://reviews.llvm.org/D14443
llvm-svn: 252402
This patch enables running lit tests on OS X:
1) Simply enable tests for Darwin (they were restricted to Linux and FreeBSD).
2) Disable using instrumented libcxx (libcxx_tsan) on Darwin.
3) On Darwin, override abort_on_error=0, otherwise all tests would generate crash logs and take much longer to process.
Differential Revision: http://reviews.llvm.org/D14439
llvm-svn: 252309
The current implementation does not work on darwin and can have issues with other OSes in future.
See http://reviews.llvm.org/D14427
Make it portable once and for all (minus usleep call).
Reviewed in:
http://reviews.llvm.org/D14434
llvm-svn: 252292
Hi, this patch adds a CMake flag called `COMPILER_RT_ENABLE_TSAN_OSX`, which is off by default. If enabled, the build system will be building the OS X version of the TSan runtime library (called `libclang_rt.tsan_osx_dynamic.dylib`). I'll submit patches that fix OS X build errors shortly.
This is part of an effort to port TSan to OS X, and it's one the very first steps. Don't expect TSan on OS X to actually work or pass tests at this point.
llvm-svn: 251915
Instead, assume we're going to target triple specified by
COMPILER_RT_DEFAULT_TARGET_TRIPLE and build runtimes for this triple
(and hope that the host compiler can target them).
This will help users that use cross-compiler on their host to build
Clang that would work on a different architecture. This will also come in
handy if one would want to configure several compiler-rt build trees on
the same host, using just-built Clang that can target many
architectures.
This doesn't change the behavior in the default build configuration.
llvm-svn: 247099
Race deduplication code proved to be a performance bottleneck in the past if suppressions/annotations are used, or just some races left unaddressed. And we still get user complaints about this:
https://groups.google.com/forum/#!topic/thread-sanitizer/hB0WyiTI4e4
ReportRace already has several layers of caching for racy pcs/addresses to make deduplication faster. However, ReportRace still takes a global mutex (ThreadRegistry and ReportMutex) during deduplication and also calls mmap/munmap (which take process-wide semaphore in kernel), this makes deduplication non-scalable.
This patch moves race deduplication outside of global mutexes and also removes all mmap/munmap calls.
As the result, race_stress.cc with 100 threads and 10000 iterations become 30x faster:
before:
real 0m21.673s
user 0m5.932s
sys 0m34.885s
after:
real 0m0.720s
user 0m23.646s
sys 0m1.254s
http://reviews.llvm.org/D12554
llvm-svn: 246758
This patch adds support for tsan on aarch64-linux with 42-bit VMA
(current default config for 64K pagesize kernels). The support is
enabled by defining the SANITIZER_AARCH64_VMA to 42 at build time
for both clang/llvm and compiler-rt. The default VMA is 39 bits.
It also enabled tsan for previous supported VMA (39).
llvm-svn: 246330
This patch enabled TSAN for aarch64 with 39-bit VMA layout. As defined by
tsan_platform.h the layout used is:
0000 4000 00 - 0200 0000 00: main binary
2000 0000 00 - 4000 0000 00: shadow memory
4000 0000 00 - 5000 0000 00: metainfo
5000 0000 00 - 6000 0000 00: -
6000 0000 00 - 6200 0000 00: traces
6200 0000 00 - 7d00 0000 00: -
7d00 0000 00 - 7e00 0000 00: heap
7e00 0000 00 - 7fff ffff ff: modules and main thread stack
Which gives it about 8GB for main binary, 4GB for heap and 8GB for
modules and main thread stack.
Most of tests are passing, with the exception of:
* ignore_lib0, ignore_lib1, ignore_lib3 due a kernel limitation for
no support to make mmap page non-executable.
* longjmp tests due missing specialized assembly routines.
These tests are xfail for now.
The only tsan issue still showing is:
rtl/TsanRtlTest/Posix.ThreadLocalAccesses
Which still required further investigation. The test is disable for
aarch64 for now.
llvm-svn: 244055
POSIX states that "It shall be safe to destroy an initialized condition
variable upon which no threads are currently blocked", and later clarifies
"A condition variable can be destroyed immediately after all the threads
that are blocked on it are awakened) (in examples section). Tsan reported
such destruction as a data race.
Fixes https://llvm.org/bugs/show_bug.cgi?id=23616
Reviewed in http://reviews.llvm.org/D10693
llvm-svn: 241082
The new suppression type is called "race_top" and is matched only against top frame in report stacks.
This is required for situations when we want to suppress a race in a "thread pool" or "event loop" implementation.
If we simply use "race:ThreadPool::Execute" suppression, that can suppress everything in the program.
Reviewed in http://reviews.llvm.org/D10686
llvm-svn: 240949
Previously tsan modelled dup2(oldfd, newfd) as write on newfd.
We hit several cases where the write lead to false positives:
1. Some software dups a closed pipe in place of a socket before closing
the socket (to prevent races actually).
2. Some daemons dup /dev/null in place of stdin/stdout.
On the other hand we have not seen cases when write here catches real bugs.
So model dup2 as read on newfd instead.
llvm-svn: 240687
We see false reports between dlopen and dl_iterate_phdr.
This happens because tsan does not see dynamic linker
internal synchronization. Unpoison module names
in dl_iterate_phdr callback.
llvm-svn: 240576
This happens only in corner cases, but we observed this on a real app.
See the test for description of the exact scenario that lead to unbounded memory consumption.
llvm-svn: 240535
Mark longjmp tests as XFAIL because longjmp assembly for mips is not yet implemented.
Reviewers: dsanders, dvyukov, samsonov
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential Revision: http://reviews.llvm.org/D9526
llvm-svn: 236847
We incorrectly replaced shadow slots
when the new value is not stronger than the old one.
The bug can lead to false negatives.
The bug was detected by Go race test suite:
https://github.com/golang/go/issues/10589
llvm-svn: 236008
Fixes https://llvm.org/bugs/show_bug.cgi?id=23235
If pthread_create is followed by pthread_detach,
the new thread may not acquire synchronize with
the parent thread.
llvm-svn: 235293
Munmap interceptor did not reset meta shadow for the range,
and __tsan_java_move crashed because it encountered
non-zero meta shadow for the destination.
llvm-svn: 232029
If the thread receives a signal concurrently with PTRACE_ATTACH,
we can get notification about the signal before notification about stop.
In such case we need to forward the signal to the thread, otherwise
the signal will be missed (as we do PTRACE_DETACH with arg=0) and
any logic relying on signals will break. After forwarding we need to
continue to wait for stopping, because the thread is not stopped yet.
We do ignore delivery of SIGSTOP, because we want to make stop-the-world
as invisible as possible.
http://reviews.llvm.org/D7723
--This line, and those below, will be ignored--
M lib/sanitizer_common/sanitizer_stoptheworld_linux_libcdep.cc
M test/tsan/signal_segv_handler.cc
llvm-svn: 229832
Long story short: stop-the-world briefly resets SIGSEGV handler to SIG_DFL.
This breaks programs that handle and continue after SIGSEGV (namely JVM).
See the test and comments for details.
http://reviews.llvm.org/D7722
llvm-svn: 229678
If a memory access is unaligned, emit __tsan_unaligned_read/write
callbacks instead of __tsan_read/write.
Required to change semantics of __tsan_unaligned_read/write to not do the user memory.
But since they were unused (other than through __sanitizer_unaligned_load/store) this is fine.
Fixes long standing issue 17:
https://code.google.com/p/thread-sanitizer/issues/detail?id=17
llvm-svn: 227230
Even sleep(1) lead to episodical flakes on some machines.
Use an invisible by tsan barrier to enforce required execution order instead.
This makes the tests deterministic and faster.
llvm-svn: 226659
signal handler reads sa_sigaction when a concurrent sigaction call can modify it
as the result in could try to call SIG_DFL or a partially overwritten function pointer
llvm-svn: 224530
Summary:
Always quote suppressions files given to *_OPTIONS.
This will make it not break when given full Windows paths (otherwise,
parsing would stop after the drive's letter + ':').
Also fix one or two cases where the suppression files' extensions were
not *.supp.
Reviewers: samsonov, kcc
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6680
llvm-svn: 224529
currently it fails in cmake build with weird errors:
/tmp/real_deadlock_detector_stress_test-68a5ae.o: In function `__clang_call_terminate':
/ssd/src/llvm/projects/compiler-rt/test/tsan/real_deadlock_detector_stress_test.cc:(.text.__clang_call_terminate[__clang_call_terminate]+0x12): undefined reference to `__cxa_begin_catch'
/ssd/src/llvm/projects/compiler-rt/test/tsan/real_deadlock_detector_stress_test.cc:(.text.__clang_call_terminate[__clang_call_terminate]+0x17): undefined reference to `std::terminate()'
/tmp/real_deadlock_detector_stress_test-68a5ae.o: In function `std::vector<int, std::allocator<int> >::_M_check_len(unsigned long, char const*) const':
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_vector.h:1339: undefined reference to `std::__throw_length_error(char const*)'
/tmp/real_deadlock_detector_stress_test-68a5ae.o: In function `__gnu_cxx::new_allocator<int>::allocate(unsigned long, void const*)':
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ext/new_allocator.h:102: undefined reference to `std::__throw_bad_alloc()'
/tmp/real_deadlock_detector_stress_test-68a5ae.o:(.eh_frame+0x63): undefined reference to `__gxx_personality_v0'
clang-3.5: error: linker command failed with exit code 1 (use -v to see invocation)
llvm-svn: 224511
this test is flaky because of ASLR
app memory is 7e8000000000-800000000000,
there may or may not be a 1TB hole depending on
where ASLR will choose to map libraries
llvm-svn: 223469
This commit changes the place where TSan runtime turns full path
to binary or shared library into its basename
(/usr/foo/mybinary -> mybinary). Instead of doing it as early as possible
(when we obtained the full path from the symbolizer), we now do it as
late as possible (right before printing the error report).
This seems like a right thing to do - stripping to basename is a detail
of report formatting implementation, and should belong there. Also, we
might need the full path at some point - for example, to match the
suppressions.
llvm-svn: 221225
The current handling (manual execution of atexit callbacks)
is overly complex and leads to constant problems due to mutual ordering of callbacks.
Instead simply wrap callbacks into our wrapper to establish
the necessary synchronization.
Fixes issue https://code.google.com/p/thread-sanitizer/issues/detail?id=80
llvm-svn: 219675
In debug mode tsan checks that user accesses
access user memory. NULL is not user memory.
So the test fails. Allocate real inaccessible
memory for the test.
llvm-svn: 218069
I don't remember that crash on mmap in internal allocator
ever yielded anything useful, only crashes in rare wierd untested situations.
One of the reasons for crash was to catch if tsan starts allocating
clocks using mmap. Tsan does not allocate clocks using internal_alloc anymore.
Solve it once and for all by allowing mmaps.
llvm-svn: 217929
We are interested in verifying that -gline-tables-only provides enough
debug information for verbose error reports and symbolized stack traces.
llvm-svn: 217284
Vector clocks is the most actively allocated object in tsan runtime.
Current internal allocator is not scalable enough to handle allocation
of clocks in scalable way (too small caches). This changes transforms
clocks to 2-level array with 512-byte blocks. Since all blocks are of
the same size, it's possible to cache them more efficiently in per-thread caches.
llvm-svn: 214912
This is a minor fix to two tsan tests that were expecting the wrong
column information. Now that clang emits column information by default
in its debugging output, the tests had started failing.
llvm-svn: 212779
The tsan's deadlock detector has been used in Chromium for a while;
it found a few real bugs and reported no false positives.
So, it's time to give it a bit more exposure.
llvm-svn: 212533
Introduce new public header <sanitizer/allocator_interface.h> and a set
of functions __sanitizer_get_ownership(), __sanitizer_malloc_hook() etc.
that will eventually replace their tool-specific equivalents
(__asan_get_ownership(), __msan_get_ownership() etc.). Tool-specific
functions are now deprecated and implemented as stubs redirecting
to __sanitizer_ versions (which are implemented differently in each tool).
Replace all uses of __xsan_ versions with __sanitizer_ versions in unit
and lit tests.
llvm-svn: 212469
Add a script that is used to deflake inherently flaky tsan tests.
It is invoked from lit tests as:
%deflake %run %t
The script runs the target program up to 10 times,
until it produces a tsan warning.
llvm-svn: 209898
The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists).
This solves a number of problems:
- eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26)
- eliminates contention in SyncTab
- eliminates contention in internal allocator during allocation of sync objects
- removes a bunch of ad-hoc code in java interface
- reduces java shadow from 2x to 1/2x
- allows to memorize heap block meta info for Java and Go
- allows to cleanup sync object meta info for Go
- which in turn enabled deadlock detector for Go
llvm-svn: 209810
TSan can produce false positives in code that uses C++11 threading,
as it doesn't see synchronization inside standard library. See
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-February/035408.html
for an example of such case.
We may build custom TSan-instrumented version libcxx to fight with that.
This change adds build rules for libcxx_tsan and integrates it into
testing infrastructure.
llvm-svn: 208737
ocasionally it fails with a slightly different report:
WARNING: ThreadSanitizer: signal handler spoils errno (pid=3674)
#0 MyHandler(int, siginfo*, void*)
...
#4 __tsan_free_hook
#5 main signal_errno.cc:40
llvm-svn: 206754
if the thread is cancelled in pthread_cond_wait, it locks the mutex before
processing pthread_cleanup stack
but tsan was missing that, thus reporting false double-lock/wrong-unlock errors
see the test for details
llvm-svn: 203648
Adhemerval Zanella