Commit Graph

1504 Commits

Author SHA1 Message Date
Kuba Brecka 09d3e53a93 [tsan] dispatch_once interceptor will cause a crash/deadlock when the original dispatch_once is used
Because we use SCOPED_TSAN_INTERCEPTOR in the dispatch_once interceptor, the original dispatch_once can also be sometimes called (when ignores are enabled or when thr->is_inited is false). However the original dispatch_once function doesn’t expect to find “2” in the storage and it will spin forever (but we use “2” to indicate that the initialization is already done, so no waiting is necessary). This patch makes sure we never call the original dispatch_once.

Differential Revision: http://reviews.llvm.org/D21976

llvm-svn: 274548
2016-07-05 13:39:54 +00:00
Kuba Brecka 4d81bbdf53 [tsan] Stop extending the block’s lifetime in dispatch_group_async
The dispatch_group_async interceptor actually extends the lifetime of the executed block. This means the destructor of the block (and captured variables) is called *after* dispatch_group_leave, which changes the semantics of dispatch_group_async.  This patch fixes that.

Differential Revision: http://reviews.llvm.org/D21816

llvm-svn: 274117
2016-06-29 10:30:50 +00:00
Kuba Brecka 2621dea6eb [tsan] Add HB edges for GCD barrier blocks
Adding support for GCD barrier blocks in concurrent queues.  This uses two sync object in the same way as read-write locks do.  This also simplifies the use of dispatch groups (the notifications act as barrier blocks).

Differential Revision: http://reviews.llvm.org/D21604

llvm-svn: 273893
2016-06-27 16:49:23 +00:00
Kuba Brecka ca2668d038 [tsan] Treat non-barrier OSAtomic* functions as barriers to avoid false positives
The non-barrier versions of OSAtomic* functions are semantically mo_relaxed, but the two variants (e.g. OSAtomicAdd32 and OSAtomicAdd32Barrier) are actually aliases of each other, and we cannot have different interceptors for them, because they're actually the same function. Thus, we have to stay conservative and treat the non-barrier versions as mo_acq_rel.

Differential Revision: http://reviews.llvm.org/D21733

llvm-svn: 273890
2016-06-27 16:10:14 +00:00
Dmitry Vyukov b3a51bdcd7 tsan: don't create sync objects on acquire
Creating sync objects on acquire is pointless:
acquire of a just created sync object if a no-op.

llvm-svn: 273862
2016-06-27 11:14:59 +00:00
Kuba Brecka 30ad0c941a [tsan] Intercept libcxx __release_shared to avoid false positive with weak_ptrs and destructors in C++
There is a "well-known" TSan false positive when using C++ weak_ptr/shared_ptr and code in destructors, e.g. described at <https://llvm.org/bugs/show_bug.cgi?id=22324>. The "standard" solution is to build and use a TSan-instrumented version of libcxx, which is not trivial for end-users. This patch tries a different approach (on OS X): It adds an interceptor for the specific function in libc++.dylib, which implements the atomic operation that needs to be visible to TSan.

Differential Revision: http://reviews.llvm.org/D21609

llvm-svn: 273806
2016-06-26 08:14:01 +00:00
Evgeniy Stepanov 3c17c73924 [msan] Intercept send/sendto/sendmsg.
send/sendmsg moved from tsan to sanitizer_common; sendto is new.

llvm-svn: 272980
2016-06-17 00:43:11 +00:00
Kostya Serebryany bf6a04fde8 [sanitizers] introduce yet another API function: __sanitizer_install_malloc_and_free_hooks
llvm-svn: 272943
2016-06-16 20:06:06 +00:00
Dmitry Vyukov 88f47d3521 tsan: intercept epoll_pwait
llvm-svn: 272779
2016-06-15 12:22:05 +00:00
Kuba Brecka 304b2c2a45 [tsan] Switch to InternalAlloc everywhere __libc_malloc is currently used
This patch replaces all uses of __libc_malloc and friends with the internal allocator.

It seems that the only reason why we have calls to __libc_malloc in the first place was the lack of the internal allocator at the time. Using the internal allocator will also make sure that the system allocator is never used (this is the same behavior as ASan), and we don’t have to worry about working with unknown pointers coming from the system allocator.

Differential Revision: http://reviews.llvm.org/D21025

llvm-svn: 271916
2016-06-06 18:18:47 +00:00
Kuba Brecka cdf3492191 [tsan] On OS X, optimize main thread’s ThreadState accesses
This is a very simple optimization that gets about 10% speedup for certain programs. We’re currently storing the pointer to the main thread’s ThreadState, but we can store the state directly in a static variable, which avoid the load acquire.

Differential Revision: http://reviews.llvm.org/D20910

llvm-svn: 271906
2016-06-06 16:27:38 +00:00
Kuba Brecka 312071943c [tsan] On OS X, build Go runtime with -mmacosx-version-min
We're not building the Go runtime with -mmacosx-version-min, which means it'll have a minimum deployment target set to the system you're building on. Let's make the code compile (and link) with -mmacosx-version-min=10.7.

Differential Revision: http://reviews.llvm.org/D20670

llvm-svn: 271833
2016-06-05 15:15:36 +00:00
Dmitry Vyukov 0f7e949a42 tsan: rely on AnnotateRWLockCreateStatic to detect linker-initialized mutexes
The new annotation was added a while ago, but was not actually used.
Use the annotation to detect linker-initialized mutexes instead
of the broken IsGlobalVar which has both false positives and false
negatives. Remove IsGlobalVar mess.

llvm-svn: 271663
2016-06-03 11:48:27 +00:00
Dmitry Vyukov 1ae44e6a2b tsan: clean up dynamic TLS memory between reuse
Currently the added test produces false race reports with glibc 2.19,
because DLTS memory is reused by pthread under the hood.

Use the DTLS machinery to intercept new DTLS ranges.
__tls_get_addr known to cause issues for tsan in the past,
so write the interceptor more carefully.

Reviewed in http://reviews.llvm.org/D20927

llvm-svn: 271568
2016-06-02 19:18:22 +00:00
Kostya Serebryany d9a451ab0d [sanitizer] Move termination functions in their own file
Summary:
As suggested by kcc@ in http://reviews.llvm.org/D20084#441418, move the CheckFailed and Die functions, and their associated callback functionalities in their own separate file.
I expended the build rules to include a new rule that would not include those termination functions, so that another project can define their own.
The tests check-{a,t,m,ub,l,e,df}san are all passing.

Reviewers: llvm-commits, kcc

Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D20742

llvm-svn: 271055
2016-05-27 21:57:17 +00:00
Vitaly Buka 132639120a Init tsan with .preinit_array section
Summary: Some libraries, like OpenSSL, runs code from .init section.

Reviewers: kcc, eugenis

Subscribers: kubabrecka, llvm-commits

Differential Revision: http://reviews.llvm.org/D20646

llvm-svn: 270873
2016-05-26 17:05:36 +00:00
Benjamin Kramer f9679e89a1 Revert "[sanitizer] Move *fstat to the common interceptors"
This reverts commit r269981. Breaks msan tests on linux
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/24019/steps/test%20standalone%20compiler-rt/logs/stdio

llvm-svn: 270076
2016-05-19 16:03:10 +00:00
Kuba Brecka ed29c21d5d [tsan] Add support for GCD's dispatch_after and dispatch_after_f
We're missing interceptors for dispatch_after and dispatch_after_f. Let's add them to avoid false positives. Added a test case.

Differential Revision: http://reviews.llvm.org/D20426

llvm-svn: 270071
2016-05-19 15:31:42 +00:00
Mike Aizatsky 522afdd77e [sanitizer] Move *fstat to the common interceptors
Summary:
Adds *fstat to the common interceptors.

Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.

Resubmit of http://reviews.llvm.org/D20318 with ios build fixes.

Reviewers: eugenis, vitalybuka, aizatsky

Subscribers: zaks.anna, kcc, bruening, kubabrecka, srhines, danalbert, tberghammer

Differential Revision: http://reviews.llvm.org/D20350

llvm-svn: 269981
2016-05-18 20:49:49 +00:00
Kuba Brecka daac6a0083 [sanitizer] Fix a crash when demangling Swift symbols, take 3
The previous patch (r269291) was reverted (commented out) because the patch caused leaks that
were detected by LSan and they broke some lit tests.  The actual reason was that dlsym allocates
an error string buffer in TLS, and some LSan lit tests are intentionally not scanning TLS for
root pointers.  This patch simply makes LSan ignore the allocation from dlsym, because it's
not interesting anyway.

llvm-svn: 269917
2016-05-18 13:00:20 +00:00
Mike Aizatsky 41c2afe5d9 Revert "[sanitizer] Move *fstat to the common interceptors"
This reverts commit http://reviews.llvm.org/rL269856

llvm-svn: 269863
2016-05-17 23:28:56 +00:00
Mike Aizatsky 924acb50c9 [sanitizer] Move *fstat to the common interceptors
Summary:
Adds *fstat to the common interceptors.

Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.

Reviewers: eugenis, vitalybuka, aizatsky

Subscribers: tberghammer, danalbert, srhines, kubabrecka, bruening, kcc

Differential Revision: http://reviews.llvm.org/D20318

llvm-svn: 269856
2016-05-17 22:26:50 +00:00
Anna Zaks 7d47c990e6 [tsan] Ensure mmap respects ignore_interceptors_accesses
The ignore_interceptors_accesses setting did not have an effect on mmap, so
let's change that. It helps in cases user code is accessing the memory
written to by mmap when the synchronization is ensured by the code that
does not get rebuilt.

(This effects Swift interoperability since it's runtime is mapping memory
which gets accessed by the code emitted into the Swift application by the
compiler.)

Differential Revision: http://reviews.llvm.org/D20294

llvm-svn: 269855
2016-05-17 22:24:55 +00:00
Mike Aizatsky 599eef49e5 [sanitizers] disabling LateInitialize call to fix the build.
http://reviews.llvm.org/rL269291 introduced a memory leak.
Disabling offending call temprorary rather than rolling back the chain
of CLs.

llvm-svn: 269799
2016-05-17 18:44:21 +00:00
Kuba Brecka 3a0736e279 Fixup for r269291, which broke the Go TSan build. Let's not use the symbolizer in Go builds.
llvm-svn: 269293
2016-05-12 13:28:45 +00:00
Kuba Brecka 5bdf86ec7c [sanitizer] Fix a crash when demangling Swift symbols, take 2
To invoke the Swift demangler, we use dlsym to locate swift_demangle. However, dlsym malloc's storage and stores it in thread-local storage. Since allocations from the symbolizer are done with the system allocator (at least in TSan, interceptors are skipped when inside the symbolizer), we will crash when we try to deallocate later using the sanitizer allocator again.

To fix this, let's just not call dlsym from the demangler, and call it during initialization. The dlsym function calls malloc, so it needs to be only used after our allocator is initialized. Adding a Symbolizer::LateInitialize call that is only invoked after all other initializations.

Differential Revision: http://reviews.llvm.org/D20015

llvm-svn: 269291
2016-05-12 13:11:03 +00:00
Evgeniy Stepanov feb73c8566 [sanitizer] Move *stat to the common interceptors
Adds *stat to the common interceptors.

Removes the now-duplicate *stat interceptor from msan/tsan/esan.
This adds *stat to asan, which previously did not intercept it.

Patch by Qin Zhao.

llvm-svn: 269223
2016-05-11 20:02:15 +00:00
Dmitry Vyukov 2498629e34 tsan: fix another crash due to processors
Another stack where we try to free sync objects,
but don't have a processors is:

  //   ResetRange
  //   __interceptor_munmap
  //   __deallocate_stack
  //   start_thread
  //   clone

Again, it is a latent bug that lead to memory leaks.
Also, increase amount of memory we scan in MetaMap::ResetRange.
Without that the test does not fail, as we fail to free
the sync objects on stack.

llvm-svn: 269041
2016-05-10 11:19:50 +00:00
Dmitry Vyukov 144eafd9ee tsan: fix a crash
Fixes crash reported in:
https://bugs.chromium.org/p/v8/issues/detail?id=4995

The problem is that we don't have a processor in a free interceptor
during thread exit.

The crash was introduced by introduction of Processors.
However, previously we silently leaked memory which wasn't any better.

llvm-svn: 268782
2016-05-06 19:35:22 +00:00
Mike Aizatsky c826e634cc [sanitizer] Move stat/__xstat to the common interceptors
Summary:
Adds stat/__xstat to the common interceptors.

Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.

Resubmit of http://reviews.llvm.org/D19875 with win build fixes.

Reviewers: aizatsky, eugenis

Subscribers: tberghammer, llvm-commits, danalbert, vitalybuka, bruening, srhines, kubabrecka, kcc

Differential Revision: http://reviews.llvm.org/D19890

llvm-svn: 268466
2016-05-03 23:43:45 +00:00
Mike Aizatsky 7e72f66bf2 Revert "[sanitizer] Move stat/__xstat to the common interceptors"
This reverts commit 268440 because it breaks the windows bot.

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/21425/steps/build%20compiler-rt/logs/stdio

llvm-svn: 268448
2016-05-03 21:49:56 +00:00
Mike Aizatsky 3eb521d417 [sanitizer] Move stat/__xstat to the common interceptors
Summary:
Adds stat/__xstat to the common interceptors.

Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.

Reviewers: aizatsky, eugenis

Subscribers: tberghammer, danalbert, srhines, kubabrecka, llvm-commits, vitalybuka, eugenis, kcc, bruening

Differential Revision: http://reviews.llvm.org/D19875

llvm-svn: 268440
2016-05-03 21:22:06 +00:00
Dmitry Vyukov 7a1b7addf8 tsan: update tsan_analyze to what tip clang generates
We used to depend on host gcc. But some distributions got
new gcc recently which broke the check. Generally, we can't
depend that an arbitrary host gcc generates something stable.

Switch to clang.
This has an additional advantage of catching regressions in
clang codegen.

llvm-svn: 268382
2016-05-03 13:59:41 +00:00
Kuba Brecka 9ccde5ace4 [tsan] Return 0 from malloc_size for non-malloc'd pointers
In http://reviews.llvm.org/D19100, I introduced a bug: On OS X, existing programs rely on malloc_size() to detect whether a pointer comes from heap memory (malloc_size returns non-zero) or not. We have to distinguish between a zero-sized allocation (where we need to return 1 from malloc_size, due to other binary compatibility reasons, see http://reviews.llvm.org/D19100), and pointers that are not returned from malloc at all.

Differential Revision: http://reviews.llvm.org/D19653

llvm-svn: 268157
2016-04-30 07:14:41 +00:00
Kuba Brecka dc7b607b09 [tsan] Fix Darwin GCD support after separation of Processor and ThreadState
Recent TSan changes (r267678) which factor out parts of ThreadState into a Processor structure broke worker threads on OS X.  This fixes it by properly calling ProcCreate for GCD worker threads and by replacing some CHECKs with RAW_CHECK in early process initialization.  CHECK() in TSan calls the allocator, which requires a valid Processor.

llvm-svn: 267864
2016-04-28 09:26:30 +00:00
Marcin Koscielnicki 66f0deacb5 [sanitizers] Get the proper symbol version when long double transition is involved.
On linux, some architectures had an ABI transition from 64-bit long double
(ie. same as double) to 128-bit long double.  On those, glibc symbols
involving long doubles come in two versions, and we need to pass the
correct one to dlvsym when intercepting them.

A few more functions we intercept are also versioned (all printf, scanf,
strtold variants), but there's no need to fix these, as the REAL() versions
are never called.

Differential Revision: http://reviews.llvm.org/D19555

llvm-svn: 267794
2016-04-27 21:24:21 +00:00
Marcin Koscielnicki b7b5ac60c4 [sanitizer] [SystemZ] Abort if the kernel might be vulnerable to CVE-2016-2143.
In short, CVE-2016-2143 will crash the machine if a process uses both >4TB
virtual addresses and fork().  ASan, TSan, and MSan will, by necessity, map
a sizable chunk of virtual address space, which is much larger than 4TB.
Even worse, sanitizers will always use fork() for llvm-symbolizer when a bug
is detected.  Disable all three by aborting on process initialization if
the running kernel version is not known to contain a fix.

Unfortunately, there's no reliable way to detect the fix without crashing
the kernel.  So, we rely on whitelisting - I've included a list of upstream
kernel versions that will work.  In case someone uses a distribution kernel
or applied the fix themselves, an override switch is also included.

Differential Revision: http://reviews.llvm.org/D19576

llvm-svn: 267747
2016-04-27 17:42:00 +00:00
Dmitry Vyukov 221e2c61ec tsan: fix windows Go support
Unmap can't unmap arbitrary regions on windows.

llvm-svn: 267716
2016-04-27 14:03:14 +00:00
Dmitry Vyukov 3e0315c044 tsan: make windows shadow mapping a bijection
CheckShadowMapping function started catching that
mem->shadow->mem mapping is not bijection.
Make it bijection.

llvm-svn: 267713
2016-04-27 13:34:01 +00:00
Dmitry Vyukov 5aac22fc23 tsan: fix build
error: implicit declaration of function 'abort' is invalid in C99
llvm-svn: 267710
2016-04-27 12:59:35 +00:00
Dmitry Vyukov 3efe395788 tsan: change tsan/Go interface for obtaining the current Processor
Current interface assumes that Go calls ProcWire/ProcUnwire
to establish the association between thread and proc.
With the wisdom of hindsight, this interface does not work
very well. I had to sprinkle Go scheduler with wire/unwire
calls, and any mistake leads to hard to debug crashes.
This is not something one wants to maintian.
Fortunately, there is a simpler solution. We can ask Go
runtime as to what is the current Processor, and that
question is very easy to answer on Go side.
Switch to such interface.

llvm-svn: 267703
2016-04-27 12:30:48 +00:00
Dmitry Vyukov f52c7c7abf tsan: fix windows build
llvm-svn: 267681
2016-04-27 08:39:32 +00:00
Dmitry Vyukov 3c65833a51 tsan: add missing file
llvm-svn: 267680
2016-04-27 08:34:55 +00:00
Dmitry Vyukov 066fefcbf3 tsan: fix compiler warning
tsan_debugging.cc: In function ‘void* __tsan_get_current_report()’:
tsan_debugging.cc:61:18: warning: cast from type ‘const __tsan::ReportDesc*’
                         to type ‘void*’ casts away qualifiers [-Wcast-qual]
   return (void *)rep;

llvm-svn: 267679
2016-04-27 08:28:08 +00:00
Dmitry Vyukov d87c7b321a tsan: split thread into logical and physical state
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).

Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:

Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.

Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.

This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.

Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 267678
2016-04-27 08:23:02 +00:00
Marcin Koscielnicki 3f9d7a217d [sanitizers] [NFC] Add defines for the various PowerPC ABIs.
Differential Revision: http://reviews.llvm.org/D19542

llvm-svn: 267586
2016-04-26 18:44:13 +00:00
Kostya Serebryany 99ed605799 [sanitizer] rename MmapNoAccess to MmapFixedNoAccess; NFC
llvm-svn: 267253
2016-04-22 23:46:53 +00:00
Kuba Brecka bf8b5f8dd2 [tsan] Rename ReportThread->pid to ReportThread->os_id
The field "pid" in ReportThread is used to store the OS-provided thread ID (pthread_self or gettid). The name "pid" suggests it's a process ID, which it isn't. Let's rename it.

Differential Revision: http://reviews.llvm.org/D19365

llvm-svn: 266994
2016-04-21 14:49:25 +00:00
Chuang-Yu Cheng e480455fd6 Always inlining PrintCurrentStackSlow of tsan library to fix tail-call issue
The real problem is that sanitizer_print_stack_trace obtains current PC and
expects the PC to be in the stack trace after function calls. We don't
prevent tail calls in sanitizer runtimes, so this assumption does not
necessary hold.

We add "always inline" attribute on PrintCurrentStackSlow to address this
issue, however this solution is not reliable enough, but unfortunately, we
don't see any simple, reliable solution.

Reviewers: samsonov hfinkel kbarton tjablin dvyukov kcc

http://reviews.llvm.org/D19148

Thanks Hal, dvyukov, and kcc for invaluable discussion, I have even borrowed
part of dvyukov's summary as my commit message!

llvm-svn: 266869
2016-04-20 10:28:41 +00:00
Marcin Koscielnicki 0a91cf8a84 Revert "[sanitizer] [SystemZ] Abort if the kernel might be vulnerable to CVE-2016-2143."
This reverts commit r266297.

llvm-svn: 266470
2016-04-15 20:00:12 +00:00
Marcin Koscielnicki c8dda336bb [sanitizer] [SystemZ] Abort if the kernel might be vulnerable to CVE-2016-2143.
In short, CVE-2016-2143 will crash the machine if a process uses both >4TB
virtual addresses and fork().  ASan, TSan, and MSan will, by necessity, map
a sizable chunk of virtual address space, which is much larger than 4TB.
Even worse, sanitizers will always use fork() for llvm-symbolizer when a bug
is detected.  Disable all three by aborting on process initialization if
the running kernel version is not known to contain a fix.

Unfortunately, there's no reliable way to detect the fix without crashing
the kernel.  So, we rely on whitelisting - I've included a list of upstream
kernel versions that will work.  In case someone uses a distribution kernel
or applied the fix themselves, an override switch is also included.

Differential Revision: http://reviews.llvm.org/D18915

llvm-svn: 266297
2016-04-14 12:56:24 +00:00
Kuba Brecka 173c690a61 [tsan] Fix size reporting for OS X zone allocator with 0-sized allocations
The custom zone implementation for OS X must not return 0 (even for 0-sized allocations). Returning 0 indicates that the pointer doesn't belong to the zone. This can break existing applications. The underlaying allocator allocates 1 byte for 0-sized allocations anyway, so returning 1 in this case is okay.

Differential Revision: http://reviews.llvm.org/D19100

llvm-svn: 266283
2016-04-14 09:05:19 +00:00
Kuba Brecka 41dd5296b3 [tsan] Fix a crash with dispatch_source_set_cancel_handler(NULL) on OS X
We need to handle the case when handler is NULL in dispatch_source_set_cancel_handler and similar interceptors.

Differential Revision: http://reviews.llvm.org/D18968

llvm-svn: 266080
2016-04-12 15:18:11 +00:00
Kuba Brecka 74f7f399ac [tsan] Add support for OS X OSAtomic* functions
OS X provides atomic functions in libkern/OSAtomic.h. These provide atomic guarantees and they have alternatives which have barrier semantics. This patch adds proper TSan support for the functions from libkern/OSAtomic.h.

Differential Revision: http://reviews.llvm.org/D18500

llvm-svn: 265665
2016-04-07 12:05:09 +00:00
Kuba Brecka 06bbca9ec4 [tsan] Add TSan __tsan_atomic* functions to tsan_interface.h
To avoid using the public header (tsan_interface_atomic.h), which has different data types, let's add all the __tsan_atomic* functions to tsan_interface.h. 

Differential Revision: http://reviews.llvm.org/D18543

llvm-svn: 265663
2016-04-07 11:59:36 +00:00
Kuba Brecka 399af93242 [tsan] Add interceptors for dispatch_apply
Adding an interceptor with two more release+acquire pairs to avoid false positives with dispatch_apply.

Differential Revision: http://reviews.llvm.org/D18722

llvm-svn: 265662
2016-04-07 11:52:51 +00:00
Kuba Brecka e316bb61b3 [tsan] Add XPC support (OS X)
XPC APIs have async callbacks, and we need some more happen-before edges to avoid false positives. This patch add them, plus a test case (sorry for the long boilerplate code, but XPC just needs all that).

Differential Revision: http://reviews.llvm.org/D18493

llvm-svn: 265661
2016-04-07 11:47:11 +00:00
Kuba Brecka cecb7faea2 [tsan] Add support for dispatch event sources
GCD has APIs for event sources, we need some more release-acquire pairs to avoid false positives in TSan.

Differential Revision: http://reviews.llvm.org/D18515

llvm-svn: 265660
2016-04-07 11:38:53 +00:00
Kuba Brecka 33c15c91a6 [tsan] Fix synchronization in dispatch_sync
In the interceptor for dispatch_sync, we're currently missing synchronization between the callback and the code *after* the call to dispatch_sync. This patch fixes this by adding an extra release+acquire pair to dispatch_sync() and similar APIs. Added a testcase.

Differential Revision: http://reviews.llvm.org/D18502

llvm-svn: 265659
2016-04-07 11:33:44 +00:00
Yabin Cui c7db569cc1 [tsan] Fix freebsd build.
Summary:
Freebsd doesn't have <sys/personality.h>, so call personality() only in
SANITIZER_LINUX.

Reviewers: llvm-commits, dvyukov, zatrazz, rengolin, beanz

Subscribers: beanz, emaste

Differential Revision: http://reviews.llvm.org/D18785

llvm-svn: 265378
2016-04-05 05:37:48 +00:00
Yabin Cui e3f558ba14 [tsan] Disable randomized address space on aarch64 linux.
Summary:
After patch https://lkml.org/lkml/2015/12/21/340 is introduced in
linux kernel, the random gap between stack and heap is increased
from 128M to 36G on 39-bit aarch64. And it is almost impossible
to cover this big range. So we need to disable randomized virtual
space on aarch64 linux.

Reviewers: llvm-commits, zatrazz, dvyukov, rengolin

Subscribers: aemerson, rengolin, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D18526

llvm-svn: 265366
2016-04-04 23:48:25 +00:00
Dmitry Vyukov 0d9b7bc9ad tsan: fix ignore handling in signal handlers
We've reset thr->ignore_reads_and_writes, but forget to do
thr->fast_state.ClearIgnoreBit(). So ignores were not effective
reset and fast_state.ignore_bit was corrupted if signal handler
itself uses ignores.

Properly reset/restore fast_state.ignore_bit around signal handlers.

llvm-svn: 265288
2016-04-04 10:52:59 +00:00
Kuba Brecka 0d026d9e9e [tsan] Fix a crash when exiting the main thread (e.g. dispatch_main)
This patch fixes the custom ThreadState destruction on OS X to avoid crashing when dispatch_main calls pthread_exit which quits the main thread.

Differential Revision: http://reviews.llvm.org/D18496

llvm-svn: 264627
2016-03-28 19:36:25 +00:00
Derek Bruening e988af9073 [sanitizer] Add memset, memmove, and memcpy to the common interceptors
Summary:
Currently, sanitizer_common_interceptors.inc has an implicit, undocumented
assumption that the sanitizer including it has previously declared
interceptors for memset and memmove.  Since the memset, memmove, and memcpy
routines require interception by many sanitizers, we add them to the
set of common interceptions, both to address the undocumented assumption
and to speed future tool development.  They are intercepted under a new
flag intercept_intrin.

The tsan interceptors are removed in favor of the new common versions.  The
asan and msan interceptors for these are more complex (they incur extra
interception steps and their function bodies are exposed to the compiler)
so they opt out of the common versions and keep their own.

Reviewers: vitalybuka

Subscribers: zhaoqin, llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D18465

llvm-svn: 264451
2016-03-25 19:33:45 +00:00
Kuba Brecka 3bab18d4af [tsan] Fix fork() and fork-based tests for OS X
On OS X, fork() under TSan asserts (in debug builds only) because REAL(fork) calls some intercepted functions, which check that no internal locks are held via CheckNoLocks(). But the wrapper of fork intentionally holds some locks. This patch fixes that by using ScopedIgnoreInterceptors during the call to REAL(fork). After that, all the fork-based tests seem to pass on OS X, so let's just remove all the UNSUPPORTED: darwin annotations we have.

Differential Revision: http://reviews.llvm.org/D18409

llvm-svn: 264261
2016-03-24 11:54:33 +00:00
Renato Golin c8622e8cde Revert "[tsan] Disable randomized address space on linux aarch64."
This reverts commits r264068 and r264079, and they were breaking the build and
weren't reverted in time, nor they exhibited expected behaviour from the
reviewers. There is more to discuss than just a test fix.

llvm-svn: 264150
2016-03-23 11:24:30 +00:00
Yabin Cui 28f367d7b3 [tsan] Fix check-tsan build by using CHECK_NE.
Reviewers: llvm-commits, srhines, dvyukov

Subscribers: srhines

Differential Revision: http://reviews.llvm.org/D18361

llvm-svn: 264079
2016-03-22 18:12:18 +00:00
Yabin Cui 909e600abd [tsan] Disable randomized address space on linux aarch64.
Summary:
After patch https://lkml.org/lkml/2015/12/21/340 is introduced in
linux kernel, the random gap between stack and heap is increased
from 128M to 36G on 39-bit aarch64. And it is almost impossible
to cover this big range. So I think we need to disable randomized
virtual space on aarch64 linux.

Reviewers: kcc, llvm-commits, eugenis, zatrazz, dvyukov, rengolin

Subscribers: rengolin, aemerson, tberghammer, danalbert, srhines, enh

Differential Revision: http://reviews.llvm.org/D18003

llvm-svn: 264068
2016-03-22 17:16:26 +00:00
Evgeniy Stepanov accf390a75 [sanitizer] Add strchr* to the common interceptors
Adds strchr, strchrnul, and strrchr to the common interceptors, under a new
common flag intercept_strchr.

Removes the now-duplicate strchr interceptor from asan and all 3
interceptors from tsan. Previously, asan did not intercept strchrnul, but
does now; previously, msan did not intercept strchr, strchrnul, or strrchr,
but does now.

http://reviews.llvm.org/D18329

Patch by Derek Bruening!

llvm-svn: 263992
2016-03-21 21:36:17 +00:00
Kuba Brecka 4b3833df7d [tsan] Add some NULL pointer checks into the debugging API
`__tsan_get_report_thread` and others can crash if a stack trace is missing, let's add the missing checks.

Differential Revision: http://reviews.llvm.org/D18306

llvm-svn: 263939
2016-03-21 12:12:44 +00:00
Mike Aizatsky 6d8a876159 [sancov] common flags initialization.
Summary:
Introducing InitializeCommonFlags accross all sanitizers to simplify
common flags management.

Setting coverage=1 when html_cov_report is requested.

Differential Revision: http://reviews.llvm.org/D18273

llvm-svn: 263820
2016-03-18 19:28:07 +00:00
Kuba Brecka 911e229f30 [tsan] Add interceptor for pthread_cond_timedwait_relative_np
On OS X, we have pthread_cond_timedwait_relative_np. TSan needs to intercept this API to avoid false positives when using condition variables.

Differential Revision: http://reviews.llvm.org/D18184

llvm-svn: 263782
2016-03-18 10:54:11 +00:00
Kuba Brecka 4c80867ecf [sanitizer] On OS X, verify that interceptors work and abort if not, take 2
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.

TSan unit tests run with a statically linked runtime, where interceptors don't work. To avoid aborting the process in this case, the patch replaces `DisableReexec()` with a weak `ReexecDisabled()` function which is defined to return true in unit tests.

Differential Revision: http://reviews.llvm.org/D18212

llvm-svn: 263695
2016-03-17 08:37:25 +00:00
Kuba Brecka 46bf454d18 [tsan] Detect uses of uninitialized, destroyed and invalid mutexes
This patch adds a new TSan report type, ReportTypeMutexInvalidAccess, which is triggered when pthread_mutex_lock or pthread_mutex_unlock returns EINVAL (this means the mutex is invalid, uninitialized or already destroyed).

Differential Revision: http://reviews.llvm.org/D18132

llvm-svn: 263641
2016-03-16 15:39:20 +00:00
Alexey Samsonov ed3d347e25 [sanitizer] Add strlen to the common interceptors
Summary:
Adds strlen to the common interceptors, under a new common flag
intercept_strlen.  This provides better sharing of interception code among
sanitizers and cleans up the inconsistent type declarations of the
previously duplicated interceptors.

Removes the now-duplicate strlen interceptor from asan, msan, and tsan.
The entry check semantics are normalized now for msan and asan, whose
private strlen interceptors contained multiple layers of checks that
included impossible-to-reach code.  The new semantics are identical to the
old: bypass interception if in the middle of init or if both on Mac and not
initialized; else, call the init routine and proceed.

Patch by Derek Bruening!

Reviewers: samsonov, vitalybuka

Subscribers: llvm-commits, kcc, zhaoqin

Differential Revision: http://reviews.llvm.org/D18020

llvm-svn: 263177
2016-03-11 00:45:49 +00:00
Kuba Brecka a1496f7c37 [tsan] Add TSan debugger APIs
Currently, TSan only reports everything in a formatted textual form. The idea behind this patch is to provide a consistent API that can be used to query information contained in a TSan-produced report. User can use these APIs either in a debugger (via a script or directly), or they can use it directly from the process (e.g. in the __tsan_on_report callback). ASan already has a similar API, see http://reviews.llvm.org/D4466.

Differential Revision: http://reviews.llvm.org/D16191

llvm-svn: 263126
2016-03-10 17:00:29 +00:00
Alexey Samsonov c1424fc7c8 sanitizer: Fix endianness checks for gcc
Summary:
__BIG_ENDIAN__ and __LITTLE_ENDIAN__ are not supported by gcc, which
eg. for ubsan Value::getFloatValue will silently fall through to
the little endian branch, breaking display of float values by ubsan.
Use __BYTE_ORDER__ == __ORDER_BIG/LITTLE_ENDIAN__ as the condition
instead, which is supported by both clang and gcc.

Noticed while porting ubsan to s390x.

Patch by Marcin Kościelnicki!

Differential Revision: http://reviews.llvm.org/D17660

llvm-svn: 263077
2016-03-09 23:39:40 +00:00
Filipe Cabecinhas b08c76f5f8 [cmake] Address Samsonov's post-commit review of r262723
Reviewers: samsonov

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D17896

llvm-svn: 262770
2016-03-05 10:01:04 +00:00
Alexey Samsonov 4362a994d2 [TSan] Fix compiler warning in Go sanity test.
llvm-svn: 262680
2016-03-04 00:56:15 +00:00
Dmitry Vyukov a029b79e1a tsan: describe heap/data locations in Go
llvm-svn: 262343
2016-03-01 15:38:12 +00:00
Dmitry Vyukov 7f022ae4c2 tsan: revert r262037
Broke aarch64 and darwin bots.

llvm-svn: 262046
2016-02-26 18:26:48 +00:00
Dmitry Vyukov b8868b9bea tsan: split thread into logical and physical state
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 262037
2016-02-26 16:57:14 +00:00
Maxim Ostapenko 7389936f57 [sanitizer] Move recvmsg and recv interceptors to sanitizer_common.
This patch moves recv and recvfrom interceptors from MSan and TSan to
sanitizer_common to enable them in ASan.

Differential Revision: http://reviews.llvm.org/D17479

llvm-svn: 261841
2016-02-25 08:44:25 +00:00
Dmitry Vyukov 0c202893ae tsan: clean up code after r261658
llvm-svn: 261660
2016-02-23 17:58:23 +00:00
Dmitry Vyukov b2be098026 tsan: fix signal handling in ignored libraries
The first issue is that we longjmp from ScopedInterceptor scope
when called from an ignored lib. This leaves thr->in_ignored_lib set.
This, in turn, disables handling of sigaction. This, in turn,
corrupts tsan state since signals delivered asynchronously.
Another issue is that we can ignore synchronization in asignal
handler, if the signal is delivered into an IgnoreSync region.
Since signals are generally asynchronous, they should ignore
memory access/synchronization/interceptor ignores.
This could lead to false positives in signal handlers.

llvm-svn: 261658
2016-02-23 17:16:26 +00:00
Richard Trieu ec3d332748 Change comma to semi-colon, no functionality change.
Cleanup for upcoming Clang warning -Wcomma.

llvm-svn: 261269
2016-02-18 22:07:42 +00:00
Ed Maste 371310e1ae [tsan] Disable sysroot flag on FreeBSD
FreeBSD does not install a number of Clang-provided headers for the
compiler in the base system due to incompatibilities between FreeBSD's     
and Clang's versions. As a workaround do not use --sysroot=. on FreeBSD    
until this is addressed.

llvm.org/pr26651
Differential Revision:	http://reviews.llvm.org/D17383

llvm-svn: 261229
2016-02-18 17:35:30 +00:00
Ed Maste cef252ea4c [tsan] Fix signal number definitions for FreeBSD
The change in r253983 for OS X also applies to FreeBSD.

llvm-svn: 261121
2016-02-17 18:25:27 +00:00
Ed Maste a36d555c33 [tsan] Fix build warnings on FreeBSD
The change in r252165 for OS X applies to FreeBSD as well.

llvm-svn: 261120
2016-02-17 18:22:50 +00:00
Dimitry Andric 042c4858b0 Fix lib/tsan/go/buildgo.sh for FreeBSD after r243051.
FreeBSD also needs to have sanitizer_linux_libcdep.cc included,
otherwise linking will fail with "undefined reference to
`__sanitizer::GetRSS()'".

While here, tabify the FreeBSD part, similar to the other parts.

llvm-svn: 260839
2016-02-14 00:26:32 +00:00
Alexey Samsonov 5b79ede65a [TSan] Adjust expectation for check_analyze.sh
r260695 caused extra push/pop instruction pair in __tsan_read1
implementation. Still, that change in InstCombine is believed to
be good, as it reduces the number of instructions performed.

Adjust the expectations to match the newly generated code.

llvm-svn: 260775
2016-02-13 00:55:58 +00:00
Mohit K. Bhakkad 4199f3df29 [TSan] Fix PrintMatchedSuppressions: Read hit count for suppression atomically
Reviewers: dvyukov.
Subscribers: jaydeep, sagar, dsanders, llvm-commits.
Differential Revision: http://reviews.llvm.org/D16845

llvm-svn: 259755
2016-02-04 05:28:48 +00:00
Daniel Sanders 2ec482a347 Re-commit r259512: [tsan] Add a libc++ and lit testsuite for each ${TSAN_SUPPORTED_ARCH}.
Summary:
This is a workaround to a problem in the 3.8 release that affects MIPS and
possibly other targets where the default is not supported but a sibling is
supported.

When TSAN_SUPPORTED_ARCH is not empty, cmake currently attempts to build a
tsan'd libcxx as well as test tsan for the default target regardless of whether
the default target is supported or not. This causes problems on MIPS32 since
tsan is supported for MIPS64 but not MIPS32.

This patch causes cmake to only build the libcxx and run the lit test-suite for
archictures in ${TSAN_SUPPORTED_ARCH}

This re-commit fixes an issue where 'check-tsan' continued to look for the
tsan'd libc++ in the directory it used to be built in.

Reviewers: hans, samsonov

Subscribers: tberghammer, llvm-commits, danalbert, srhines, dvyukov

Differential Revision: http://reviews.llvm.org/D16685

llvm-svn: 259542
2016-02-02 18:43:53 +00:00
Daniel Sanders bac108ac5f Revert r259512 - [tsan] Add a libc++ and lit testsuite for each ${TSAN_SUPPORTED_ARCH}.
check-tsan does not pick up the correct libc++.so. It succeeded on my machine
by picking up the libc++.so that was built before making this change.

llvm-svn: 259519
2016-02-02 16:05:52 +00:00
Daniel Sanders 470c2acc7c [tsan] Add a libc++ and lit testsuite for each ${TSAN_SUPPORTED_ARCH}.
Summary:
This is a workaround to a problem in the 3.8 release that affects MIPS and
possibly other targets where the default is not supported but a sibling is
supported.

When TSAN_SUPPORTED_ARCH is not empty, cmake currently attempts to build a
tsan'd libcxx as well as test tsan for the default target regardless of whether
the default target is supported or not. This causes problems on MIPS32 since
tsan is supported for MIPS64 but not MIPS32.

This patch causes cmake to only build the libcxx and run the lit test-suite for
archictures in ${TSAN_SUPPORTED_ARCH}

Reviewers: hans, samsonov

Subscribers: tberghammer, llvm-commits, danalbert, srhines, dvyukov

Differential Revision: http://reviews.llvm.org/D16685

llvm-svn: 259512
2016-02-02 15:04:35 +00:00
Nico Weber 72f564f818 Fix another -Wexpansion-to-defined warning in compiler-rt.
llvm-svn: 258202
2016-01-19 22:07:55 +00:00
Dmitry Vyukov f8f2d46002 tsan: add back __tls_get_addr interceptor
Removal of the interceptor broke libtsan interface in gcc:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68824
Add back a simple interceptor.

llvm-svn: 258119
2016-01-19 12:28:00 +00:00
Yabin Cui f7ebaf889a [tsan] Do nothing in ScopedInterceptor's destructor if thr is not inited.
Summary:
It is part of http://reviews.llvm.org/D15301, but missed when I committed
that patch.

Reviewers: kubabrecka, kcc, eugenis, llvm-commits, dvyukov

Differential Revision: http://reviews.llvm.org/D16235

llvm-svn: 258021
2016-01-17 21:33:42 +00:00
Yabin Cui 19e8c0e2f7 [tsan] Fix some tiny errors.
Summary:
1. Fix spell error for sigaction.
2. Make line_length <= 80.

Reviewers: llvm-commits, eugenis, kcc, dvyukov

Subscribers: tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D16210

llvm-svn: 257872
2016-01-15 06:21:46 +00:00
Yabin Cui bd3a772e9f [tsan] Store the pointer to ThreadState in TLS slot on Android.
Summary:
1. Android doesn't support __thread keyword. So allocate ThreadState
dynamically and store its pointer in one TLS slot provided by Android.
2. On Android, intercepted functions can be called before ThreadState
is initialized. So add test of thr_->is_inited in some places.
3. On Android, intercepted functions can be called after ThreadState
is destroyed. So add a fake dead_thread_state to represent all
destroyed ThreadStates. And that is also why we don't store the pointer
to ThreadState in shadow memory of pthread_self().

Reviewers: kcc, eugenis, dvyukov

Subscribers: kubabrecka, llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15301

llvm-svn: 257866
2016-01-15 03:39:04 +00:00
Yabin Cui 66e5db99b3 [tsan] Use internal_sigfillset to replace REAL(sigfillset).
Summary:
Android doesn't intercept sigfillset, so REAL(sigfillset) is null.
And we can use internal_sigfillset() for all cases.

Reviewers: kcc, eugenis, kubabrecka, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert

Differential Revision: http://reviews.llvm.org/D15296

llvm-svn: 257862
2016-01-15 02:59:23 +00:00
Sumanth Gundapaneni b76bf106b1 Fix the cross compilation of unit tests. NFC (second attempt)
With COMPILER_RT_INCLUDE_TESTS turned ON and in a cross compiling
environment, the unit tests fail to link. This patch does the following changes

>Rename COMPILER_RT_TEST_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS to reflect the 
way it's used.
>Add COMPILER_RT_TEST_COMPILER_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS so 
that cross-compiler would be able to build/compile the unit tests
>Add COMPILER_RT_UNITTEST_LINKFLAGS to COMPILER_RT_UNITTEST_CFLAGS so 
that cross-compiler would be able to link the unit tests (if needed)

Differential Revision: http://reviews.llvm.org/D16165

llvm-svn: 257783
2016-01-14 18:18:49 +00:00
Kuba Brecka 0626dd0d3b [tsan] Introduce a "ignore_interceptors_accesses" option
On OS X, TSan already passes all unit and lit tests, but for real-world applications (even very simple ones), we currently produce a lot of false positive reports about data races. This makes TSan useless at this point, because the noise dominates real bugs. This introduces a runtime flag, "ignore_interceptors_accesses", off by default, which makes TSan ignore all memory accesses that happen from interceptors. This will significantly lower the coverage and miss a lot of bugs, but it eliminates most of the current false positives on OS X.

Differential Revision: http://reviews.llvm.org/D15189

llvm-svn: 257760
2016-01-14 12:24:37 +00:00
Kuba Brecka 9f5fde7081 [tsan] Fix the value of PTHREAD_MUTEX_RECURSIVE for OS X and FreeBSD
The value of the constant PTHREAD_MUTEX_RECURSIVE is not "1" on FreeBSD and OS X.

Differential Revision: http://reviews.llvm.org/D16075

llvm-svn: 257758
2016-01-14 12:12:58 +00:00
Hans Wennborg 7b9d2b6c87 Revert r257686 "With COMPILER_RT_INCLUDE_TESTS turned ON and in a cross compiling"
This broke the build. For example, from
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/1191/steps/cmake%20stage%201/logs/stdio:

	-- Compiler-RT supported architectures: aarch64
	CMake Error at projects/compiler-rt/cmake/Modules/AddCompilerRT.cmake:170 (string):
		string sub-command REPLACE requires at least four arguments.
	Call Stack (most recent call first):
		projects/compiler-rt/lib/CMakeLists.txt:4 (include)

llvm-svn: 257694
2016-01-13 22:50:24 +00:00
Sumanth Gundapaneni 2d2f2b5c80 With COMPILER_RT_INCLUDE_TESTS turned ON and in a cross compiling
environment, the unit tests fail to link. This patch does the following changes

>Rename COMPILER_RT_TEST_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS to reflect the 
way it's used.
>Add COMPILER_RT_TEST_COMPILER_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS so that 
cross-compiler would be able to build/compile the unit tests
>Add COMPILER_RT_UNITTEST_LINKFLAGS to COMPILER_RT_UNITTEST_CFLAGS so that 
cross-compiler would be able to link the unit tests (if needed)

Differential Revision:http://reviews.llvm.org/D15082

llvm-svn: 257686
2016-01-13 22:09:47 +00:00
Kostya Serebryany abc2c998d2 [tsan] don't crash on closedir(0)
llvm-svn: 257223
2016-01-08 22:48:19 +00:00
Adhemerval Zanella b2910ea67e [compiler-rt] [tsan] Add support for PIE build on AArch64
This patch adds PIE executable support for aarch64-linux.  It adds
two more segments:

- 0x05500000000-0x05600000000: 39-bits PIE program segments
- 0x2aa00000000-0x2ab00000000: 42-bits PIE program segments

Fortunately it is possible to use the same transformation formula for
the new segments range with some adjustments in shadow to memory
formula (it adds a constant offset based on the VMA size).

A simple testcase is also added, however it is disabled on x86 due the
fact it might fail on newer kernels [1].

[1] https://git.kernel.org/linus/d1fd836dcf00d2028c700c7e44d2c23404062c90

llvm-svn: 256184
2015-12-21 19:27:57 +00:00
Kuba Brecka 3aac36ad01 Revert r255996 ("[tsan] Add a DCHECK to verify __tsan_read* and __tsan_write function aren't called from ScopedInterceptor").
There are some test failures on the Linux buildbots.

llvm-svn: 255997
2015-12-18 13:52:08 +00:00
Kuba Brecka 15dd456236 [tsan] Add a DCHECK to verify __tsan_read* and __tsan_write function aren't called from ScopedInterceptor
Interceptors using ScopedInteceptor should never call into user's code before the ScopedInterceptor is out of scope (and its destructor is called). Let's add a DCHECK to enforce that.

Differential Revision: http://reviews.llvm.org/D15381

llvm-svn: 255996
2015-12-18 13:08:15 +00:00
Kuba Brecka f9cc9d7392 [tsan] Fix scoping of ScopedInteceptor in libdispatch support
Some interceptors in tsan_libdispatch_mac.cc currently wrongly use TSAN_SCOPED_INTERCEPTOR/ScopedInterceptor. Its constructor can start ignoring memory accesses, and the destructor the stops this -- however, e.g. dispatch_sync can call user's code, so the ignoring will extend to user's code as well. This is not expected and we should only limit the scope of ScopedInterceptor to TSan code.  This patch introduces annotations that mark the beginning and ending of a callback into user's code.

Differential Revision: http://reviews.llvm.org/D15419

llvm-svn: 255995
2015-12-18 11:16:24 +00:00
Bill Schmidt e193989c17 [TSAN, PPC64] Fix obvious typo of supported virtual memory sizes
llvm-svn: 255507
2015-12-14 16:26:00 +00:00
Kuba Brecka 2cdb522a5a [tsan] Update dispatch_group support to avoid using a disposed group object
We're using the dispatch group itself to synchronize (to call Release() and Acquire() on it), but in dispatch group notifications, the group can already be disposed/deallocated. This causes a later assertion failure at `DCHECK_EQ(*meta, 0);` in `MetaMap::AllocBlock` when the same memory is reused (note that the failure only happens in debug builds).

Fixing this by retaining the group and releasing it in the notification. Adding a stress test case that reproduces this.

Differential Revision: http://reviews.llvm.org/D15380

llvm-svn: 255494
2015-12-14 13:32:57 +00:00
Alexey Samsonov c63c0d1cc0 [TSan] Try harder to avoid compiler-generated memcpy calls.
check_memcpy test added in r254959 fails on some configurations due to
memcpy() calls inserted by Clang. Try harder to avoid them by using
internal_memcpy() where applicable.

llvm-svn: 255287
2015-12-10 19:24:27 +00:00
Chris Bieneman 5c64ddfa88 Revert "[CMake] Provide options for toggling on and off various runtime libraries."
This reverts r255170. This change caused a bunch of bot failures and needs to be revised.

llvm-svn: 255184
2015-12-10 00:40:58 +00:00
Chris Bieneman e82ad46f15 [CMake] Provide options for toggling on and off various runtime libraries.
Summary:
Rather than having to add new "experimental" options each time someone wants to work on bringing a sanitizer to a new platform, this patch makes options for all of them.

The default values for the options are set by the platform checks that would have enabled them, but they can be overridden on or off.

Reviewers: kubabrecka, samsonov

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D14846

llvm-svn: 255170
2015-12-09 22:45:03 +00:00
Yabin Cui 7f48808882 [tsan] Move emptyset/oldset to ThreadSignalContext.
Summary:
Android doesn't support __thread keyword. So move emptyset/oldset
from THREADLOCAL to ThreadSignalContext.

Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert

Differential Revision: http://reviews.llvm.org/D15299

llvm-svn: 255168
2015-12-09 22:40:31 +00:00
Yabin Cui 4ddbe8508a [tsan] Use REAL(malloc) instead of __libc_malloc for Android.
Summary:
Android doesn't have __libc_malloc and related allocation
functions. As its dynamic linker doesn't use malloc, so
we can use REAL(malloc) to replace __libc_malloc safely.

Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15297

llvm-svn: 255167
2015-12-09 22:32:38 +00:00
Yabin Cui 903ab8a2a3 [tsan] Disable interceptors not supported in Android.
Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15295

llvm-svn: 255164
2015-12-09 22:23:47 +00:00
Alexey Samsonov 9c54984d43 [TSan] Try harder to avoid compiler-generated memset calls.
check_memcpy test added in r254959 fails on some configurations due to
memset() calls inserted by Clang. Try harder to avoid them:
* Explicitly use internal_memset() instead of empty braced-initializer.
* Replace "new T()" with "new T", as the former generates zero-initialization
  for structs in C++11.

llvm-svn: 255136
2015-12-09 18:48:10 +00:00
Yabin Cui d28ffacc99 [tsan] Define sigaction_t for Android.
Reviewers: kcc, eugenis, dvyukov

Subscribers: llvm-commits, tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D15298

llvm-svn: 255135
2015-12-09 18:37:27 +00:00
Bill Schmidt 4514a87c4d Avoid extended mnemonic 'mfvrsave' in assembly code
llvm-svn: 255116
2015-12-09 15:18:30 +00:00
Bill Schmidt 6b63d576c3 [PPC64, TSAN] Provide setjmp interceptor support for PPC64
This patch provides the assembly support for setjmp/longjmp for use
with the thread sanitizer.  This is a big more complicated than for
aarch64, because sibcalls are only legal under our ABIs if the TOC
pointer is unchanged.  Since the true setjmp function trashes the TOC
pointer, and we have to leave the stack in a correct state, we emulate
the setjmp function rather than branching to it.

We also need to materialize the TOC for cases where the _setjmp code
is called from libc.  This is done differently under the ELFv1 and
ELFv2 ABIs.

llvm-svn: 255059
2015-12-08 22:14:34 +00:00
Bill Schmidt 2979162732 [PPC64, TSAN] LLVM basic enablement of thread sanitizer for PPC64 (BE and LE)
This patch is by Simone Atzeni with portions by Adhemerval Zanella.

This contains the LLVM patches to enable the thread sanitizer for
PPC64, both big- and little-endian.  Two different virtual memory
sizes are supported:  Old kernels use a 44-bit address space, while
newer kernels require a 46-bit address space.

There are two companion patches that will be added shortly.  There is
a Clang patch to actually turn on the use of the thread sanitizer for
PPC64.  There is also a patch that I wrote to provide interceptor
support for setjmp/longjmp on PPC64.

Patch discussion at reviews.llvm.org/D12841.

llvm-svn: 255057
2015-12-08 21:54:39 +00:00
Alexey Samsonov 02824d0e59 [TSan] Remove legacy Makefile.old!
Summary:
It was barely supported for a several years for now, somewhat
rotten and doesn't correspond to the way we build/test TSan runtime
in Clang anymore.

CMake build has proper compile flags, library layout, build
dependencies etc.

Shell scripts that depended on the output of Makefile.old are
either obsolete now (check_cmake.sh), or moved to lit tests
(check_memcpy.sh), or kept as a standalone scripts not suitable
for generic test suite, but invoked on bots (check_analyze.sh).

It is not used on bots anymore: all "interesting" configurations
(gcc/clang as a host compiler; debug/release build types) are now
tested via CMake.

Reviewers: dvyukov, kcc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D15316

llvm-svn: 255032
2015-12-08 17:59:33 +00:00
Alexey Samsonov f49d159c8a [CMake] Remove debug output leftovers.
llvm-svn: 255031
2015-12-08 17:59:30 +00:00
Dmitry Vyukov eee690b29a tsan: fix test invisible barrier
Another attempt at fixing tsan_invisible_barrier.
Current implementation causes:
https://llvm.org/bugs/show_bug.cgi?id=25643

There were several unsuccessful iterations for this functionality:

Initially it was implemented in user code using REAL(pthread_barrier_wait). But pthread_barrier_wait is not supported on MacOS. Futexes are linux-specific for this matter.
Then we switched to atomics+usleep(10). But usleep produced parasitic "as-if synchronized via sleep" messages in reports which failed some output tests.
Then we switched to atomics+sched_yield. But this produced tons of tsan- visible events, which lead to "failed to restore stack trace" failures.
Move implementation into runtime and use internal_sched_yield in the wait loop.
This way tsan should see no events from the barrier, so not trace overflows and
no "as-if synchronized via sleep" messages.

llvm-svn: 255030
2015-12-08 17:54:47 +00:00
Kuba Brecka 25dba9b781 [tsan] Add dispatch_group API interceptors and synchronization
This patch adds release and acquire semantics for dispatch groups, plus a test case.

Differential Revision: http://reviews.llvm.org/D15048

llvm-svn: 255020
2015-12-08 14:54:43 +00:00
Alexey Samsonov d20a13023b [TSan] Enforce TSan runtime doesn't include system headers with --sysroot flag.
llvm-svn: 254966
2015-12-07 23:21:36 +00:00
Alexey Samsonov dc37ee2226 [TSan] Port check_memcpy.sh script to a regular lit test.
Check that TSan runtime doesn't contain compiler-inserted calls
to memset/memmove functions.

In future, we may consider moving this test to test/sanitizer_common,
as we don't want to have compiler-inserted memcpy/memmove calls in
any sanitizer runtime.

llvm-svn: 254955
2015-12-07 21:53:59 +00:00
Alexey Samsonov a8bf5099d6 [TSan] Slightly improve check_analyze script.
De-hardcode path to TSan-ified executable: pass it as an input to
the scripts. Fix them so that they don't write to the current
directory. Remove their invocation from Makefile.old: they are
broken there anyway, as check_analyze.sh now matches trunk Clang.

llvm-svn: 254936
2015-12-07 20:18:50 +00:00
Alexey Samsonov 5d47be186d [TSan] Delete legacy test_output.sh script.
This script is superseded by lit test suite integrated into CMake
for quite a while now. It doesn't support many tests, and require
custom hacks for a few other.

llvm-svn: 254932
2015-12-07 19:40:33 +00:00
Alexey Samsonov a330293af7 [CMake] Add COMPILER_RT_TSAN_DEBUG_OUTPUT option.
This option builds TSan runtime with extra debug printfs
and stats collection. This build configuration is developer-only
and should rarely be used, but we need to keep it to make sure
it doesn't bitrot.

llvm-svn: 254818
2015-12-05 01:37:17 +00:00
Alexander Potapenko a75f826117 [TSan] Fix the wikipage link.
llvm-svn: 254736
2015-12-04 17:38:47 +00:00
Alexander Potapenko 6f3fa20c76 [ASan] Retire mac_ignore_invalid_free, remove some dead code.
mac_ignore_invalid_free was helpful when ASan runtime used to intercept
CFAllocator and sometimes corrupted its memory. This behavior had been long
gone, and the flag was unused.
This patch also deletes ReportMacCfReallocUnknown(), which was used by the
CFAllocator realloc() wrapper.

llvm-svn: 254722
2015-12-04 16:17:55 +00:00
Kuba Brecka 87cd9c63df [tsan] Enable building and testing TSan Go runtime on OS X
The build and test actually work now, so let's just enable them.

Differential Revision: http://reviews.llvm.org/D15184

llvm-svn: 254716
2015-12-04 14:29:11 +00:00
Kuba Brecka 0423e5cd57 [tsan] Add interceptors for Darwin-specific locking APIs
On OS X, there are other-than-pthread locking APIs that are used quite extensively - OSSpinLock and os_lock_lock. Let's add interceptors for those.

Differential Revision: http://reviews.llvm.org/D14987

llvm-svn: 254611
2015-12-03 15:10:52 +00:00
Kuba Brecka 65aa45e872 [tsan] Use re-exec method to enable interceptors on older versions of OS X
In AddressSanitizer, we have the MaybeReexec method to detect when we're running without DYLD_INSERT_LIBRARIES (in which case interceptors don't work) and re-execute with the environment variable set. On OS X 10.11+, this is no longer necessary, but to have ThreadSanitizer supported on older versions of OS X, let's use the same method as well. This patch moves the implementation from `asan/` into `sanitizer_common/`.

Differential Revision: http://reviews.llvm.org/D15123

llvm-svn: 254600
2015-12-03 10:39:43 +00:00
Kuba Brecka ac5f5d10d5 [tsan] Add interceptors and sychronization for libdispatch semaphores on OS X
This patch adds release and acquire semantics for libdispatch semaphores and a test case.

Differential Revision: http://reviews.llvm.org/D14992

llvm-svn: 254412
2015-12-01 13:11:42 +00:00
Kuba Brecka 77ff411247 [tsan] Fix signals and setjmp/longjmp on OS X
1) There's a few wrongly defined things in tsan_interceptors.cc,
2) a typo in tsan_rtl_amd64.S which calls setjmp instead of sigsetjmp in the interceptor, and
3) on OS X, accessing an mprotected page results in a SIGBUS (and not SIGSEGV).

Differential Revision: http://reviews.llvm.org/D15052

llvm-svn: 254299
2015-11-30 14:11:02 +00:00
Kuba Brecka 6fd0675925 [tsan] Fix weakly imported functions on OS X
On OS X, for weak function (that user can override by providing their own implementation in the main binary), we need extern `"C" SANITIZER_INTERFACE_ATTRIBUTE SANITIZER_WEAK_ATTRIBUTE NOINLINE`.

Fixes a broken test case on OS X, java_symbolization.cc, which uses a weak function __tsan_symbolize_external.

Differential Revision: http://reviews.llvm.org/D14907

llvm-svn: 254298
2015-11-30 13:27:18 +00:00
Kuba Brecka e14261a4c9 Fixup for r254228 ("Port tsan_rtl_amd64.S to OS X to add support for setjmp/longjmp") to fix the build of unit tests. We need to add the ASM file into RTTsan_dynamic as well.
llvm-svn: 254249
2015-11-29 08:48:47 +00:00
Kuba Brecka 8f9607a616 [tsan] Add release+acquire semantics for serial dispatch queues
Serial queues need extra happens-before between individual tasks executed in the same queue. This patch adds `Acquire(queue)` before the executed task and `Release(queue)` just after it (for serial queues only). Added a test case.

Differential Revision: http://reviews.llvm.org/D15011

llvm-svn: 254229
2015-11-28 12:53:57 +00:00
Kuba Brecka f94bb99adc [tsan] Port tsan_rtl_amd64.S to OS X to add support for setjmp/longjmp
This patch ports the assembly file tsan_rtl_amd64.S to OS X, where we need several changes:

* Some assembler directives are not available on OS X (.hidden, .type, .size)
* Symbol names need to start with an underscore (added a ASM_TSAN_SYMBOL macro for that).
* To make the interceptors work, we ween to name the function "_wrap_setjmp" (added ASM_TSAN_SYMBOL_INTERCEPTOR for that).
* Calling the original setjmp is done with a simple "jmp _setjmp".
* __sigsetjmp doesn't exist on OS X.

Differential Revision: http://reviews.llvm.org/D14947

llvm-svn: 254228
2015-11-28 12:44:23 +00:00
Kuba Brecka 11d8c1f8dd [tsan] Fix an assertion failure in FindThreadByUidLocked with recycled threads
When a race on file descriptors is detected, `FindThreadByUidLocked()` is called to retrieve ThreadContext with a specific unique_id. However, this ThreadContext might not exist in the thread registry anymore (it may have been recycled), in which case `FindThreadByUidLocked` will cause an assertion failure in `GetThreadLocked`. Adding a test case that reproduces this, producing:

    FATAL: ThreadSanitizer CHECK failed: sanitizer_common/sanitizer_thread_registry.h:92 "((tid)) < ((n_contexts_))" (0x34, 0x34)

This patch fixes this by replacing the loop with `FindThreadContextLocked`.

Differential Revision: http://reviews.llvm.org/D14984

llvm-svn: 254223
2015-11-28 09:16:34 +00:00
Adhemerval Zanella 066c5f0f92 [compiler-rt] [tsan] Unify aarch64 mapping
This patch unify the 39 and 42-bit support for AArch64 by using an external
memory read to check the runtime detected VMA and select the better mapping
and transformation.  Although slower, this leads to same instrumented binary
to be independent of the kernel.

Along with this change this patch also fix some 42-bit failures with
ALSR disable by increasing the upper high app memory threshold and also
the 42-bit madvise value for non large page set.

llvm-svn: 254151
2015-11-26 13:10:47 +00:00
Kuba Brecka 8443d187e6 [tsan] Fix signal number definitions for OS X
On OS X, SIGBUS is 10 and SIGSYS is 12.

Differential Revision: http://reviews.llvm.org/D14946

llvm-svn: 253983
2015-11-24 13:44:54 +00:00
Kuba Brecka 440d08600b [tsan] Implement basic GCD interceptors for OS X
We need to intercept libdispatch APIs (dispatch_sync, dispatch_async, etc.) to add synchronization between the code that submits the task and the code that gets executed (possibly on a different thread). This patch adds release+acquire semantics for dispatch_sync, and dispatch_async (plus their "_f" and barrier variants). The synchronization is done on malloc'd contexts (separate for each submitted block/callback). Added tests to show usage of dispatch_sync and dispatch_async, for cases where we expect no warnings and for cases where TSan finds races.

Differential Revision: http://reviews.llvm.org/D14745

llvm-svn: 253982
2015-11-24 13:36:06 +00:00
Kuba Brecka 377cafbbbc [tsan] Fix CallocOverflow unit test on OS X
On OS X, interceptors don't work in unit tests, so calloc() calls the system allocator. We need to use user_calloc() instead.

Differential Revision: http://reviews.llvm.org/D14918

llvm-svn: 253979
2015-11-24 13:27:33 +00:00
Kuba Brecka 6bbb8519e2 [tsan] Fix __cxa_guard_* interceptors on OS X
This patch fixes the __cxa_guard_acquire, __cxa_guard_release and __cxa_guard_abort interceptors on OS X. They apparently work on Linux just by having the same name, but on OS X, we actually need to use TSAN_INTERCEPTOR.

Differential Revision: http://reviews.llvm.org/D14868

llvm-svn: 253776
2015-11-21 12:53:34 +00:00
Kuba Brecka 5cbe122439 [tsan] Fix detached threads in unit tests on OS X
We need to call the intercepted version of pthread_detach. Secondly, PTHREAD_CREATE_JOINABLE and PTHREAD_CREATE_DETACHED are not 0 and 1 on OS X, so we need to properly pass these constants and not just a bool.

Differential Revision: http://reviews.llvm.org/D14837

llvm-svn: 253775
2015-11-21 12:41:36 +00:00
Kuba Brecka 4e6adaac4c [tsan] Make tsan_test_util_posix.cc call pthread interceptors on OS X
The tsan_test_util_posix.cc implementation of mutexes call pthread APIs directly, which on OS X don't end up calling the intercepted versions and we miss the synchronization. This patch changes the unit tests to directly call the intercepted versions. This fixes several test failures on OS X.

Differential Revision: http://reviews.llvm.org/D14835

llvm-svn: 253774
2015-11-21 12:38:54 +00:00
Anna Zaks 2ce9f9447c Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
of the report to stderr. This patch changes this behavior and writes the full
text of the report to syslog before we terminate the process. It also calls
os_trace (Activity Tracing available on OS X and iOS) with a message saying
that the report is available in syslog. This is useful, because this message
will be shown in the crash log.

For this to work, the patch makes sure we store the full report into
error_message_buffer unconditionally, and it also strips out ANSI escape
sequences from the report (they are used when producing colored reports).

I've initially tried to log to syslog during printing, which is done on Android
right now. The advantage is that if we crash during error reporting or the
produced error does not go through ScopedInErrorReport, we would still get a
(partial) message in the syslog. However, that solution is very problematic on
OS X. One issue is that the logging routine uses GCD, which may spawn a new
thread on its behalf. In many cases, the reporting logic locks threadRegistry,
which leads to deadlocks.

Reviewed at http://reviews.llvm.org/D13452

(In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
build failures on Linux.)

llvm-svn: 253688
2015-11-20 18:41:44 +00:00
Alexey Samsonov 3e49a7bc32 [TSan] Fix custom scripts that validate TSan on non-standard buildbot.
llvm-svn: 253682
2015-11-20 18:13:03 +00:00
Alexey Samsonov 145e4d6471 Tell clang-format that (most) sanitizers are written using Google style guide.
llvm-svn: 253608
2015-11-19 22:11:10 +00:00
Kuba Brecka c7157bfd55 [tsan] Replace new/delete with a local variable in ThreadSpecificDtors unit test
On OS X, this unit test (ThreadSpecificDtors) fails, because the new and delete operators actually call the overridden operators, which end up using TLVs and crash. Since C++'s new and delete is not important in this test, let's just replace them with a local variable. This fixes the test on OS X.

Differential Revision: http://reviews.llvm.org/D14826

llvm-svn: 253583
2015-11-19 17:50:07 +00:00
Kuba Brecka f503d51e5b [tsan] For OS X thread finalization, remove g_thread_finalize_key in favor of libpthread hooks
On OS X, the thread finalization is fragile due to thread-local variables destruction order. I've seen cases where the we destroy the ThreadState too early and subsequent thread-local values' destructors call interceptors again. Let's replace the TLV-based thread finalization method with libpthread hooks. The notification PTHREAD_INTROSPECTION_THREAD_TERMINATE is called *after* all TLVs have been destroyed.

Differential Revision: http://reviews.llvm.org/D14777

llvm-svn: 253560
2015-11-19 12:06:20 +00:00
Kuba Brecka 9eb6df9f1e [tsan] Recognize frames coming from "libclang_rt.tsan_*" module as internal
On OS X, we build a dylib of the TSan runtime, which doesn't necessarily need to contain debugging symbols (and file and line information), so llvm-symbolizer might not be able to find file names for TSan internal frames.  FrameIsInternal currently only considers filenames, but we should simply treat all frames within `libclang_rt.tsan_osx_dynamic.dylib` as internal.  This patch treats all modules starting with `libclang_rt.tsan_` as internal, because there may be more runtimes for other platforms in the future.

Differential Revision: http://reviews.llvm.org/D14813

llvm-svn: 253559
2015-11-19 12:03:48 +00:00
Kuba Brecka efd92b3d16 [tsan] Handle dispatch_once on OS X
Reimplement dispatch_once in an interceptor to solve these issues that may produce false positives with TSan on OS X:

1) there is a racy load inside an inlined part of dispatch_once,
2) the fast path in dispatch_once doesn't perform an acquire load, so we don't properly synchronize the initialization and subsequent uses of whatever is initialized,
3) dispatch_once is already used in a lot of already-compiled code, so TSan doesn't see the inlined fast-path.

This patch uses a trick to avoid ever taking the fast path (by never storing ~0 into the predicate), which means the interceptor will always be called even from already-compiled code. Within the interceptor, our own atomic reads and writes are not written into shadow cells, so the race in the inlined part is not reported (because the accesses are only loads).

Differential Revision: http://reviews.llvm.org/D14811

llvm-svn: 253552
2015-11-19 10:35:35 +00:00
Kuba Brecka b2b07aa925 [tsan] Skip malloc/free interceptors when we're inside symbolizer on OS X
Symbolizers can call malloc/realloc/free/..., which we don't want to intercept. This is already implemented on Linux, let's do it for OS X as well.

Differential Revision: http://reviews.llvm.org/D14710

llvm-svn: 253460
2015-11-18 14:50:40 +00:00
Alexey Samsonov 1f4cb4d3d4 [TSan] List only amd64 asm file in Makefile.old (attempt 2)
llvm-svn: 253416
2015-11-18 00:56:37 +00:00
Alexey Samsonov 3408ea64ed [TSan] List only amd64 asm file in Makefile.old
llvm-svn: 253407
2015-11-18 00:31:00 +00:00
Dmitry Vyukov 3dd7fe6d86 tsan: replace macro check with constant check
As per comments in 252892 commit.

llvm-svn: 253216
2015-11-16 15:55:38 +00:00
Adhemerval Zanella 2f7f5e3535 [compiler-rt] [tsan] Enable intercept setjmp/longjmp for AArch64
This patch adds assembly routines to enable setjmp/longjmp for aarch64
on linux.  It fixes:

 * test/tsan/longjmp2.cc
 * test/tsan/longjmp3.cc
 * test/tsan/longjmp4.cc
 * test/tsan/signal_longjmp.cc

I also checked with perlbench from specpu2006 (it fails to run
with missing setjmp/longjmp intrumentation).

llvm-svn: 253205
2015-11-16 13:55:19 +00:00
Kuba Brecka 7291b88e9a [tsan] Fix finalization of detached threads on OS X
Currently, we crash on finalization of detached threads, because we'll try to clear the ThreadState twice.

Differential Revision: http://reviews.llvm.org/D14644

llvm-svn: 253079
2015-11-13 20:45:27 +00:00
Kuba Brecka d07e069d79 [tsan] Fix a typo in tsan_test_util.h
There is a typo in tsan_test_util.h, it shouldn't be `APPLE`, but `__APPLE__`.

Differential Revision: http://reviews.llvm.org/D14647

llvm-svn: 253078
2015-11-13 20:42:57 +00:00
Kuba Brecka 3ecce92a52 [tsan] Allow symbolizers that don't obtain global symbol sizes
The default symbolizer, `llvm-symbolizer` provides sizes for global symbols. On OS X, we want to also allow using `atos` (because it's available everywhere and users don't need to copy/install it) and `dladdr` (it's the only available option when running in a sandbox). However, these symbolizers do not supply the symbol sizes, only names and starting addresses. This patch changes the reporting functions to hide the size of the symbol when this value is unavailable, and modifies tests to make this part of the report "optional".

Differential Revision: http://reviews.llvm.org/D14608

llvm-svn: 252896
2015-11-12 14:34:17 +00:00
Dmitry Vyukov bab2216dbb tsan: disable abort_on_error for Go
It does not work as expected.
Go runtime handles SIGABRT and crashes with a loud message.

llvm-svn: 252892
2015-11-12 14:03:20 +00:00
Dmitry Vyukov 110094017c tsan: fix Go build on linux
librt is not linked in in Go build.

llvm-svn: 252877
2015-11-12 11:54:25 +00:00
Dmitry Vyukov 317e98b1a2 tsan: fix unused function warning in Go build
llvm-svn: 252875
2015-11-12 11:14:35 +00:00
Dmitry Vyukov 1421ed42a6 tsan: fix mac Go build
cur_thread does not exist in Go.

llvm-svn: 252874
2015-11-12 11:12:18 +00:00
Kuba Brecka 14fb164a8e [tsan] Filter OS X architectures for unit testing
The TSan unit test build currently fails if we're also building the iOS parts of compiler-rt, because `TSAN_SUPPORTED_ARCH` contains ARM64. For unit tests, we need to filter this only to host architecture(s).

Differential Revision: http://reviews.llvm.org/D14604

llvm-svn: 252873
2015-11-12 11:00:38 +00:00
Dmitry Vyukov 3ed274e3ce tsan: fix unused variable in Go build
llvm-svn: 252746
2015-11-11 16:09:20 +00:00
Kuba Brecka fcbb0ec06e [tsan] Rename tsan_test_util_linux.cc to tsan_test_util_posix.cc
Differential Revision: http://reviews.llvm.org/D14575

llvm-svn: 252741
2015-11-11 15:42:00 +00:00
Kuba Brecka c603d25a19 Fixing #include order in tsan_new_delete.cc. Follow-up commit for r252284.
llvm-svn: 252735
2015-11-11 15:15:14 +00:00
Kuba Brecka c47620a838 [tsan] Add TSan unit test support for OS X
This patch enables building and running TSan unit tests on OS X.

Differential Revision: http://reviews.llvm.org/D14546

llvm-svn: 252731
2015-11-11 14:53:57 +00:00
Ismail Pazarbasi fcb8c7e407 [tsan] Pass correct interposed function prefix to report function
Summary:
On Darwin, interposed functions are prefixed with "wrap_". On Linux,
they are prefixed with "__interceptor_".

Reviewers: dvyukov, samsonov, glider, kcc, kubabrecka

Subscribers: zaks.anna, llvm-commits

Differential Revision: http://reviews.llvm.org/D14512

llvm-svn: 252695
2015-11-11 02:38:38 +00:00
Kuba Brecka 3a618b0317 [tsan] Enable new/delete C++ interceptors for OS X
This patch adds `tsan_new_delete.cc` into the OS X build.

Differential Revision: http://reviews.llvm.org/D14424

llvm-svn: 252284
2015-11-06 11:07:26 +00:00
Kuba Brecka dccffd4fcc Trying to fix the FreeBSD build breakage due to r251916.
http://lab.llvm.org:8011/builders/sanitizer_x86_64-freebsd/builds/6395

llvm-svn: 252277
2015-11-06 08:16:24 +00:00
Kuba Brecka 537b6090d1 [tsan] Fix build warnings on OS X
Fixing `tsan_interceptors.cc`, which on OS X produces a bunch of warnings about unused constants and functions.

Differential Revision: http://reviews.llvm.org/D14381

llvm-svn: 252165
2015-11-05 14:32:56 +00:00
Kuba Brecka 9e0cf38d9b Lint warning fixup for r252160 ("[tsan] Fix pthread_once interceptor for OS X").
llvm-svn: 252163
2015-11-05 14:16:42 +00:00
Kuba Brecka 245bcf9eb9 [tsan] Fix the memcpy interceptor to be memmove compatible on OS X
On OS X, memcpy and memmove are actually aliases of the same implementation, which means the interceptor of memcpy is also invoked when memmove is called. The current implementation of the interceptor uses `internal_memcpy` to perform the actual memory operation, which can produce an incorrect result when memmove semantics are expected. Let's call `internal_memmove` instead.

Differential Revision: http://reviews.llvm.org/D14336

llvm-svn: 252162
2015-11-05 14:03:26 +00:00
Kuba Brecka 3d8536240a [tsan] Allow memmove interceptor to be used when TSan is not initialized
A call to memmove is used early during new thread initialization on OS X. This patch uses the `COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED` check, similarly to how we deal with other early-used interceptors.

Differential Revision: http://reviews.llvm.org/D14377

llvm-svn: 252161
2015-11-05 14:01:53 +00:00
Kuba Brecka 12bba1c2a0 [tsan] Fix pthread_once interceptor for OS X
TSan has a re-implementation of `pthread_once` in its interceptor, which assumes that the `pthread_once_t *once_control` pointer is actually pointing to a "storage" which is zero-initialized and used for the atomic operations. However, that's not true on OS X, where pthread_once_t is a structure, that contains a header (with a magic value) and the actual storage follows after that. This patch skips the header to make the interceptor work on OS X.

Differential Revision: http://reviews.llvm.org/D14379

llvm-svn: 252160
2015-11-05 13:59:07 +00:00
Kuba Brecka cd18f28751 [tsan] Alternative ThreadState storage for OS X
This implements a "poor man's TLV" to be used for TSan's ThreadState on OS X. Based on the fact that `pthread_self()` is always available and reliable and returns a valid pointer to memory, we'll use the shadow memory of this pointer as a thread-local storage. No user code should ever read/write to this internal libpthread structure, so it's safe to use it for this purpose. We lazily allocate the ThreadState object and store the pointer here.

Differential Revision: http://reviews.llvm.org/D14288

llvm-svn: 252159
2015-11-05 13:54:50 +00:00
Kuba Brecka 40ad607be6 [tsan] Use malloc zone interceptors on OS X, part 2
TSan needs to use a custom malloc zone on OS X, which is already implemented in ASan.  This patch uses the sanitizer_common implementation in `sanitizer_malloc_mac.inc` for TSan as well.

Reviewed at http://reviews.llvm.org/D14330

llvm-svn: 252155
2015-11-05 10:31:21 +00:00
Alexey Samsonov e5739981d5 [TSan] Fix mmap/mmap64 interceptor signature.
mmap() offset argument has type off_t, not unsigned. off_t is usually
64-bit on 64-bit Linux.

llvm-svn: 252103
2015-11-05 00:00:36 +00:00
Juergen Ributzka d7dcffaea4 Revert "Reapply: [asan] On OS X, log reports to syslog and os_trace"
Looks like this commit is deadlocking the ASAN tests on the green dragon bot
(http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA/).

llvm-svn: 252076
2015-11-04 21:03:12 +00:00
Kuba Brecka a1186b828f [tsan] Handle libdispatch worker threads on OS X
On OS X, GCD worker threads are created without a call to pthread_create. We need to properly register these threads with ThreadCreate and ThreadStart. This patch uses a libpthread API (`pthread_introspection_hook_install`) to get notifications about new threads and about threads that are about to be destroyed.

Differential Revision: http://reviews.llvm.org/D14328

llvm-svn: 252049
2015-11-04 15:09:14 +00:00
Kuba Brecka 88bccf0344 [tsan] Shadow memory setup for OS X
Updating the shadow memory initialization in `tsan_platform_mac.cc` to also initialize the meta shadow and to mprotect the memory ranges that need to be avoided.

Differential Revision: http://reviews.llvm.org/D14324

llvm-svn: 252044
2015-11-04 11:39:40 +00:00
Kuba Brecka 127f669913 Fixup for r251923 to fix a warning about an extra semicolon.
llvm-svn: 251924
2015-11-03 14:45:32 +00:00
Kuba Brecka 149012153c Reapply r251916 ("[tsan] Port TSan interceptors on OS X").
llvm-svn: 251923
2015-11-03 14:43:39 +00:00
Kuba Brecka 9d71caa9a4 Revert r251916 ("[tsan] Port TSan interceptors on OS X").
llvm-svn: 251922
2015-11-03 14:42:03 +00:00
Kuba Brecka cf41c0c930 Reapply r251918 ("[tsan] Fix build errors for TSan on OS X").
llvm-svn: 251920
2015-11-03 14:33:39 +00:00
Kuba Brecka 646e1bef09 Revert r251918 ("[tsan] Fix build errors for TSan on OS X").
llvm-svn: 251919
2015-11-03 14:30:31 +00:00
Kuba Brecka 350a9aaeda [tsan] Fix build errors for TSan on OS X
This patch moves a few functions from `sanitizer_linux_libcdep.cc` to `sanitizer_posix_libcdep.cc` in order to use them on OS X as well. Plus a few more small build fixes.

This is part of an effort to port TSan to OS X, and it's one the very first steps. Don't expect TSan on OS X to actually work or pass tests at this point.

Differential Revision: http://reviews.llvm.org/D14235

llvm-svn: 251918
2015-11-03 14:26:00 +00:00
Kuba Brecka 0ca1e2c3f9 [tsan] Port TSan interceptors on OS X
This patch modifies `tsan_interceptors.cc` to be buildable on OS X. Several of the intercepted methods are not available on OS X, so we need to `#if !SANITIZER_MAC` them. Plus a few other fixes, e.g. `pthread_yield` doesn't exist, let's use `internal_sched_yield` instead.

This is part of an effort to port TSan to OS X, and it's one the very first steps. Don't expect TSan on OS X to actually work or pass tests at this point.

Differential Revision: http://reviews.llvm.org/D14237

llvm-svn: 251916
2015-11-03 14:22:51 +00:00
Kuba Brecka c9249498ce [tsan] CMake support for TSan on OS X
Hi, this patch adds a CMake flag called `COMPILER_RT_ENABLE_TSAN_OSX`, which is off by default. If enabled, the build system will be building the OS X version of the TSan runtime library (called `libclang_rt.tsan_osx_dynamic.dylib`). I'll submit patches that fix OS X build errors shortly.

This is part of an effort to port TSan to OS X, and it's one the very first steps. Don't expect TSan on OS X to actually work or pass tests at this point.

llvm-svn: 251915
2015-11-03 14:13:59 +00:00
Alexey Samsonov a49cfd8f94 Revert "Apply modernize-use-default to compiler-rt."
This reverts commit r250823.

Replacing at least some of empty
constructors with "= default" variants is a semantical change which we
don't want. E.g. __tsan::ClockBlock contains a union of large arrays,
and it's critical for correctness and performance that we don't memset()
these arrays in the constructor.

llvm-svn: 251717
2015-10-30 18:52:31 +00:00
Anna Zaks 7920eb0b1f Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
        of the report to stderr. This patch changes this behavior and writes the full
        text of the report to syslog before we terminate the process. It also calls
        os_trace (Activity Tracing available on OS X and iOS) with a message saying
        that the report is available in syslog. This is useful, because this message
        will be shown in the crash log.

        For this to work, the patch makes sure we store the full report into
        error_message_buffer unconditionally, and it also strips out ANSI escape
        sequences from the report (they are used when producing colored reports).

        I've initially tried to log to syslog during printing, which is done on Android
        right now. The advantage is that if we crash during error reporting or the
        produced error does not go through ScopedInErrorReport, we would still get a
        (partial) message in the syslog. However, that solution is very problematic on
        OS X. One issue is that the logging routine uses GCD, which may spawn a new
        thread on its behalf. In many cases, the reporting logic locks threadRegistry,
        which leads to deadlocks.

        Reviewed at http://reviews.llvm.org/D13452

        (In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
         build failures on Linux.)

llvm-svn: 251577
2015-10-28 23:18:44 +00:00