Commit Graph

65 Commits

Author SHA1 Message Date
Kuba Brecka 312071943c [tsan] On OS X, build Go runtime with -mmacosx-version-min
We're not building the Go runtime with -mmacosx-version-min, which means it'll have a minimum deployment target set to the system you're building on. Let's make the code compile (and link) with -mmacosx-version-min=10.7.

Differential Revision: http://reviews.llvm.org/D20670

llvm-svn: 271833
2016-06-05 15:15:36 +00:00
Kostya Serebryany d9a451ab0d [sanitizer] Move termination functions in their own file
Summary:
As suggested by kcc@ in http://reviews.llvm.org/D20084#441418, move the CheckFailed and Die functions, and their associated callback functionalities in their own separate file.
I expended the build rules to include a new rule that would not include those termination functions, so that another project can define their own.
The tests check-{a,t,m,ub,l,e,df}san are all passing.

Reviewers: llvm-commits, kcc

Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D20742

llvm-svn: 271055
2016-05-27 21:57:17 +00:00
Dmitry Vyukov d87c7b321a tsan: split thread into logical and physical state
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).

Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:

Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.

Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.

This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.

Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 267678
2016-04-27 08:23:02 +00:00
Alexey Samsonov 4362a994d2 [TSan] Fix compiler warning in Go sanity test.
llvm-svn: 262680
2016-03-04 00:56:15 +00:00
Dmitry Vyukov 7f022ae4c2 tsan: revert r262037
Broke aarch64 and darwin bots.

llvm-svn: 262046
2016-02-26 18:26:48 +00:00
Dmitry Vyukov b8868b9bea tsan: split thread into logical and physical state
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 262037
2016-02-26 16:57:14 +00:00
Dimitry Andric 042c4858b0 Fix lib/tsan/go/buildgo.sh for FreeBSD after r243051.
FreeBSD also needs to have sanitizer_linux_libcdep.cc included,
otherwise linking will fail with "undefined reference to
`__sanitizer::GetRSS()'".

While here, tabify the FreeBSD part, similar to the other parts.

llvm-svn: 260839
2016-02-14 00:26:32 +00:00
Anna Zaks 2ce9f9447c Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
of the report to stderr. This patch changes this behavior and writes the full
text of the report to syslog before we terminate the process. It also calls
os_trace (Activity Tracing available on OS X and iOS) with a message saying
that the report is available in syslog. This is useful, because this message
will be shown in the crash log.

For this to work, the patch makes sure we store the full report into
error_message_buffer unconditionally, and it also strips out ANSI escape
sequences from the report (they are used when producing colored reports).

I've initially tried to log to syslog during printing, which is done on Android
right now. The advantage is that if we crash during error reporting or the
produced error does not go through ScopedInErrorReport, we would still get a
(partial) message in the syslog. However, that solution is very problematic on
OS X. One issue is that the logging routine uses GCD, which may spawn a new
thread on its behalf. In many cases, the reporting logic locks threadRegistry,
which leads to deadlocks.

Reviewed at http://reviews.llvm.org/D13452

(In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
build failures on Linux.)

llvm-svn: 253688
2015-11-20 18:41:44 +00:00
Dmitry Vyukov 110094017c tsan: fix Go build on linux
librt is not linked in in Go build.

llvm-svn: 252877
2015-11-12 11:54:25 +00:00
Juergen Ributzka d7dcffaea4 Revert "Reapply: [asan] On OS X, log reports to syslog and os_trace"
Looks like this commit is deadlocking the ASAN tests on the green dragon bot
(http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA/).

llvm-svn: 252076
2015-11-04 21:03:12 +00:00
Anna Zaks 7920eb0b1f Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
        of the report to stderr. This patch changes this behavior and writes the full
        text of the report to syslog before we terminate the process. It also calls
        os_trace (Activity Tracing available on OS X and iOS) with a message saying
        that the report is available in syslog. This is useful, because this message
        will be shown in the crash log.

        For this to work, the patch makes sure we store the full report into
        error_message_buffer unconditionally, and it also strips out ANSI escape
        sequences from the report (they are used when producing colored reports).

        I've initially tried to log to syslog during printing, which is done on Android
        right now. The advantage is that if we crash during error reporting or the
        produced error does not go through ScopedInErrorReport, we would still get a
        (partial) message in the syslog. However, that solution is very problematic on
        OS X. One issue is that the logging routine uses GCD, which may spawn a new
        thread on its behalf. In many cases, the reporting logic locks threadRegistry,
        which leads to deadlocks.

        Reviewed at http://reviews.llvm.org/D13452

        (In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
         build failures on Linux.)

llvm-svn: 251577
2015-10-28 23:18:44 +00:00
Evgeniy Stepanov 5a268b10b3 [sanitizer] Implement logging to syslog.
Previously, Android target had a logic of duplicating all sanitizer
output to logcat. This change extends it to all posix platforms via
the use of syslog, controlled by log_to_syslog flag. Enabled by
default on Android, off everywhere else.

A bit of cmake magic is required to allow Printf() to call a libc
function. I'm adding a stub implementation to support no-libc builds
like dfsan and safestack.

This is a second attempt. I believe I've fixed all the issues that
prompted the revert: Mac build, and all kinds of non-CMake builds
(there are 3 of those).

llvm-svn: 243051
2015-07-23 22:05:20 +00:00
Evgeniy Stepanov 8e9c70be7f Add descriptive names to sanitizer entries in /proc/self/maps. Helps debugging.
This is done by creating a named shared memory region, unlinking it
and setting up a private (i.e. copy-on-write) mapping of that instead
of a regular anonymous mapping. I've experimented with regular
(sparse) files, but they can not be scaled to the size of MSan shadow
mapping, at least on Linux/X86_64 and ext3 fs.

Controlled by a common flag, decorate_proc_maps, disabled by default.

This patch has a few shortcomings:
* not all mappings are annotated, especially in TSan.
* our handling of memset() of shadow via mmap() puts small anonymous
  mappings inside larger named mappings, which looks ugly and can, in
  theory, hit the mapping number limit.

llvm-svn: 238621
2015-05-29 22:31:28 +00:00
Dmitri Gribenko 9cbc6f42aa Revert r230171 "TSan runtime: unbreak the build with ccache"
It broke a buildbot with a non-clang (?) compiler.

llvm-svn: 230174
2015-02-22 12:20:05 +00:00
Dmitri Gribenko bc4c42c345 TSan runtime: unbreak the build with ccache
llvm-svn: 230171
2015-02-22 11:04:59 +00:00
Dmitry Vyukov 620ebaa9c1 tsan: remove everything related to rss/background thread in Go mode
In Go mode the background thread is not started (internal_thread_start is empty).
There is no sense in having this code compiled in.
Also removes dependency on sanitizer_linux_libcdep.cc which is good,
ideally Go runtime does not depend on libc at all.

llvm-svn: 229396
2015-02-16 14:49:34 +00:00
Alexey Samsonov ec80d97c77 [TSan] Fix remaining bashism in buildgo.sh script.
llvm-svn: 227893
2015-02-02 23:14:46 +00:00
Alexey Samsonov 1270c530e8 [TSan] Fix Go sanity check script.
llvm-svn: 227870
2015-02-02 22:07:39 +00:00
Alexey Samsonov 46fe4a640d [TSan] Add Go runtime sanity check to CMake build.
llvm-svn: 227852
2015-02-02 20:37:28 +00:00
Alexey Samsonov a11a86b9ce [TSan] Fix Go build.
llvm-svn: 227634
2015-01-30 23:04:57 +00:00
Dmitry Vyukov cc5e40811b tsan: remove TSAN_SHADOW_COUNT
TSAN_SHADOW_COUNT is defined to 4 in all environments.
Other values of TSAN_SHADOW_COUNT were never tested and
were broken by recent changes to shadow mapping.
Remove it as there is no reason to fix nor maintain it.

llvm-svn: 226466
2015-01-19 15:01:07 +00:00
Alexey Samsonov df3aeb8e71 Remove TSAN_DEBUG in favor of SANITIZER_DEBUG.
llvm-svn: 225111
2015-01-03 04:29:12 +00:00
Kostya Serebryany 83ed889bbd [tsan] remove TSAN_GO in favor of SANITIZER_GO
llvm-svn: 223732
2014-12-09 01:31:14 +00:00
Alexey Samsonov 0e90668f14 Simplify Symbolizer::SymbolizePC() interface.
Return a linked list of AddressInfo objects, instead of using an array of
these objects as an output parameter. This simplifies the code in callers
of this function (especially TSan).

Fix a few memory leaks from internal allocator, when the returned
AddressInfo objects were not properly cleared.

llvm-svn: 223145
2014-12-02 19:48:40 +00:00
Dmitry Vyukov 1f84559b7d tsan: fix Go build
llvm-svn: 222342
2014-11-19 10:43:11 +00:00
Alexey Samsonov 46ecdeecc4 [TSan} Build Go version with -std=c++11
llvm-svn: 221192
2014-11-03 22:18:12 +00:00
Alexey Samsonov 71b471fa09 [TSan] Try to fix Go runtime build
llvm-svn: 215000
2014-08-06 23:13:23 +00:00
Dmitry Vyukov a8df247f5a tsan: expose atomic operations in Go runtime
llvm-svn: 213382
2014-07-18 14:54:02 +00:00
Dmitry Vyukov 73e1804c21 tsan: add Go build script for freebsd
llvm-svn: 211419
2014-06-20 22:53:43 +00:00
Dmitry Vyukov 8744b1c1e3 tsan: allow to build Go runtime with clang + enable SSE3
llvm-svn: 210344
2014-06-06 15:49:49 +00:00
Dmitry Vyukov bde4c9c773 tsan: refactor storage of meta information for heap blocks and sync objects
The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists).
This solves a number of problems:
 - eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26)
 - eliminates contention in SyncTab
 - eliminates contention in internal allocator during allocation of sync objects
 - removes a bunch of ad-hoc code in java interface
 - reduces java shadow from 2x to 1/2x
 - allows to memorize heap block meta info for Java and Go
 - allows to cleanup sync object meta info for Go
 - which in turn enabled deadlock detector for Go

llvm-svn: 209810
2014-05-29 13:50:54 +00:00
Evgeniy Stepanov f77f2ece71 [tsan] Fix gotsan build.
llvm-svn: 209286
2014-05-21 09:42:56 +00:00
Dmitry Vyukov bbbe6bba8a tsan: preliminary support for Go deadlock detector
llvm-svn: 204228
2014-03-19 12:50:47 +00:00
Alexey Samsonov 18adbc361a Remove sanitizer_linux_libcdep.cc from TSan-Go build
llvm-svn: 203238
2014-03-07 13:17:32 +00:00
Dmitry Vyukov 6cfab724ec tsan: refactor deadlock detector
Introduce DDetector interface between the tool and the DD itself.
It will help to experiment with other DD implementation,
as well as reuse DD in other tools.

llvm-svn: 202485
2014-02-28 10:48:13 +00:00
Dmitry Vyukov 9ba840865f tsan: simplify Go build script
we don't use assembly files

llvm-svn: 200916
2014-02-06 09:23:12 +00:00
Alexey Samsonov 782ad0eb71 [Sanitizer] Remove now unused symbolization functionality from MemoryMappingLayout
llvm-svn: 198014
2013-12-25 12:11:06 +00:00
Alexey Samsonov ceeb0e0657 [TSan] fix Go build
llvm-svn: 198005
2013-12-25 08:56:08 +00:00
Dmitry Vyukov 302ec7b9bc tsan: add memory_limit_mb flag
The flag allows to bound maximum process memory consumption (best effort).
If RSS reaches memory_limit_mb, tsan flushes all shadow memory.

llvm-svn: 191913
2013-10-03 17:14:35 +00:00
Dmitry Vyukov 2ea2d9cb29 tsan: remove unneeded ___chkstk_ms function from Go runtime
llvm-svn: 191161
2013-09-22 02:49:56 +00:00
Dmitry Vyukov f6ea6a5a19 tsan: better report formatting for Go
Say that gorotuine 1 is main goroutine.
Remove excessive new line.

llvm-svn: 188542
2013-08-16 11:15:14 +00:00
Alexey Samsonov ecdcf11bd6 [TSan] try to fix Go build
llvm-svn: 185063
2013-06-27 07:57:53 +00:00
Dmitry Vyukov a8570d38fe tsan: add -Wno-maybe-uninitialized to Go build script
this is how the rest of the codebase is built

llvm-svn: 183738
2013-06-11 11:44:43 +00:00
Dmitry Vyukov c9e304afbd tsan: fix darwin Go build
llvm-svn: 183402
2013-06-06 13:00:32 +00:00
Peter Collingbourne 8575195772 [nolibc] Move libc-dependent sanitizer_posix.cc code to sanitizer_posix_libcdep.cc.
llvm-svn: 182366
2013-05-21 10:27:07 +00:00
Peter Collingbourne 0c8df4e1d8 [nolibc] Move libc-dependent sanitizer_linux.cc code to sanitizer_linux_libcdep.cc.
llvm-svn: 182292
2013-05-20 15:57:44 +00:00
Dmitry Vyukov 50160030e1 tsan: fix Go build
llvm-svn: 177260
2013-03-18 08:52:46 +00:00
Dmitry Vyukov 714accd917 tsan: add sanity test for Go runtime into presubmit test
llvm-svn: 170185
2012-12-14 10:10:57 +00:00
Dmitry Vyukov e829b6231a tsan: change TSAN_OPTIONS->GORACE in build script as well
llvm-svn: 167578
2012-11-08 13:23:13 +00:00
Dmitry Vyukov c015712992 tsan: lazily allocate shadow for Go
llvm-svn: 167464
2012-11-06 16:00:16 +00:00