Commit Graph

164 Commits

Author SHA1 Message Date
Alexey Samsonov b3053d9cbe Allow UBSan+MSan and UBSan+TSan combinations (Clang part).
Embed UBSan runtime into TSan and MSan runtimes in the same as we do
in ASan. Extend UBSan test suite to also run tests for these
combinations.

llvm-svn: 235954
2015-04-28 00:56:48 +00:00
Timur Iskhodzhanov e8a6fbbfd3 Use WriteToFile instead of internal_write in non-POSIX code
llvm-svn: 234487
2015-04-09 14:11:25 +00:00
Timur Iskhodzhanov daa9e2d4e8 [Sanitizers] Make OpenFile more portable
llvm-svn: 234410
2015-04-08 16:03:22 +00:00
Alexander Potapenko b9c928bd10 [libsanitizer] Fix OpenFile() usage in TSan and DFSan.
This is a follow-up for r232936.

llvm-svn: 232937
2015-03-23 10:32:28 +00:00
Viktor Kutuzov a7d323e996 [Tsan] Do not sanitize memcpy() during thread initialization on FreeBSD
Differential Revision: http://reviews.llvm.org/D8324

llvm-svn: 232381
2015-03-16 14:42:21 +00:00
Mohit K. Bhakkad a46d5a7438 [TSan][MIPS] Adding support for MIPS64
Patch by Sagar Thakur

Reviewers: dvyukov, samsonov, petarj, kcc, dsanders.

Subscribers:  mohit.bhakkad, Anand.Takale, llvm-commits.

Differential Revision: http://reviews.llvm.org/D6291

llvm-svn: 229972
2015-02-20 06:42:41 +00:00
Alexey Samsonov e194dfa6be [TSan] Provide default values for compile definitions.
Provide defaults for TSAN_COLLECT_STATS and TSAN_NO_HISTORY.
Replace #ifdef directives with #if. This fixes a bug introduced
in r229112, where building TSan runtime with -DTSAN_COLLECT_STATS=0
would still enable stats collection and reporting.

llvm-svn: 229581
2015-02-17 23:23:10 +00:00
Dmitry Vyukov 620ebaa9c1 tsan: remove everything related to rss/background thread in Go mode
In Go mode the background thread is not started (internal_thread_start is empty).
There is no sense in having this code compiled in.
Also removes dependency on sanitizer_linux_libcdep.cc which is good,
ideally Go runtime does not depend on libc at all.

llvm-svn: 229396
2015-02-16 14:49:34 +00:00
Dmitry Vyukov 13fd535bcb tsan: fix shadow memory mapping on windows
llvm-svn: 229391
2015-02-16 13:52:04 +00:00
Dmitry Vyukov a3bdbadfd2 tsan: fix build
Revision 229127 introduced a bug:
zero value is not OK for trace headers,
because stack0 needs constructor call.
Instead unmap the unused part of trace after
all ctors have been executed.

llvm-svn: 229263
2015-02-14 16:14:10 +00:00
Dmitry Vyukov 8a9d23bf53 tsan: don't initialize trace header in release mode
We are going to use only a small part of the trace with the default
value of history_size. However, the constructor writes to the whole trace.
It writes mostly zeros, so freshly mmaped memory will do.
The only non-zero field if mutex type used for debugging.

Reduces per-goroutine overhead by 8K.

https://code.google.com/p/thread-sanitizer/issues/detail?id=89

llvm-svn: 229127
2015-02-13 17:39:03 +00:00
Dmitry Vyukov 41f4eba02d tsan: remove stats from ThreadState ifndef TSAN_COLLECT_STATS
Issue 89: Uses a lot of memory for each goroutine
https://code.google.com/p/thread-sanitizer/issues/detail?id=89

llvm-svn: 229112
2015-02-13 15:25:47 +00:00
Dmitry Vyukov 8070ac6bd6 tsan: don't unroll memory access loop in debug mode
MemoryAccess function consumes ~4K of stack in debug mode,
in significant part due to the unrolled loop.
And gtest gives only 4K of stack to death test
threads, which causes stack overflows in debug mode.

llvm-svn: 226644
2015-01-21 08:09:13 +00:00
Evgeniy Stepanov f074b3c2de [asan] Allow changing verbosity in activation flags.
This change removes some debug output in asan_flags.cc that
was reading the verbosity level before all the flags were parsed.

llvm-svn: 226566
2015-01-20 13:21:20 +00:00
Dmitry Vyukov cc5e40811b tsan: remove TSAN_SHADOW_COUNT
TSAN_SHADOW_COUNT is defined to 4 in all environments.
Other values of TSAN_SHADOW_COUNT were never tested and
were broken by recent changes to shadow mapping.
Remove it as there is no reason to fix nor maintain it.

llvm-svn: 226466
2015-01-19 15:01:07 +00:00
Alexey Samsonov df3aeb8e71 Remove TSAN_DEBUG in favor of SANITIZER_DEBUG.
llvm-svn: 225111
2015-01-03 04:29:12 +00:00
Dmitry Vyukov a4cd69217b tsan: disable flaky debug check
see the comment for details

llvm-svn: 224507
2014-12-18 10:17:01 +00:00
Kostya Serebryany 83ed889bbd [tsan] remove TSAN_GO in favor of SANITIZER_GO
llvm-svn: 223732
2014-12-09 01:31:14 +00:00
Alexey Samsonov 656c29b08f Replace InternalScopedBuffer<char> with InternalScopedString where applicable.
Summary: No functionality change.

Test Plan: make check-all

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D6472

llvm-svn: 223164
2014-12-02 22:20:11 +00:00
Alexey Samsonov 40733a8024 [TSan] Use StackTrace from sanitizer_common where applicable
Summary:
This change removes `__tsan::StackTrace` class. There are
now three alternatives:
  # Lightweight `__sanitizer::StackTrace`, which doesn't own a buffer
  of PCs. It is used in functions that need stack traces in read-only
  mode, and helps to prevent unnecessary allocations/copies (e.g.
  for StackTraces fetched from StackDepot).
  # `__sanitizer::BufferedStackTrace`, which stores buffer of PCs in
  a constant array. It is used in TraceHeader (non-Go version)
  # `__tsan::VarSizeStackTrace`, which owns buffer of PCs, dynamically
  allocated via TSan internal allocator.

Test Plan: compiler-rt test suite

Reviewers: dvyukov, kcc

Reviewed By: kcc

Subscribers: llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D6004

llvm-svn: 221194
2014-11-03 22:23:44 +00:00
Alexey Samsonov 3741ab82ba Change StackDepot interface to use StackTrace more extensively
llvm-svn: 220637
2014-10-26 06:23:07 +00:00
Dmitry Vyukov e9a5f03716 tsan: support mmap(MAP_32BIT)
Allow user memory in the first TB of address space.
This also enabled non-pie binaries and freebsd.
Fixes issue:
https://code.google.com/p/thread-sanitizer/issues/detail?id=5

llvm-svn: 220571
2014-10-24 17:07:29 +00:00
Alexey Samsonov 5c825967ea [TSan] Use common flags in the same way as all the other sanitizers
llvm-svn: 217559
2014-09-10 23:08:06 +00:00
Alexey Samsonov cd21e2f7e4 [TSan] Initialize flags as early as possible. Disables back coredump, accidentally enabled in r215479. Add a test.
llvm-svn: 215763
2014-08-15 19:53:51 +00:00
Dmitry Vyukov 5f86aaa27b tsan: fix unaligned memory access routine
It was possimitically handling an aligned 8-byte access as 2 4-byte accesses.

llvm-svn: 215546
2014-08-13 13:20:23 +00:00
Alexey Samsonov 1440105338 [Sanitizer] Simplify Symbolizer creation interface.
Get rid of Symbolizer::Init(path_to_external) in favor of
thread-safe Symbolizer::GetOrInit(), and use the latter version
everywhere. Implicitly depend on the value of external_symbolizer_path
runtime flag instead of passing it around manually.

No functionality change.

llvm-svn: 214005
2014-07-26 01:37:23 +00:00
Dmitry Vyukov 6819cf4974 tsan: query RSS every 100ms
Now that it become faster, it's OK to query it every 100ms again.

llvm-svn: 213943
2014-07-25 13:48:25 +00:00
Dmitry Vyukov 13793e5121 tsan: fix CurrentStackId
FuncEnter adds FuncEnter entry to trace that nobody removes later

llvm-svn: 210359
2014-06-06 18:05:12 +00:00
Dmitry Vyukov 9546afec62 tsan: minor optimizations for Go runtime
llvm-svn: 210351
2014-06-06 16:06:19 +00:00
Dmitry Vyukov 75f5cf657e tsan: fix out-of-bounds access in Go runtime
FuncEntry can resize the shadow stack, while "thr->shadow_stack_pos[0] = pc" writes out-of-bounds.

llvm-svn: 210349
2014-06-06 15:56:08 +00:00
Dmitry Vyukov 32336152af tsan: fix mapping of meta shadow for Go
Go maps heap and data+bss, these regions are not adjacent.
data+bss is mapped first.

llvm-svn: 210348
2014-06-06 15:54:49 +00:00
Dmitry Vyukov afdcc96d9f tsan: optimize memory access functions
The optimization is two-fold:
First, the algorithm now uses SSE instructions to
handle all 4 shadow slots at once. This makes processing
faster.
Second, if shadow contains the same access, we do not
store the event into trace. This increases effective
trace size, that is, tsan can remember up to 10x more
previous memory accesses.

Perofrmance impact:
Before:
[       OK ] DISABLED_BENCH.Mop8Read (2461 ms)
[       OK ] DISABLED_BENCH.Mop8Write (1836 ms)
After:
[       OK ] DISABLED_BENCH.Mop8Read (1204 ms)
[       OK ] DISABLED_BENCH.Mop8Write (976 ms)
But this measures only fast-path.
On large real applications the speedup is ~20%.

Trace size impact:
On app1:
Memory accesses                   :       1163265870
  Including same                  :        791312905 (68%)
on app2:
Memory accesses                   :        166875345
  Including same                  :        150449689 (90%)
90% of filtered events means that trace size is effectively 10x larger.

llvm-svn: 209897
2014-05-30 13:36:29 +00:00
Dmitry Vyukov 65dce1e4f7 tsan: write memory profile in one line (which is much more readable)
e.g.:
RSS 420 MB: shadow:35 meta:231 file:2 mmap:129 trace:19 heap:0 other:0 nthr=1/31
RSS 365 MB: shadow:3 meta:231 file:2 mmap:106 trace:19 heap:0 other:0 nthr=1/31
RSS 429 MB: shadow:23 meta:234 file:2 mmap:143 trace:19 heap:6 other:0 nthr=1/31
RSS 509 MB: shadow:78 meta:241 file:2 mmap:147 trace:19 heap:19 other:0 nthr=1/31

llvm-svn: 209813
2014-05-29 14:11:38 +00:00
Dmitry Vyukov ef5f26bf19 tsan: allow to write memory profile to stdout/stderr
llvm-svn: 209811
2014-05-29 14:02:06 +00:00
Dmitry Vyukov bde4c9c773 tsan: refactor storage of meta information for heap blocks and sync objects
The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists).
This solves a number of problems:
 - eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26)
 - eliminates contention in SyncTab
 - eliminates contention in internal allocator during allocation of sync objects
 - removes a bunch of ad-hoc code in java interface
 - reduces java shadow from 2x to 1/2x
 - allows to memorize heap block meta info for Java and Go
 - allows to cleanup sync object meta info for Go
 - which in turn enabled deadlock detector for Go

llvm-svn: 209810
2014-05-29 13:50:54 +00:00
Evgeniy Stepanov de9467593a [tsan] Fix tsango build.
llvm-svn: 209658
2014-05-27 14:18:43 +00:00
Dmitry Vyukov 547089e2a5 tsan: allow to disable history collection
The mode is enabled with -DTSAN_NO_HISTORY=1 flag.
Intended mostly for research purposes (how fast can it go w/o history).

llvm-svn: 208878
2014-05-15 12:51:48 +00:00
Dmitry Vyukov b7ebc53772 tsan: stop background thread when sandbox is enabled
Fixes https://code.google.com/p/thread-sanitizer/issues/detail?id=56

llvm-svn: 207114
2014-04-24 13:09:17 +00:00
Dmitry Vyukov b5eb8f0212 tsan: fix vector clocks
the new optimizations break when thread ids gets reused (clocks go backwards)
add the necessary tests as well

llvm-svn: 206035
2014-04-11 15:38:03 +00:00
Alexey Samsonov 425314a65f [TSan] Fix a rare deadlock on multithreaded fork.
If a multi-threaded program calls fork(), TSan ignores all memory accesses
in the child to prevent deadlocks in TSan runtime. This is OK, as child is
probably going to call exec() as soon as possible. However, a rare deadlocks
could be caused by ThreadIgnoreBegin() function itself.

ThreadIgnoreBegin() remembers the current stack trace and puts it into the
StackDepot to report a warning later if a thread exited with ignores enabled.
Using StackDepotPut in a child process is dangerous: it locks a mutex on
a slow path, which could be already locked in a parent process.

The fix is simple: just don't put current stack traces to StackDepot in
ThreadIgnoreBegin() and ThreadIgnoreSyncBegin() functions if we're
running after a multithreaded fork. We will not report any
"thread exited with ignores enabled" errors in this case anyway.

Submitting this without a testcase, as I believe the standalone reproducer
is pretty hard to construct.

llvm-svn: 205534
2014-04-03 12:51:26 +00:00
Alexey Samsonov b4525218d4 [TSan] Fix a typo ThreadIgnoreSyncEnd. Found by inspection
llvm-svn: 205531
2014-04-03 12:28:16 +00:00
Dmitry Vyukov 5da31e03bf tsan: fix deadlock during fork
ReportRace takes the two mutexes in the opposite order

llvm-svn: 204809
2014-03-26 14:13:31 +00:00
Dmitry Vyukov d23118c3b2 tsan: optimize vector clock operations
Make vector clock operations O(1) for several important classes of use cases.
See comments for details.
Below are stats from a large server app, 77% of all clock operations are handled as O(1).

Clock acquire                     :         25983645
  empty clock                     :          6288080
  fast from release-store         :         14917504
  contains my tid                 :          4515743
  repeated (fast)                 :          2141428
  full (slow)                     :          2636633
  acquired something              :          1426863
Clock release                     :          2544216
  resize                          :             6241
  fast1                           :           197693
  fast2                           :          1016293
  fast3                           :             2007
  full (slow)                     :          1797488
  was acquired                    :           709227
  clear tail                      :                1
  last overflow                   :                0
Clock release store               :          3446946
  resize                          :           200516
  fast                            :           469265
  slow                            :          2977681
  clear tail                      :                0
Clock acquire-release             :           820028

llvm-svn: 204656
2014-03-24 18:54:20 +00:00
Dmitry Vyukov c9e12aa323 tsan: deobfuscate global ctx variable
llvm-svn: 204327
2014-03-20 10:36:20 +00:00
Dmitry Vyukov bbbe6bba8a tsan: preliminary support for Go deadlock detector
llvm-svn: 204228
2014-03-19 12:50:47 +00:00
Dmitry Vyukov 3cd028c0b2 tsan: deadlock detector: add deadlock detector flags
the first flags is to enable printing of the second stack per edge

llvm-svn: 204150
2014-03-18 13:13:47 +00:00
Dmitry Vyukov 6cfab724ec tsan: refactor deadlock detector
Introduce DDetector interface between the tool and the DD itself.
It will help to experiment with other DD implementation,
as well as reuse DD in other tools.

llvm-svn: 202485
2014-02-28 10:48:13 +00:00
Dmitry Vyukov 6fd67f9c7a tsan: fix internal deadlock detector for external deadlock detector
we must go deeper!

llvm-svn: 202365
2014-02-27 09:02:58 +00:00
Kostya Serebryany 0548c79859 [tsan] add coarse-grained lock around the DeadlockDetector. We can do better than that, but that's a start.
llvm-svn: 201861
2014-02-21 15:07:18 +00:00
Evgeniy Stepanov 05938a23f5 [sanitizer] Use mmap to zero-fill large shadow regions.
This is covered by existing ASan test.
This does not change anything for TSan by default (but provides a flag to 
change the threshold size).
Based on a patch by florent.bruneau here:
  https://code.google.com/p/address-sanitizer/issues/detail?id=256

llvm-svn: 201400
2014-02-14 11:41:26 +00:00