Commit Graph

70 Commits

Author SHA1 Message Date
Daniel S Fava 3b222ef246 tsan: fixes to ThreadClock::releaseStoreAcquire and tests
Fixes:
1. Setting the number of entries in a thread's clock to max between
   the thread and the SyncClock the thread is acquiring from
2. Setting last_acquire_

Unit- and stress-test for releaseStoreAcquire added to
tests/unit/tsan_clock_test.cpp
2020-04-16 22:53:26 +02:00
Nico Weber 5de29a4b0e compiler-rt: Rename .cc file in lib/tsan/tests/{rtl,unit} to .cpp
Like r367463, but for tsan/tests/{rtl,unit}.

llvm-svn: 367566
2019-08-01 14:26:37 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Alex Shlyapnikov c75d47b52d [TSan] Report proper error on allocator failures instead of CHECK(0)-ing
Summary:
Following up on and complementing D44404 and other sanitizer allocators.

Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, no stack, no details, not too
helpful nor informative. To improve the situation, detailed and structured
common errors were defined and reported under the appropriate conditions.

Common tests were generalized a bit to cover a slightly different TSan
stack reporting format, extended to verify errno value and returned
pointer value check is now explicit to facilitate debugging.

Reviewers: dvyukov

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48087

llvm-svn: 334975
2018-06-18 20:03:31 +00:00
Kamil Rytarowski 64fc9cf2e5 Move __tsan::Vector to __sanitizer
Summary:
The low-fat STL-like vector container will be reused in MSan.

It is needed to implement an atexit(3) interceptor on NetBSD/amd64 in MSan.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc

Reviewed By: dvyukov

Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40726

llvm-svn: 319650
2017-12-04 12:30:09 +00:00
Kostya Kortchinsky 0357e8de3b [tsan] Check for pvalloc overlow
Summary:
`CheckForPvallocOverflow` was introduced with D35818 to detect when pvalloc
would wrap when rounding up to the next multiple of the page size.

Add this check to TSan's pvalloc implementation.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D36245

llvm-svn: 309897
2017-08-02 22:47:54 +00:00
Alex Shlyapnikov 132689243e [Sanitizers] TSan allocator set errno on failure.
Summary:
Set proper errno code on allocation failures and change realloc, pvalloc,
aligned_alloc, memalign and posix_memalign implementation to satisfy
their man-specified requirements.

Modify allocator API implementation to bring it closer to other
sanitizers allocators.

Reviewers: dvyukov

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D35690

llvm-svn: 308929
2017-07-24 21:22:59 +00:00
Dmitry Vyukov 9f2c6207d5 tsan: optimize sync clock memory consumption
This change implements 2 optimizations of sync clocks that reduce memory consumption:

Use previously unused first level block space to store clock elements.
Currently a clock for 100 threads consumes 3 512-byte blocks:

2 64-bit second level blocks to store clock elements
+1 32-bit first level block to store indices to second level blocks
Only 8 bytes of the first level block are actually used.
With this change such clock consumes only 2 blocks.

Share similar clocks differing only by a single clock entry for the current thread.
When a thread does several release operations on fresh sync objects without intervening
acquire operations in between (e.g. initialization of several fields in ctor),
the resulting clocks differ only by a single entry for the current thread.
This change reuses a single clock for such release operations. The current thread time
(which is different for different clocks) is stored in dirty entries.

We are experiencing issues with a large program that eats all 64M clock blocks
(32GB of non-flushable memory) and crashes with dense allocator overflow.
Max number of threads in the program is ~170 which is currently quite unfortunate
(consume 4 blocks per clock). Currently it crashes after consuming 60+ GB of memory.
The first optimization brings clock block consumption down to ~40M and
allows the program to work. The second optimization further reduces block consumption
to "modest" 16M blocks (~8GB of RAM) and reduces overall RAM consumption to ~30GB.

Measurements on another real world C++ RPC benchmark show RSS reduction
from 3.491G to 3.186G and a modest speedup of ~5%.

Go parallel client/server HTTP benchmark:
https://github.com/golang/benchmarks/blob/master/http/http.go
shows RSS reduction from 320MB to 240MB and a few percent speedup.

Reviewed in https://reviews.llvm.org/D35323

llvm-svn: 308018
2017-07-14 11:30:06 +00:00
Dmitry Vyukov 5f924089e5 tsan: prepare clock for future changes
Pass ClockCache to ThreadClock::set and introduce ThreadCache::ResetCached.
For now both are unused, but will reduce future diffs.

llvm-svn: 307784
2017-07-12 12:45:20 +00:00
Dmitry Vyukov 0a9265c26c tsan: add another test for clock growth
llvm-svn: 307777
2017-07-12 12:25:03 +00:00
Kuba Brecka a56c2ad594 Follow-up for r277458: Update the tsan_mman_test.cc unit test.
llvm-svn: 277463
2016-08-02 14:41:03 +00:00
Dmitry Vyukov b3a51bdcd7 tsan: don't create sync objects on acquire
Creating sync objects on acquire is pointless:
acquire of a just created sync object if a no-op.

llvm-svn: 273862
2016-06-27 11:14:59 +00:00
Dmitry Vyukov 3efe395788 tsan: change tsan/Go interface for obtaining the current Processor
Current interface assumes that Go calls ProcWire/ProcUnwire
to establish the association between thread and proc.
With the wisdom of hindsight, this interface does not work
very well. I had to sprinkle Go scheduler with wire/unwire
calls, and any mistake leads to hard to debug crashes.
This is not something one wants to maintian.
Fortunately, there is a simpler solution. We can ask Go
runtime as to what is the current Processor, and that
question is very easy to answer on Go side.
Switch to such interface.

llvm-svn: 267703
2016-04-27 12:30:48 +00:00
Dmitry Vyukov d87c7b321a tsan: split thread into logical and physical state
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).

Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:

Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.

Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.

This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.

Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 267678
2016-04-27 08:23:02 +00:00
Kuba Brecka 4c80867ecf [sanitizer] On OS X, verify that interceptors work and abort if not, take 2
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.

TSan unit tests run with a statically linked runtime, where interceptors don't work. To avoid aborting the process in this case, the patch replaces `DisableReexec()` with a weak `ReexecDisabled()` function which is defined to return true in unit tests.

Differential Revision: http://reviews.llvm.org/D18212

llvm-svn: 263695
2016-03-17 08:37:25 +00:00
Dmitry Vyukov 7f022ae4c2 tsan: revert r262037
Broke aarch64 and darwin bots.

llvm-svn: 262046
2016-02-26 18:26:48 +00:00
Dmitry Vyukov b8868b9bea tsan: split thread into logical and physical state
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.

llvm-svn: 262037
2016-02-26 16:57:14 +00:00
Kuba Brecka 377cafbbbc [tsan] Fix CallocOverflow unit test on OS X
On OS X, interceptors don't work in unit tests, so calloc() calls the system allocator. We need to use user_calloc() instead.

Differential Revision: http://reviews.llvm.org/D14918

llvm-svn: 253979
2015-11-24 13:27:33 +00:00
Kuba Brecka c47620a838 [tsan] Add TSan unit test support for OS X
This patch enables building and running TSan unit tests on OS X.

Differential Revision: http://reviews.llvm.org/D14546

llvm-svn: 252731
2015-11-11 14:53:57 +00:00
Alexey Samsonov bb79b06f4e [Sanitizers] Unify the semantics and usage of "exitcode" runtime flag across all sanitizers.
Summary:
Merge "exitcode" flag from ASan, LSan, TSan and "exit_code" from MSan
into one entity. Additionally, make sure sanitizer_common now uses the
value of common_flags()->exitcode when dying on error, so that this
flag will automatically work for other sanitizers (UBSan and DFSan) as
well.

User-visible changes:
* "exit_code" MSan runtime flag is now deprecated. If explicitly
  specified, this flag will take precedence over "exitcode".
  The users are encouraged to migrate to the new version.
* __asan_set_error_exit_code() and __msan_set_exit_code() functions
  are removed. With few exceptions, we don't support changing runtime
  flags during program execution - we can't make them thread-safe.
  The users should use __sanitizer_set_death_callback()
  that would call _exit() with proper exit code instead.
* Plugin tools (LSan and UBSan) now inherit the exit code of the parent
  tool. In particular, this means that ASan would now crash the program
  with exit code "1" instead of "23" if it detects leaks.

Reviewers: kcc, eugenis

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D12120

llvm-svn: 245734
2015-08-21 20:49:37 +00:00
Alexey Samsonov 8b3334d909 [TSan] Fix -Werror=sign-compare warning. NFC.
llvm-svn: 227527
2015-01-30 01:02:12 +00:00
Alexey Samsonov df3aeb8e71 Remove TSAN_DEBUG in favor of SANITIZER_DEBUG.
llvm-svn: 225111
2015-01-03 04:29:12 +00:00
Alexey Samsonov 40733a8024 [TSan] Use StackTrace from sanitizer_common where applicable
Summary:
This change removes `__tsan::StackTrace` class. There are
now three alternatives:
  # Lightweight `__sanitizer::StackTrace`, which doesn't own a buffer
  of PCs. It is used in functions that need stack traces in read-only
  mode, and helps to prevent unnecessary allocations/copies (e.g.
  for StackTraces fetched from StackDepot).
  # `__sanitizer::BufferedStackTrace`, which stores buffer of PCs in
  a constant array. It is used in TraceHeader (non-Go version)
  # `__tsan::VarSizeStackTrace`, which owns buffer of PCs, dynamically
  allocated via TSan internal allocator.

Test Plan: compiler-rt test suite

Reviewers: dvyukov, kcc

Reviewed By: kcc

Subscribers: llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D6004

llvm-svn: 221194
2014-11-03 22:23:44 +00:00
Alexey Samsonov e6bbcfa3df [TSan] Fix signed-compare warning in the unit test
llvm-svn: 221190
2014-11-03 22:17:39 +00:00
Dmitry Vyukov e9a5f03716 tsan: support mmap(MAP_32BIT)
Allow user memory in the first TB of address space.
This also enabled non-pie binaries and freebsd.
Fixes issue:
https://code.google.com/p/thread-sanitizer/issues/detail?id=5

llvm-svn: 220571
2014-10-24 17:07:29 +00:00
Alexey Samsonov 5c825967ea [TSan] Use common flags in the same way as all the other sanitizers
llvm-svn: 217559
2014-09-10 23:08:06 +00:00
Dmitry Vyukov 70db9d4d72 tsan: allocate vector clocks using slab allocator
Vector clocks is the most actively allocated object in tsan runtime.
Current internal allocator is not scalable enough to handle allocation
of clocks in scalable way (too small caches). This changes transforms
clocks to 2-level array with 512-byte blocks. Since all blocks are of
the same size, it's possible to cache them more efficiently in per-thread caches.

llvm-svn: 214912
2014-08-05 18:45:02 +00:00
Dmitry Vyukov 1015f8d753 tsan: remove special handling of false reports coming from JVM
There is now a more common functionality in the form of called_from_lib suppressions.

llvm-svn: 213057
2014-07-15 09:38:30 +00:00
Dmitry Vyukov 67ccf9851c tsan: reapply 212531 and 212532 with a fix
don't reset s->addr as well

llvm-svn: 212565
2014-07-08 20:37:16 +00:00
Alexey Samsonov 047ea1717b [TSan] Revert r212531 and r212532.
They cause "check-tsan" command to hang. Details in r212532 review thread.

llvm-svn: 212562
2014-07-08 20:17:19 +00:00
Dmitry Vyukov a480d3013a tsan: fix a bug in metamap
The bug happens in the following case:
Mutex is located at heap block beginning,
when we call MutexDestroy, s->next is set to 0,
so free can't find the MBlock related to the block.

llvm-svn: 212531
2014-07-08 13:28:01 +00:00
Alexey Samsonov 91bb8e0e3a Generalize sanitizer allocator public interface.
Introduce new public header <sanitizer/allocator_interface.h> and a set
of functions __sanitizer_get_ownership(), __sanitizer_malloc_hook() etc.
that will eventually replace their tool-specific equivalents
(__asan_get_ownership(), __msan_get_ownership() etc.). Tool-specific
functions are now deprecated and implemented as stubs redirecting
to __sanitizer_ versions (which are implemented differently in each tool).

Replace all uses of __xsan_ versions with __sanitizer_ versions in unit
and lit tests.

llvm-svn: 212469
2014-07-07 17:39:31 +00:00
Alexey Samsonov 06ff6cbf4d [TSan] Equalize the behavior of __tsan_get_allocated_size and user_alloc_usable_size.
The former used to crash with a null deref if it was given a not owned pointer,
while the latter returned 0. Now they both return 0. This is still not the best possible
behavior: it is better to print an error report with a stack trace, pointing
to the error in user code, as we do in ASan.

llvm-svn: 212112
2014-07-01 18:01:20 +00:00
Dmitry Vyukov afdcc96d9f tsan: optimize memory access functions
The optimization is two-fold:
First, the algorithm now uses SSE instructions to
handle all 4 shadow slots at once. This makes processing
faster.
Second, if shadow contains the same access, we do not
store the event into trace. This increases effective
trace size, that is, tsan can remember up to 10x more
previous memory accesses.

Perofrmance impact:
Before:
[       OK ] DISABLED_BENCH.Mop8Read (2461 ms)
[       OK ] DISABLED_BENCH.Mop8Write (1836 ms)
After:
[       OK ] DISABLED_BENCH.Mop8Read (1204 ms)
[       OK ] DISABLED_BENCH.Mop8Write (976 ms)
But this measures only fast-path.
On large real applications the speedup is ~20%.

Trace size impact:
On app1:
Memory accesses                   :       1163265870
  Including same                  :        791312905 (68%)
on app2:
Memory accesses                   :        166875345
  Including same                  :        150449689 (90%)
90% of filtered events means that trace size is effectively 10x larger.

llvm-svn: 209897
2014-05-30 13:36:29 +00:00
Dmitry Vyukov bde4c9c773 tsan: refactor storage of meta information for heap blocks and sync objects
The new storage (MetaMap) is based on direct shadow (instead of a hashmap + per-block lists).
This solves a number of problems:
 - eliminates quadratic behaviour in SyncTab::GetAndLock (https://code.google.com/p/thread-sanitizer/issues/detail?id=26)
 - eliminates contention in SyncTab
 - eliminates contention in internal allocator during allocation of sync objects
 - removes a bunch of ad-hoc code in java interface
 - reduces java shadow from 2x to 1/2x
 - allows to memorize heap block meta info for Java and Go
 - allows to cleanup sync object meta info for Go
 - which in turn enabled deadlock detector for Go

llvm-svn: 209810
2014-05-29 13:50:54 +00:00
Dmitry Vyukov c845decce1 tsan: better reports for "double lock of a mutex"
+ fixes crashes due to races on symbolizer, see:
https://code.google.com/p/thread-sanitizer/issues/detail?id=55

llvm-svn: 207204
2014-04-25 07:42:55 +00:00
Dmitry Vyukov ae5c1070ba tsan: reduce number of iterations in the test
otherwise it runs for 50 seconds in debug mode

llvm-svn: 206950
2014-04-23 06:49:09 +00:00
Dmitry Vyukov b5eb8f0212 tsan: fix vector clocks
the new optimizations break when thread ids gets reused (clocks go backwards)
add the necessary tests as well

llvm-svn: 206035
2014-04-11 15:38:03 +00:00
Evgeniy Stepanov 1611ed8879 [tsan] Fix more warnings in TSan tests.
llvm-svn: 204715
2014-03-25 13:01:32 +00:00
Evgeniy Stepanov 0f08043aeb [tsan] Fix compiler warnings in TSan tests.
llvm-svn: 204712
2014-03-25 11:17:14 +00:00
Dmitry Vyukov d23118c3b2 tsan: optimize vector clock operations
Make vector clock operations O(1) for several important classes of use cases.
See comments for details.
Below are stats from a large server app, 77% of all clock operations are handled as O(1).

Clock acquire                     :         25983645
  empty clock                     :          6288080
  fast from release-store         :         14917504
  contains my tid                 :          4515743
  repeated (fast)                 :          2141428
  full (slow)                     :          2636633
  acquired something              :          1426863
Clock release                     :          2544216
  resize                          :             6241
  fast1                           :           197693
  fast2                           :          1016293
  fast3                           :             2007
  full (slow)                     :          1797488
  was acquired                    :           709227
  clear tail                      :                1
  last overflow                   :                0
Clock release store               :          3446946
  resize                          :           200516
  fast                            :           469265
  slow                            :          2977681
  clear tail                      :                0
Clock acquire-release             :           820028

llvm-svn: 204656
2014-03-24 18:54:20 +00:00
Dmitry Vyukov 3b37e8bf18 tsan: yet another attempt to fix pthread_cond interceptors
Make behavior introduced in r202820 conditional (under legacy_pthread_cond flag).
The new issue that we've hit with the satellite pthread_cond_t struct is
that pthread_condattr_getpshared does not work (satellite data is not shared between processes).
The idea is that most processes do not use pthread 2.2.5.
The rare ones that use (2.2.5 is dated by 2002) must specify legacy_pthread_cond=1
on their own risk.

llvm-svn: 204032
2014-03-17 07:51:53 +00:00
Alexey Samsonov 7304b4201f [Sanitizer] Teach external symbolizer to work with addr2line if llvm-symbolizer is unavailable. Allow this mode in TSan.
llvm-svn: 201218
2014-02-12 08:29:42 +00:00
Dmitry Vyukov 16e7a758b0 tsan: do not deadlock on fork
Currently correct programs can deadlock after fork, because atomic operations and async-signal-safe calls are not async-signal-safe under tsan.
With this change:
- if a single-threaded program forks, the child continues running with verification enabled (the tsan background thread is recreated as well)
- if a multi-threaded program forks, then the child runs with verification disabled (memory accesses, atomic operations and interceptors are disabled); it's expected that it will exec soon anyway
- if the child tries to create more threads after multi-threaded fork, the program aborts with error message
- die_after_fork flag is added that allows to continue running, but all bets are off

http://llvm-reviews.chandlerc.com/D2614

llvm-svn: 199993
2014-01-24 12:33:35 +00:00
Dmitry Vyukov ce3721057d tsan: remove in_rtl counter
This is intended to address the following problem.
Episodically we see CHECK-failures when recursive interceptors call back into user code. Effectively we are not "in_rtl" at this point, but it's very complicated and fragile to properly maintain in_rtl property. Instead get rid of it. It was used mostly for sanity CHECKs, which basically never uncover real problems.
Instead introduce ignore_interceptors flag, which is used in very few narrow places to disable recursive interceptors (e.g. during runtime initialization).

llvm-svn: 197979
2013-12-24 12:55:56 +00:00
Alexey Samsonov 25d010a98d [Sanitizer] Always initialize a Symbolizer (even if 'symbolize' is false).
If 'symbolize' flag is not set, we still want to transform virtual address
to module+offset pair in the call to Symbolizer::SymbolizeCode().
See https://code.google.com/p/address-sanitizer/issues/detail?id=251 for
more details.

llvm-svn: 197491
2013-12-17 11:15:39 +00:00
Dmitry Vyukov 89117e1fcc tsan: do not leave verbosity>0 in tests
llvm-svn: 195833
2013-11-27 10:05:22 +00:00
Dmitry Vyukov 7bd319cc08 tsan: fix flags parsing
- running_on_valgrind was not parsed in some contexts
- refactor code a bit
- add comprehensive tests for flags parsing

llvm-svn: 195831
2013-11-27 09:54:10 +00:00
Dmitry Vyukov 464ebbd67b tsan: move shadow stack from thread descriptors to fixed addresses
This allows to increase max shadow stack size to 64K,
and reliably catch shadow stack overflows instead of silently
corrupting memory.

llvm-svn: 192797
2013-10-16 15:35:12 +00:00
Alexey Samsonov e99a0b9b76 [TSan] Use Clang to compile and link TSan unit tests with TSan runtime
llvm-svn: 190503
2013-09-11 09:56:33 +00:00