We frequenty allocate sizeof(T) memory and call T ctor on that memory
(C++ new keyword effectively). Currently it's quite verbose and
usually takes 2 lines of code.
Add New<T>() helper that does it much more concisely.
Rename internal_free to Free that also sets the pointer to nullptr.
Shorter and safer.
Rename internal_alloc to Alloc, just shorter.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D107085
The updated lots_of_threads.c test with 300 threads
started running for too long on machines with low
hardware parallelism (e.g. taskset -c 0-1).
On lots of CPUs it finishes in ~2 secs. But with
taskset -c 0-1 it runs for hundreds of seconds
effectively spinning in the barrier in the sleep loop.
We now have the handy futex API in sanitizer_common.
Use it instead of the passive spin loop.
It makes the test run only faster with taskset -c 0-1,
it runs for ~1.5 secs, while with full parallelism
it still runs for ~2 secs (but consumes less CPU time).
Depends on D107131.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D107132
We call non-inlinable Initialize from all interceptors/syscalls,
but most of the time runtime is already initialized and this just
introduces unnecessary overhead.
Add LazyInitialize that (1) inlinable, (2) does nothing if
.preinit_array is enabled (expected case on Linux).
Depends on D107071.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107072
We are very paranoid with CHECKs in all Java entry points.
These CHECKs were added along with Java support.
At that point it wasn't clear what exactly to expect from JVM part
and if JVM part is correct or not. Thus CHECK paranoia.
These CHECKs never fired in practice, but we pay runtime cost
in every entry point all the time.
Replace CHECKs with DCHECKs.
Depends on D107069.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107071
We used to call Initialize in every Java point.
That was removed in 6563bb53b5 ("tsan: don't use caller/current PC in Java interfaces").
The intention was to add a single Initialize to __tsan_java_init instead.
Do that.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107069
Add intrusive doubly-linked list container template, IList.
It will be used in the new tsan runtime.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D107050
Switch x86_64 requirement for optimized code from SSE3 to SSE4.2.
The new tsan runtime will need few instructions that are only
supported by SSE4:
_mm_max_epu32
_mm_extract_epi8
_mm_insert_epi32
SSE3 was introcued in 2004 and SSE4 in 2006:
https://en.wikipedia.org/wiki/SSE3https://en.wikipedia.org/wiki/SSE4
We are still providing non-optimized C++ version of the code,
so either way it's possible to build tsan runtime for any CPU.
But for Go this will bump strict requirement for -race because
Go contains prebuilt versions and these will be built with -msse4.2.
But requiring a CPU produced at least in 2006 looks reasonable for
a debugging tool (more reasonable than disabling optimizations
for everybody).
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106948
Currently __tsan_atomic* functions do FuncEntry/Exit using caller PC
and then use current PC (pointing to __tsan_atomic* itself) during
memory access handling. As the result the top function in reports
involving atomics is __tsan_atomic* and the next frame points to user code.
Remove FuncEntry/Exit in atomic functions and use caller PC
during memory access handling. This removes __tsan_atomic*
from the top of report stacks, so that they point right to user code.
The motivation for this is performance.
Some atomic operations are very hot (mostly loads),
so removing FuncEntry/Exit is beneficial.
This also reduces thread trace consumption (1 event instead of 3).
__tsan_atomic* at the top of the stack is not necessary
and does not add any new information. We already say
"atomic write of size 4", "__tsan_atomic32_store" does not add
anything new.
It also makes reports consistent between atomic and non-atomic
accesses. For normal accesses we say "previous write" and point
to user code; for atomics we say "previous atomic write" and now
also point to user code.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106966
We strip all frames below main but in some cases it may be not enough.
Namely, when main is instrumented but does not call any other instrumented code.
In this case __tsan_func_entry in main obtains PC pointing to __libc_start_main
(as we pass caller PC to __tsan_func_entry), but nothing obtains PC pointing
to main itself (as main does not call any instrumented code).
In such case we will not have main in the stack, and stripping everything
below main won't work.
So strip __libc_start_main explicitly as well.
But keep stripping of main because __libc_start_main is glibc/linux-specific,
so looking for main is more reliable (and usually main is present in stacks).
Depends on D106957.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106958
Caller PC is plain harmful as native caller PC has nothing to do with Java code.
Current PC is not particularly useful and may be somewhat confusing for Java users
as it makes top frame to point to some magical __tsan function.
But obtaining and using these PCs adds runtime cost for every java event.
Remove these PCs. Rely only on official Java frames.
It makes execution faster, code simpler and reports better.
Depends on D106956.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106957
We maintain information about Java allocations,
but for some reason never printed it in reports.
Print it.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106956
We used to count number of allocations/bytes based on the type
and maybe record them in heap block headers.
But that's all in the past, now it's not used for anything.
Remove the mblock type.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106971
Remove pc argument of ThreadIgnoreEnd, ThreadIgnoreSyncEnd
and AcquireGlobal functions. It's unused and in some places
we don't even have a pc and pass 0 anyway.
Don't confuse readers and don't pretend that pc is needed
and that passing 0 is somehow deficient.
Use simpler convention for ThreadIgnoreBegin and ThreadIgnoreSyncBegin:
accept only pc instread of pc+save_stack. 0 pc means "don't save stack".
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106973
Currently the barrier supports only 256 threads,
this does not allow to write reliable tests that use more threads.
Bump max number of threads to 1024 to support writing
good stress tests.
Also replace sched_yield() with usleep(100) on the wait path.
If we write tests that create hundreds of threads (and dozens
of tests can run in parallel), yield would consume massive
amounts of CPU time for spinning.
Depends on D106952.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D106953
Now that Mutex is blocking there is no point in using BlockingMutex.
Switch to Mutex.
Depends on D106379.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106560
Now that sanitizer_common mutex has feature-parity with tsan mutex,
switch tsan to the sanitizer_common mutex and remove tsan's custom mutex.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106379
In preparation for replacing tsan Mutex with sanitizer_common Mutex,
which has thread-safety annotations. Thread safety analysis does not
understand MetaMap::GetAndLock which returns a locked sync object.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106548
This is preparation to switching to the sanitizer_common Mutex.
Without this change after the switch we will start failing
on existing from the runtime with runtime mutexes held.
Previously it worked because CheckNoLocks did not see sanitizer_common mutexes.
Depends on D106547.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106558
Rename Mutex class in tests to avoid conflicts with sanitizer_common Mutex.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106547
signal(2) and sigaction(2) have defined behaviors for invalid signal number
(EINVAL) and some programs rely on it.
The added test case also reveals that MSAN is too strict in this regard.
Test case passed on x86_64 Linux and AArch64 Linux.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106468
The current (default) line length is 80 columns.
That's based on old hardware and historical conventions.
There are no existent reasons to keep line length that small,
especially provided that our coding style uses quite lengthy
identifiers. The Linux kernel recently switched to 100,
let's start with 100 as well.
This change intentionally does not re-format code.
Re-formatting is intended to happen incrementally,
or on dir-by-dir basis separately.
Reviewed By: vitalybuka, melver, MaskRay
Differential Revision: https://reviews.llvm.org/D106436
I don't think the stat subsystem was ever used since tsan
development in 2012. But it adds lots of code and this
effectively dead code needs to be updated if the runtime
code changes, which adds maintanance cost for no benefit.
Normal profiler usually gives enough info and that info
is more trustworthy.
Remove the stats subsystem.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D106276
Add pragma line so that errors messages point to the actual
source files rather than to the concatenated gotsan.cpp.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106275
For some reason we have 2 switches on arch and add
half of arch flags in one place and half in another.
Merge these 2 switches.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D106274
We obtain the current PC is all interceptors and collectively
common interceptor code contributes to overall slowdown
(in particular cheaper str/mem* functions).
The current way to obtain the current PC involves:
4493e1: e8 3a f3 fe ff callq 438720 <_ZN11__sanitizer10StackTrace12GetCurrentPcEv>
4493e9: 48 89 c6 mov %rax,%rsi
and the called function is:
uptr StackTrace::GetCurrentPc() {
438720: 48 8b 04 24 mov (%rsp),%rax
438724: c3 retq
The new way uses address of a local label and involves just:
44a888: 48 8d 35 fa ff ff ff lea -0x6(%rip),%rsi
I am not switching all uses of StackTrace::GetCurrentPc to GET_CURRENT_PC
because it may lead some differences in produced reports and break tests.
The difference comes from the fact that currently we have PC pointing
to the CALL instruction, but the new way does not yield any code on its own
so the PC points to a random instruction in the function and symbolizing
that instruction can produce additional inlined frames (if the random
instruction happen to relate to some inlined function).
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106046
Semaphore is a portable way to park/unpark threads.
The plan is to use it to implement a portable blocking
mutex in subsequent changes. Semaphore can also be used
to efficiently wait for other things (e.g. we currently
spin to synchronize thread creation and start).
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D106071
Currently we don't lock ScopedErrorReportLock around fork
and it mostly works becuase tsan has own report_mtx that
is locked around fork and tsan reports.
However, sanitizer_common code prints some own reports
which are not protected by tsan's report_mtx. So it's better
to lock ScopedErrorReportLock explicitly.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D106048
The new GET_CURRENT_PC() can lead to spurious top inlined internal frames.
Here are 2 examples from bots, in both cases the malloc is supposed to be
the top frame (#0):
WARNING: ThreadSanitizer: signal-unsafe call inside of a signal
#0 __sanitizer::StackTrace::GetNextInstructionPc(unsigned long)
#1 malloc
Location is heap block of size 99 at 0xbe3800003800 allocated by thread T1:
#0 __sanitizer::StackTrace::GetNextInstructionPc(unsigned long)
#1 malloc
Let's strip these internal top frames from reports.
With other code changes I also observed some top frames
from __tsan::ScopedInterceptor, proactively remove these as well.
Differential Revision: https://reviews.llvm.org/D106081
We obtain the current PC is all interceptors and collectively
common interceptor code contributes to overall slowdown
(in particular cheaper str/mem* functions).
The current way to obtain the current PC involves:
4493e1: e8 3a f3 fe ff callq 438720 <_ZN11__sanitizer10StackTrace12GetCurrentPcEv>
4493e9: 48 89 c6 mov %rax,%rsi
and the called function is:
uptr StackTrace::GetCurrentPc() {
438720: 48 8b 04 24 mov (%rsp),%rax
438724: c3 retq
The new way uses address of a local label and involves just:
44a888: 48 8d 35 fa ff ff ff lea -0x6(%rip),%rsi
I am not switching all uses of StackTrace::GetCurrentPc to GET_CURRENT_PC
because it may lead some differences in produced reports and break tests.
The difference comes from the fact that currently we have PC pointing
to the CALL instruction, but the new way does not yield any code on its own
so the PC points to a random instruction in the function and symbolizing
that instruction can produce additional inlined frames (if the random
instruction happen to relate to some inlined function).
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D106046
Define the address ranges (similar to the C/C++ ones, but with the heap
range merged into the app range) and enable the sanity check.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D105629
Reuse the assembly glue code from sanitizer_common_interceptors.inc and
the handling logic from the __tls_get_addr interceptor.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D105629
The kernel supports a full 64-bit VMA, but we can use only 48 bits due
to the limitation imposed by SyncVar::GetId(). So define the address
ranges similar to the other architectures, except that the address
space "tail" needs to be made inaccessible in CheckAndProtect(). Since
it's for only one architecture, don't make an abstraction for this.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D105629
s390x requires ThreadRegistry.mtx_.opaque_storage_ to be 4-byte
aligned. Since other architectures may have similar requirements, use
the maximum thread_registry_placeholder alignment from other
sanitizers, which is 64 (LSan).
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D105629
A previous change brought the new, relaxed implementation of "on failure
memory ordering" for synchronization primitives in LLVM over to TSan
land [1]. It included the following assert:
```
// 31.7.2.18: "The failure argument shall not be memory_order_release
// nor memory_order_acq_rel". LLVM (2021-05) fallbacks to Monotonic
// (mo_relaxed) when those are used.
CHECK(IsLoadOrder(fmo));
static bool IsLoadOrder(morder mo) {
return mo == mo_relaxed || mo == mo_consume
|| mo == mo_acquire || mo == mo_seq_cst;
}
```
A previous workaround for a false positive when using an old Darwin
synchronization API assumed this failure mode to be unused and passed a
dummy value [2]. We update this value to `mo_relaxed` which is also the
value used by the actual implementation to avoid triggering the assert.
[1] https://reviews.llvm.org/D99434
[2] https://reviews.llvm.org/D21733
rdar://78122243
Differential Revision: https://reviews.llvm.org/D105844
Enable clang Thread Safety Analysis for sanitizers:
https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
Thread Safety Analysis can detect inconsistent locking,
deadlocks and data races. Without GUARDED_BY annotations
it has limited value. But this does all the heavy lifting
to enable analysis and allows to add GUARDED_BY incrementally.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105716
Mmap interceptor is not atomic in the sense that it
exposes unmapped shadow for a brief period of time.
This breaks programs that mmap over another mmap
and access the region concurrently.
Don't unmap shadow in the mmap interceptor to fix this.
Just mapping new shadow on top should be enough to zero it.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D104593
We have some significant amount of duplication around
CheckFailed functionality. Each sanitizer copy-pasted
a chunk of code. Some got random improvements like
dealing with recursive failures better. These improvements
could benefit all sanitizers, but they don't.
Deduplicate CheckFailed logic across sanitizers and let each
sanitizer only print the current stack trace.
I've tried to dedup stack printing as well,
but this got me into cmake hell. So let's keep this part
duplicated in each sanitizer for now.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D102221
Code patterns like this are common, `#` at the line beginning
(https://google.github.io/styleguide/cppguide.html#Preprocessor_Directives),
one space indentation for if/elif/else directives.
```
#if SANITIZER_LINUX
# if defined(__aarch64__)
# endif
#endif
```
However, currently clang-format wants to reformat the code to
```
#if SANITIZER_LINUX
#if defined(__aarch64__)
#endif
#endif
```
This significantly harms readability in my review. Use `IndentPPDirectives:
AfterHash` to defeat the diagnostic. clang-format will now suggest:
```
#if SANITIZER_LINUX
# if defined(__aarch64__)
# endif
#endif
```
Unfortunately there is no clang-format option using indent with 1 for
just preprocessor directives. However, this is still one step forward
from the current behavior.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D100238
Currently we have a bit of a mess related to tids:
- sanitizers re-declare kInvalidTid multiple times
- some call it kUnknownTid
- implicit assumptions that main tid is 0
- asan/memprof claim their tids need to fit into 24 bits,
but this does not seem to be true anymore
- inconsistent use of u32/int to store tids
Introduce kInvalidTid/kMainTid in sanitizer_common
and use them consistently.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101428
Commit efd254b636 ("tsan: fix deadlock in pthread_atfork callbacks")
fixed another deadlock related to atfork handling.
But builders with DCHECKs enabled reported failures of
pthread_atfork_deadlock2.c and pthread_atfork_deadlock3.c tests
related to the fact that we hold runtime locks on interceptor exit:
https://lab.llvm.org/buildbot/#/builders/70/builds/6727
This issue is somewhat inherent to the current approach,
we indeed execute user code (atfork callbacks) with runtime lock held.
Refactor fork handling to not run user code (atfork callbacks)
with runtime locks held. This change does this by installing
own atfork callbacks during runtime initialization.
Atfork callbacks run in LIFO order, so the expectation is that
our callbacks run last, right before the actual fork.
This way we lock runtime mutexes around fork, but not around
user callbacks.
Extend tests to also install after fork callbacks just to cover
more scenarios. Some tests also started reporting real races
that we previously suppressed.
Also extend tests to cover fork syscall support.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101517
We've got a user report about heap block allocator overflow.
Bump the L1 capacity of all dense slab allocators to maximum
and be careful to not page the whole L1 array in from .bss.
If OS uses huge pages, this still may cause a limited RSS increase
due to boundary huge pages, but avoiding that looks hard.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101161
COMPILER_RT_TSAN_DEBUG_OUTPUT enables TSAN_COLLECT_STATS,
which changes layout of runtime structs (some structs contain
stats when the option is enabled).
It's not OK to build runtime with the define, but tests without it.
The error is detected by build_consistency_stats/nostats.
Fix this by defining TSAN_COLLECT_STATS for tests to match the runtime.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101386
Commit efd254b636 ("tsan: fix deadlock in pthread_atfork callbacks")
fixed another deadlock related to atfork handling.
But builders with DCHECKs enabled reported failures of
pthread_atfork_deadlock2.c and pthread_atfork_deadlock3.c tests
related to the fact that we hold runtime locks on interceptor exit:
https://lab.llvm.org/buildbot/#/builders/70/builds/6727
This issue is somewhat inherent to the current approach,
we indeed execute user code (atfork callbacks) with runtime lock held.
Refactor fork handling to not run user code (atfork callbacks)
with runtime locks held. This change does this by installing
own atfork callbacks during runtime initialization.
Atfork callbacks run in LIFO order, so the expectation is that
our callbacks run last, right before the actual fork.
This way we lock runtime mutexes around fork, but not around
user callbacks.
Extend tests to also install after fork callbacks just to cover
more scenarios. Some tests also started reporting real races
that we previously suppressed.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101385
We take report/thread_registry locks around fork.
This means we cannot report any bugs in atfork handlers.
We resolved this by enabling per-thread ignores around fork.
This resolved some of the cases, but not all.
The added test triggers a race report from a signal handler
called from atfork callback, we reset per-thread ignores
around signal handlers, so we tried to report it and deadlocked.
But there are more cases: a signal handler can be called
synchronously if it's sent to itself. Or any other report
types would cause deadlocks as well: mutex misuse,
signal handler spoiling errno, etc.
Disable all reports for the duration of fork with
thr->suppress_reports and don't re-enable them around
signal handlers.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101154
... so that FreeBSD specific GetTls/glibc specific pthread_self code can be
removed. This also helps FreeBSD arm64/powerpc64 which don't have GetTls
implementation yet.
GetTls is the range of
* thread control block and optional TLS_PRE_TCB_SIZE
* static TLS blocks plus static TLS surplus
On glibc, lsan requires the range to include
`pthread::{specific_1stblock,specific}` so that allocations only referenced by
`pthread_setspecific` can be scanned.
This patch uses `dl_iterate_phdr` to collect TLS blocks. Find the one
with `dlpi_tls_modid==1` as one of the initially loaded module, then find
consecutive ranges. The boundaries give us addr and size.
This allows us to drop the glibc internal `_dl_get_tls_static_info` and
`InitTlsSize`. However, huge glibc x86-64 binaries with numerous shared objects
may observe time complexity penalty, so exclude them for now. Use the simplified
method with non-Android Linux for now, but in theory this can be used with *BSD
and potentially other ELF OSes.
This removal of RISC-V `__builtin_thread_pointer` makes the code compilable with
more compiler versions (added in Clang in 2020-03, added in GCC in 2020-07).
This simplification enables D99566 for TLS Variant I architectures.
Note: as of musl 1.2.2 and FreeBSD 12.2, dlpi_tls_data returned by
dl_iterate_phdr is not desired: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254774
This can be worked around by using `__tls_get_addr({modid,0})` instead
of `dlpi_tls_data`. The workaround can be shared with the workaround for glibc<2.25.
This fixes some tests on Alpine Linux x86-64 (musl)
```
test/lsan/Linux/cleanup_in_tsd_destructor.c
test/lsan/Linux/fork.cpp
test/lsan/Linux/fork_threaded.cpp
test/lsan/Linux/use_tls_static.cpp
test/lsan/many_tls_keys_thread.cpp
test/msan/tls_reuse.cpp
```
and `test/lsan/TestCases/many_tls_keys_pthread.cpp` on glibc aarch64.
The number of sanitizer test failures does not change on FreeBSD/amd64 12.2.
Differential Revision: https://reviews.llvm.org/D98926
This was reverted by f176803ef1 due to
Ubuntu 16.04 x86-64 glibc 2.23 problems.
This commit additionally calls `__tls_get_addr({modid,0})` to work around the
dlpi_tls_data==NULL issues for glibc<2.25
(https://sourceware.org/bugzilla/show_bug.cgi?id=19826)
GetTls is the range of
* thread control block and optional TLS_PRE_TCB_SIZE
* static TLS blocks plus static TLS surplus
On glibc, lsan requires the range to include
`pthread::{specific_1stblock,specific}` so that allocations only referenced by
`pthread_setspecific` can be scanned.
This patch uses `dl_iterate_phdr` to collect TLS blocks. Find the one
with `dlpi_tls_modid==1` as one of the initially loaded module, then find
consecutive ranges. The boundaries give us addr and size.
This allows us to drop the glibc internal `_dl_get_tls_static_info` and
`InitTlsSize` entirely. Use the simplified method with non-Android Linux for
now, but in theory this can be used with *BSD and potentially other ELF OSes.
This simplification enables D99566 for TLS Variant I architectures.
See https://reviews.llvm.org/D93972#2480556 for analysis on GetTls usage
across various sanitizers.
Differential Revision: https://reviews.llvm.org/D98926
Linux-only for now. Some mac bits stubbed out, but not tested.
Good enough for the tiny_race.c example at
https://clang.llvm.org/docs/ThreadSanitizer.html :
$ out/gn/bin/clang -fsanitize=address -g -O1 tiny_race.c
$ while true; do ./a.out || echo $? ; done
While here, also make `-fsanitize=address` work for .c files.
Differential Revision: https://reviews.llvm.org/D99795
GetTls is the range of
* thread control block and optional TLS_PRE_TCB_SIZE
* static TLS blocks plus static TLS surplus
On glibc, lsan requires the range to include
`pthread::{specific_1stblock,specific}` so that allocations only referenced by
`pthread_setspecific` can be scanned.
This patch uses `dl_iterate_phdr` to collect TLS ranges. Find the one
with `dlpi_tls_modid==1` as one of the initially loaded module, then find
consecutive ranges. The boundaries give us addr and size.
This allows us to drop the glibc internal `_dl_get_tls_static_info` and
`InitTlsSize` entirely. Use the simplified method with non-Android Linux for
now, but in theory this can be used with *BSD and potentially other ELF OSes.
In the future, we can move `ThreadDescriptorSize` code to lsan (and consider
intercepting `pthread_setspecific`) to avoid hacks in generic code.
See https://reviews.llvm.org/D93972#2480556 for analysis on GetTls usage
across various sanitizers.
Differential Revision: https://reviews.llvm.org/D98926
Make TSan runtime initialization and finalization hooks work
even if these hooks are not built in the main executable. When these
hooks are defined in another library that is not directly linked against
the TSan runtime (e.g., Swift runtime) we cannot rely on the "strong-def
overriding weak-def" mechanics and have to look them up via `dlsym()`.
Let's also define hooks that are easier to use from C-only code:
```
extern "C" void __tsan_on_initialize();
extern "C" int __tsan_on_finalize(int failed);
```
For now, these will call through to the old hooks. Eventually, we want
to adopt the new hooks downstream and remove the old ones.
This is part of the effort to support Swift Tasks (async/await and
actors) in TSan.
rdar://74256720
Reviewed By: vitalybuka, delcypher
Differential Revision: https://reviews.llvm.org/D98810
-mbranch-protection protects the LR on the stack with PAC.
When the frames are walked the LR need to be cleared.
This inline assembly later will be replaced with a new builtin.
Test: build with -DCMAKE_C_FLAGS="-mbranch-protection=standard".
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D98008
InternalScopedString uses InternalMmapVector internally
so it can be resized dynamically as needed.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D98751
size() is inconsistent with length().
In most size() use cases we can replace InternalScopedString with
InternalMmapVector.
Remove non-constant data() to avoid direct manipulations of internal
buffer. append() should be enought to modify InternalScopedString.
-mbranch-protection protects the LR on the stack with PAC.
When the frames are walked the LR need to be cleared.
This inline assembly later will be replaced with a new builtin.
Test: build with -DCMAKE_C_FLAGS="-mbranch-protection=standard".
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D98008
All check-tsan tests fail on aarch64-*-linux because HeapMemEnd() > ShadowBeg()
for the following code path:
```
#if defined(__aarch64__) && !HAS_48_BIT_ADDRESS_SPACE
ProtectRange(HeapMemEnd(), ShadowBeg());
```
Restore the behavior before D86377 for aarch64-*-linux.
Go requires 47 bits VA for tsan.
Go will run race_detector testcases unless tsan warns about "unsupported VMA range"
Author: mzh (Meng Zhuo)
Reviewed-in: https://reviews.llvm.org/D98238
Previously, on GLibc systems, the interceptor was calling __compat_regexec
(regexec@GLIBC_2.2.5) insead of the newer __regexec (regexec@GLIBC_2.3.4).
The __compat_regexec strips the REG_STARTEND flag but does not report an
error if other flags are present. This can result in infinite loops for
programs that use REG_STARTEND to find all matches inside a buffer (since
ignoring REG_STARTEND means that the search always starts from the first
character).
The underlying issue is that GLibc's dlsym(RTLD_NEXT, ...) appears to
always return the oldest versioned symbol instead of the default. This
means it does not match the behaviour of dlsym(RTLD_DEFAULT, ...) or the
behaviour documented in the manpage.
It appears a similar issue was encountered with realpath and worked around
in 77ef78a0a5.
See also https://sourceware.org/bugzilla/show_bug.cgi?id=14932 and
https://sourceware.org/bugzilla/show_bug.cgi?id=1319.
Fixes https://github.com/google/sanitizers/issues/1371
Reviewed By: #sanitizers, vitalybuka, marxin
Differential Revision: https://reviews.llvm.org/D96348
This reverts commit bde2e56071.
This patch produces a compile failure on linux amd64 environments, when
running:
ninja GotsanRuntimeCheck
I get various build errors:
../rtl/tsan_platform.h:608: error: use of undeclared identifier 'Mapping'
return MappingImpl<Mapping, Type>();
Here's a buildbot with the same failure during stage "check-tsan in gcc
build", there are other unrelated failures in there.
http://lab.llvm.org:8011/#/builders/37/builds/2831
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.
If CMAKE_C_FLAGS contains -Wno-unused-command-line-argument or
-Wno-error=unused-command-line-argument, the latter flag is passed into the
build script.
This is a re-commit. The previous version was reverted because of failing
tests.
Differential Revision: https://reviews.llvm.org/D96762
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.
This patch passes the CMAKE_C_FLAGS into the build script. Configuring with
-DCMAKE_C_FLAGS=-Wno-unused-command-line-argument allows successful testing.
Differential Revision: https://reviews.llvm.org/D96762
If an app mmaps lots of memory, a user mmap may end up
in the tsan region for traces. Shadow for this range
overlaps with shadow for other user regions.
This causes havok: from false positives to crashes.
Don't leave unmapped holes in the traces region.
Reviewed-in: https://reviews.llvm.org/D96697
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi lsan msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
Unsupported: 7
Passed : 65
Failed : 22
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Note: we need to place `_FILE_OFFSET_BITS` above `#include "sanitizer_platform.h"` to avoid `#define __USE_FILE_OFFSET64 1` in 32-bit ARM `features.h`
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
Suppress the warning:
```
'fake_shared_weak_count' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
```
The warning has been recently enabled [1], but the associated cleanup
missed this instance in Darwin code [2].
[1] 9c31e12609
[2] d48f2d7c02
Differential Revision: https://reviews.llvm.org/D94139
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
Unsupported: 7
Passed : 65
Failed : 22
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify no unneeded interceptors.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
rL254966 added `--sysroot=.` to prevent accidental including system headers.
It caused hassle to FreeBSD (D17383)/NetBSD. The next problem is that
we want to include `features.h` (usually `/usr/include/features.h`) to detect `__GLIBC__`.
At this point it seems that `--sysroot=.` adds lots of inconvenience so we disable it for now.
If there is a better way preventing accidental system header inclusion we can consider it again.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D93921
Disable the test on old systems.
pthread_cond_clockwait is supported by glibc-2.30.
It also supported by Android api 30 even though we
do not run tsan on Android.
Fixes https://github.com/google/sanitizers/issues/1259
Reviewed By: dvyukov
As discussed in the review for D87120 (specifically at
https://reviews.llvm.org/D87120#inline-831939), clean up PrintModuleMap
and DumpProcessMap usage differences. The former is only implemented for
Mac OSX, whereas the latter is implemented for all OSes. The former is
called by asan and tsan, and the latter by hwasan and now memprof, under
the same option. Simply rename the PrintModuleMap implementation for Mac
to DumpProcessMap, remove other empty PrintModuleMap implementations,
and convert asan/tsan to new name. The existing posix DumpProcessMap is
disabled for SANITIZER_MAC.
Differential Revision: https://reviews.llvm.org/D89630
- Fixing VS compiler and other cases settings this time.
Reviewers: dmajor, hans
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D89759
Revert "Fix compiler-rt build on Windows after D89640"
This reverts commit a7acee89d6.
This reverts commit d09b08919c.
Reason: breaks Linux / x86_64 build.
Adds a check to avoid symbolization when printing stack traces if the
stack_trace_format flag does not need it. While there is a symbolize
flag that can be turned off to skip some of the symbolization,
SymbolizePC() still unconditionally looks up the module name and offset.
Avoid invoking SymbolizePC() at all if not needed.
This is an efficiency improvement when dumping all stack traces as part
of the memory profiler in D87120, for large stripped apps where we want
to symbolize as a post pass.
Differential Revision: https://reviews.llvm.org/D88361
TSan relies on C++ headers, so when libc++ is being built as part of
the runtimes build, include an explicit dependency on cxx-headers which
is the same approach that's already used for other sanitizers.
Differential Revision: https://reviews.llvm.org/D88912
D28596 added SANITIZER_INTERFACE_WEAK_DEF which can guarantee `*_default_options` are always defined.
The weak attributes on the `__{asan,lsan,msan,ubsan}_default_options` declarations can thus be removed.
`MaybeCall*DefaultOptions` no longer need nullptr checks, so their call sites can just be replaced by `__*_default_options`.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D87175
The interceptor for the block variants of the API references the
function versions (via `REAL(name##_f)`). On Linux, this accesses the
underlying "real pointer", defined by the interceptor macro. So we need
to declare interceptors in the right order to avoid undefined symbol
compiler error:
```
error: no member named 'real_dispatch_async_and_wait_f' in namespace '__tsan::__interception'
```
rdar://68181542
Handle NULL address argument in the `mach_vm_[de]allocate()`
interceptors and fix test: `Assignment 2` is not valid if we weren't
able to re-allocate memory.
rdar://67680613
`dispatch_async_and_wait()` was introduced in macOS 10.14, which is
greater than our minimal deployment target. We need to forward declare
it as a "weak import" to ensure we generate a weak reference so the TSan
dylib continues to work on older systems. We cannot simply `#include
<dispatch.h>` or use the Darwin availability macros since this file is
multi-platform.
In addition, we want to prevent building these interceptors at all when
building with older SDKs because linking always fails.
Before:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA __interpose 0x000F5E68 pointer 0 libSystem _dispatch_async_and_wait_f
```
After:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA __got 0x000EC0A8 pointer 0 libSystem _dispatch_async_and_wait (weak import)
__DATA __interpose 0x000F5E78 pointer 0 libSystem _dispatch_async_and_wait (weak import)
```
This is a follow-up to D85854 and should fix:
https://reviews.llvm.org/D85854#2221529
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D86103
The linker errors caused by this revision have been addressed.
Add interceptors for `dispatch_async_and_wait[_f]()` which was added in
macOS 10.14. This pair of functions is similar to `dispatch_sync()`,
but does not force a context switch of the queue onto the caller thread
when the queue is active (and hence is more efficient). For TSan, we
can apply the same semantics as for `dispatch_sync()`.
From the header docs:
> Differences with dispatch_sync()
>
> When the runtime has brought up a thread to invoke the asynchronous
> workitems already submitted to the specified queue, that servicing
> thread will also be used to execute synchronous work submitted to the
> queue with dispatch_async_and_wait().
>
> However, if the runtime has not brought up a thread to service the
> specified queue (because it has no workitems enqueued, or only
> synchronous workitems), then dispatch_async_and_wait() will invoke the
> workitem on the calling thread, similar to the behaviour of functions
> in the dispatch_sync family.
Additional context:
> The guidance is to use `dispatch_async_and_wait()` instead of
> `dispatch_sync()` when it is necessary to mix async and sync calls on
> the same queue. `dispatch_async_and_wait()` does not guarantee
> execution on the caller thread which allows to reduce context switches
> when the target queue is active.
> https://gist.github.com/tclementdev/6af616354912b0347cdf6db159c37057
rdar://35757961
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D85854
Add interceptors for `dispatch_async_and_wait[_f]()` which was added in
macOS 10.14. This pair of functions is similar to `dispatch_sync()`,
but does not force a context switch of the queue onto the caller thread
when the queue is active (and hence is more efficient). For TSan, we
can apply the same semantics as for `dispatch_sync()`.
From the header docs:
> Differences with dispatch_sync()
>
> When the runtime has brought up a thread to invoke the asynchronous
> workitems already submitted to the specified queue, that servicing
> thread will also be used to execute synchronous work submitted to the
> queue with dispatch_async_and_wait().
>
> However, if the runtime has not brought up a thread to service the
> specified queue (because it has no workitems enqueued, or only
> synchronous workitems), then dispatch_async_and_wait() will invoke the
> workitem on the calling thread, similar to the behaviour of functions
> in the dispatch_sync family.
Additional context:
> The guidance is to use `dispatch_async_and_wait()` instead of
> `dispatch_sync()` when it is necessary to mix async and sync calls on
> the same queue. `dispatch_async_and_wait()` does not guarantee
> execution on the caller thread which allows to reduce context switches
> when the target queue is active.
> https://gist.github.com/tclementdev/6af616354912b0347cdf6db159c37057
rdar://35757961
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D85854
Disable huge pages in the TSan shadow regions when no_huge_pages_for_shadow == true (default).
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D85841
Not matching the (real) variadic declaration makes the interceptor take garbage inputs on Darwin/AArch64.
Differential Revision: https://reviews.llvm.org/D84570
This patch splits the handling of racy address and racy stack into separate
functions. If a race was already reported for the address, we can avoid the
cost for collecting the involved stacks.
This patch also removes the race condition in storing the racy address / racy
stack. This race condition allowed all threads to report the race.
This patch changes the transitive suppression of reports. Previously
suppression could transitively chain memory location and racy stacks.
Now racy memory and racy stack are separate suppressions.
Commit again, now with fixed tests.
Reviewed by: dvyukov
Differential Revision: https://reviews.llvm.org/D83625
This patch splits the handling of racy address and racy stack into separate
functions. If a race was already reported for the address, we can avoid the
cost for collecting the involved stacks.
This patch also removes the race condition in storing the racy address / racy
stack. This race condition allowed all threads to report the race.
This patch changes the transitive suppression of reports. Previously
suppression could transitively chain memory location and racy stacks.
Now racy memory and racy stack are separate suppressions.
Reviewed by: dvyukov
Differential Revision: https://reviews.llvm.org/D83625
This flag suppresses TSan FPs on Darwin. I removed this flag
prematurely and have been dealing with the fallout ever since.
This commit puts back the flag, reverting 7d1085cb [1].
[1] https://reviews.llvm.org/D55075
Use a struct to represent numerical versions instead of encoding release
names in an enumeration. This avoids the need to extend the enumeration
every time there is a new release.
Rename `GetMacosVersion() -> GetMacosAlignedVersion()` to better reflect
how this is used on non-MacOS platforms.
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D79970
Add ThreadClock:: global_acquire_ which is the last time another thread
has done a global acquire of this thread's clock.
It helps to avoid problem described in:
https://github.com/golang/go/issues/39186
See test/tsan/java_finalizer2.cpp for a regression test.
Note the failuire is _extremely_ hard to hit, so if you are trying
to reproduce it, you may want to run something like:
$ go get golang.org/x/tools/cmd/stress
$ stress -p=64 ./a.out
The crux of the problem is roughly as follows.
A number of O(1) optimizations in the clocks algorithm assume proper
transitive cumulative propagation of clock values. The AcquireGlobal
operation may produce an inconsistent non-linearazable view of
thread clocks. Namely, it may acquire a later value from a thread
with a higher ID, but fail to acquire an earlier value from a thread
with a lower ID. If a thread that executed AcquireGlobal then releases
to a sync clock, it will spoil the sync clock with the inconsistent
values. If another thread later releases to the sync clock, the optimized
algorithm may break.
The exact sequence of events that leads to the failure.
- thread 1 executes AcquireGlobal
- thread 1 acquires value 1 for thread 2
- thread 2 increments clock to 2
- thread 2 releases to sync object 1
- thread 3 at time 1
- thread 3 acquires from sync object 1
- thread 1 acquires value 1 for thread 3
- thread 1 releases to sync object 2
- sync object 2 clock has 1 for thread 2 and 1 for thread 3
- thread 3 releases to sync object 2
- thread 3 sees value 1 in the clock for itself
and decides that it has already released to the clock
and did not acquire anything from other threads after that
(the last_acquire_ check in release operation)
- thread 3 does not update the value for thread 2 in the clock from 1 to 2
- thread 4 acquires from sync object 2
- thread 4 detects a false race with thread 2
as it should have been synchronized with thread 2 up to time 2,
but because of the broken clock it is now synchronized only up to time 1
The global_acquire_ value helps to prevent this scenario.
Namely, thread 3 will not trust any own clock values up to global_acquire_
for the purposes of the last_acquire_ optimization.
Reviewed-in: https://reviews.llvm.org/D80474
Reported-by: nvanbenschoten (Nathan VanBenschoten)
Create a sanitizer_ptrauth.h header that #includes <ptrauth> when
available and defines just the required macros as "no ops" otherwise.
This should avoid the need for excessive #ifdef'ing.
Follow-up to and discussed in: https://reviews.llvm.org/D79132
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D79540
Fixes:
1. Setting the number of entries in a thread's clock to max between
the thread and the SyncClock the thread is acquiring from
2. Setting last_acquire_
Unit- and stress-test for releaseStoreAcquire added to
tests/unit/tsan_clock_test.cpp
When creating and destroying fibers in tsan a thread state is created and destroyed. Currently, a memory mapping is leaked with each fiber (in __tsan_destroy_fiber). This causes applications with many short running fibers to crash or hang because of linux vm.max_map_count.
The root of this is that ThreadState holds a pointer to ThreadSignalContext for handling signals. The initialization and destruction of it is tied to platform specific events in tsan_interceptors_posix and missed when destroying a fiber (specifically, SigCtx is used to lazily create the ThreadSignalContext in tsan_interceptors_posix). This patch cleans up the memory by makinh the ThreadState create and destroy the ThreadSignalContext.
The relevant code causing the leak with fibers is the fiber destruction:
void FiberDestroy(ThreadState *thr, uptr pc, ThreadState *fiber) {
FiberSwitchImpl(thr, fiber);
ThreadFinish(fiber);
FiberSwitchImpl(fiber, thr);
internal_free(fiber);
}
Author: Florian
Reviewed-in: https://reviews.llvm.org/D76073
Fix similar to https://reviews.llvm.org/D77111 but fow Windows.
gotsan.cpp:14071:16: error: missing terminating ' character [-Werror]
case 0b10'010: // c.lwsp (rd != x0)
^
Reported-by: Keith Randall
The above change used a binary literal that is not supported in c++11 mode when
using gcc. It was formalized into the c++14 standard and works when using that
mode to compile, so change the script to use c++14 instead.
Reviewed by: dvyukov
Differential Revision: https://reviews.llvm.org/D77111
Temporarily revert "tsan: fix leak of ThreadSignalContext for fibers"
because it breaks the LLDB bot on GreenDragon.
This reverts commit 93f7743851.
This reverts commit d8a0f76de7.
When creating and destroying fibers in tsan a thread state
is created and destroyed. Currently, a memory mapping is
leaked with each fiber (in __tsan_destroy_fiber).
This causes applications with many short running fibers
to crash or hang because of linux vm.max_map_count.
The root of this is that ThreadState holds a pointer to
ThreadSignalContext for handling signals. The initialization
and destruction of it is tied to platform specific events
in tsan_interceptors_posix and missed when destroying a fiber
(specifically, SigCtx is used to lazily create the
ThreadSignalContext in tsan_interceptors_posix). This patch
cleans up the memory by inverting the control from the
platform specific code calling the generic ThreadFinish to
ThreadFinish calling a platform specific clean-up routine
after finishing a thread.
The relevant code causing the leak with fibers is the fiber destruction:
void FiberDestroy(ThreadState *thr, uptr pc, ThreadState *fiber) {
FiberSwitchImpl(thr, fiber);
ThreadFinish(fiber);
FiberSwitchImpl(fiber, thr);
internal_free(fiber);
}
I would appreciate feedback if this way of fixing the leak is ok.
Also, I think it would be worthwhile to more closely look at the
lifecycle of ThreadState (i.e. it uses no constructor/destructor,
thus requiring manual callbacks for cleanup) and how OS-Threads/user
level fibers are differentiated in the codebase. I would be happy to
contribute more if someone could point me at the right place to
discuss this issue.
Reviewed-in: https://reviews.llvm.org/D76073
Author: Florian (Florian)
tsan while used by golang's race detector was not working on alpine
linux, since it is using musl-c instead of glibc. Since alpine is very
popular distribution for container deployments, having working race
detector would be nice. This commits adds some ifdefs to get it working.
It fixes https://github.com/golang/go/issues/14481 on golang's issue tracker.
Reviewed-in: https://reviews.llvm.org/D75849
Author: graywolf-at-work (Tomas Volf)
realeaseAcquire() is a new function added to TSan in support of the Go data-race detector.
It's semantics is:
void ThreadClock::releaseAcquire(SyncClock *sc) const {
for (int i = 0; i < kMaxThreads; i++) {
tmp = clock[i];
clock[i] = max(clock[i], sc->clock[i]);
sc->clock[i] = tmp;
}
}
For context see: https://go-review.googlesource.com/c/go/+/220419
Reviewed-in: https://reviews.llvm.org/D76322
Author: dfava (Daniel Fava)
arm64e adds support for pointer authentication, which was adopted by
libplatform to harden setjmp/longjmp and friends. We need to teach
the TSan interceptors for those functions about this.
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D76257
Fails with the following message in the error case:
```
CMake Error at /path/to/llvm-project/compiler-rt/lib/tsan/CMakeLists.txt:119 (message):
Building the TSan runtime requires at least macOS SDK 10.12
```
Fixes#44682.
https://bugs.llvm.org/show_bug.cgi?id=44682
Reviewed By: dmajor, delcypher
Differential Revision: https://reviews.llvm.org/D74501
Generally we ignore interceptors coming from called_from_lib-suppressed libraries.
However, we must not ignore critical interceptors like e.g. pthread_create,
otherwise runtime will lost track of threads.
pthread_detach is one of these interceptors we should not ignore as it affects
thread states and behavior of pthread_join which we don't ignore as well.
Currently we can produce very obscure false positives. For more context see:
https://groups.google.com/forum/#!topic/thread-sanitizer/ecH2P0QUqPs
The added test captures this pattern.
While we are here rename ThreadTid to ThreadConsumeTid to make it clear that
it's not just a "getter", it resets user_id to 0. This lead to confusion recently.
Reviewed in https://reviews.llvm.org/D74828
The removed loop clears reused for entries at the tail of a SyncClock.
The loop is redundant since those entries were already cleared by the
immediately preceding loop, which iterates over all entries in the
SyncClock (including the tail entries).
This skips calling `pthread_self` when `main_thread_identity` hasn't
been initialized yet. `main_thread_identity` is only ever assigned in
`__tsan::InitializePlatform`. This change should be relatively safe; we
are not changing behavior other than skipping the call to `pthread_self`
when `main_thread_identity == 0`.
rdar://57822138
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D71559
Summary:
The flag allows the user to specify a maximum allocation size that the
sanitizers will honor. Any larger allocations will return nullptr or
crash depending on allocator_may_return_null.
Reviewers: kcc, eugenis
Reviewed By: kcc, eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D69576
This #define is in the non-Go ppc64le build but not in the Go build.
Reviewed-in: https://reviews.llvm.org/D68046
Author: randall77 (Keith Randall)
llvm-svn: 374868
Summary:
Don't use weak exports when building tsan into a shared library for Go. gcc can't handle the pragmas used to make the weak references.
Include files that have been added since the last update to build.bat. (We should really find a better way to list all the files needed.)
Add windows version defines (WINVER and _WIN32_WINNT) to get AcquireSRWLockExclusive and ReleaseSRWLockExclusive defined.
Define GetProcessMemoryInfo to use the kernel32 version. This is kind of a hack, the windows header files should do this translation for us. I think we're not in the right family partition (we're using Desktop, but that translation only happens for App and System partitions???), but hacking the family partition seems equally gross and I have no idea what the consequences of that might be.
Patch by Keith Randall.
Reviewers: dvyukov, vitalybuka
Reviewed By: vitalybuka
Subscribers: jfb, delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D68599
llvm-svn: 373984
Summary:
It's needed to use __GLIBC_PREREQ from <features.h>
tsan didn't let us to include <features.h> by using --sysroot=. to disable system includes on
anything that is not named as "tsan*posix*", "tsan*mac*", "tsan*linux*".
See compiler-rt/lib/tsan/CMakeLists.txt
Reviewers: eugenis, dvyukov, kcc
Reviewed By: kcc
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D68176
llvm-svn: 373282
Adding annotation function variants __tsan_write_range_pc and
__tsan_read_range_pc to annotate ranged access to memory while providing a
program counter for the access.
Differential Revision: https://reviews.llvm.org/D66885
llvm-svn: 372730
I verified that the test is red without the interceptors.
rdar://40334350
Reviewed By: kubamracek, vitalybuka
Differential Revision: https://reviews.llvm.org/D66616
llvm-svn: 371439
Rename file `tsan_libdispatch.cpp -> tsan_libdispatch_interceptors.cpp`
to make it clear that it's main purpose is defining interceptors.
llvm-svn: 369289
The xpc_connection_* APIs that we are intercepting are available
starting at macOS 10.7. This is old enough so that we don't need to
guard them.
llvm-svn: 369150
Summary:
It appears that since https://reviews.llvm.org/D54889, BackgroundThread()
crashes immediately because cur_thread()-> will return a null pointer
which is then dereferenced. I'm not sure why I only see this issue on
FreeBSD and not Linux since it should also be unintialized on other platforms.
Reviewers: yuri, dvyukov, dim, emaste
Subscribers: kubamracek, krytarowski, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D65705
llvm-svn: 368103
in madvise mode, the shadow pages will be migrated only via madvise explicit calls.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D65775
llvm-svn: 368090
Like r367463, but for tsan/{benchmarks,dd,go}.
The files benchmarks aren't referenced in the build anywhere and where added
in 2012 with the comment "no Makefiles yet".
llvm-svn: 367567
See https://reviews.llvm.org/D58620 for discussion, and for the commands
I ran. In addition I also ran
for f in $(svn diff | diffstat | grep .cc | cut -f 2 -d ' '); do rg $f . ; done
and manually updated (many) references to renamed files found by that.
llvm-svn: 367463
Summary:
Combine few relatively small changes into one:
- implement internal_ptrace() and internal_clone() for NetBSD
- add support for stoptheworld based on the ptrace(2) API
- define COMPILER_RT_HAS_LSAN for NetBSD
- enable tests for NetBSD/amd64
Inspired by the original implementation by Christos Zoulas in netbsd/src for GCC.
The implementation is in theory CPU independent through well defined macros
across all NetBSD ports, however only the x86_64 version was tested.
Reviewers: mgorny, dvyukov, vitalybuka, joerg, jfb
Reviewed By: vitalybuka
Subscribers: dexonsmith, jfb, srhines, kubamracek, llvm-commits, christos
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64057
llvm-svn: 365735
Fix compilation errors related to `SANITIZER_GO` `#ifdef`s.
Refine longjmp key management. For Linux, re-implement key retrieval in
C (instead of assembly). Removal of `InitializeGuardPtr` and a final
round of cleanups will be done in the next commit.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D64092
> llvm-svn: 365513
llvm-svn: 365560
Refine longjmp key management. For Linux, re-implement key retrieval in
C (instead of assembly). Removal of `InitializeGuardPtr` and a final
round of cleanups will be done in the next commit.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D64092
llvm-svn: 365513
For x86_64, since we don't clobber %rsi (2nd argument) anymore, we don't
have to save/restore it.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D63946
llvm-svn: 365229
Factor out `ExtractLongJmpSp` helper function and move platform-specific
code to tsan_platform_{linux,mac}.cc.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D64050
llvm-svn: 364947
Switch `LongJmp` over to lookup JmpBuf via plain old (unmangled) SP.
This makes the computation of mangled SPs in the TSan assembly files
unnecessary, which will be cleaned up in follow-up revisions.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D63942
llvm-svn: 364818
TSan needs to infer which calls to setjmp/longjmp are corresponding
pairs. My understanding is, that we can't simply use the jmp_buf
address, since this buffer is just a plain data structure storing the
environment (registers) with no additional semantics, i.e., it can be
copied around and is still expected to work. So we use the stack pointer
(SP) instead.
The setjmp interceptor stores some metadata, which is then consumed in
the corresponding call to longjmp. We use the SP as an "index" (stable
identifier) into the metadata table. So far so good.
However, when mangling is used, the setjmp interceptor observes the
UNmangled SP, but the longjmp interceptor only knows the mangled value
for SP. To still correlate corresponding pairs of calls, TSan currently
derives the mangled representation in setjmp and uses it as the stable
identifer, so that longjmp can do it's lookup.
Currently, this works since "mangling" simply means XOR with a secret
value. However, in the future we want to use operations that do not
allow us to easily go from unmangled -> mangled (pointer
authentication). Going from mangled -> unmangled should still be
possible (for pointer authentication it means zeroing a few bits).
This patch is part 1 of changing set/longjmp interceptors to use the
unmangled SP for metadata lookup. Instead of deriving the mangled SP in
setjmp, we will derive the unmangled SP in longjmp. Since this change
involves difficult-to-test code, it will be done in (at least) 2 parts:
This patch only replicates the existing behavior and checks that the
newly computed value for SP matches with what we have been doing so far.
This should help me to fix issues on architectures I cannot test
directly. I tested this patch on x86-64 (Linux/Darwin) and arm64
(Darwin).
This patch will also address an orthogonal issue: there is a lot of code
duplication in the assembly files, because the
`void __tsan_setjmp(uptr sp, uptr mangled_sp)` already demands the
mangled SP. This means that the code for computing the mangled SP is
duplicated at every call site (in assembly).
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D60981
llvm-svn: 364662
Summary:
It looks like LLVM has started doing less tail duplication in this code,
or something like that, resulting in a significantly smaller number of
pop instructions (16 -> 12). Removing the check.
Reviewers: vitalybuka, dvyukov
Subscribers: kubamracek, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D63450
llvm-svn: 363615
On Linux both version of the INTERCEPT_FUNCTION macro now return true
when interception was successful. Adapt and cleanup some usages.
Also note that `&(func) == &WRAP(func)` is a link-time property, but we
do a runtime check.
Tested on Linux and macOS.
Previous attempt reverted by: 5642c3feb0
This attempt to bring order to the interceptor macro goes the other
direction and aligns the Linux implementation with the way things are
done on Windows.
Reviewed By: vitalybuka, rnk
Differential Revision: https://reviews.llvm.org/D61358
llvm-svn: 359725
They need to have same AddressSpaceView and MapUnmapCallback.
Reviewers: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D61168
llvm-svn: 359719
Changing INTERCEPT_FUNCTION to return void is not functionally correct.
IMO the best way to communicate failure or success of interception is
with a return value, not some external address comparison.
This change was also creating link errors for _except_handler4_common,
which is exported from ucrtbase.dll in 32-bit Windows.
Also revert dependent changes r359362 and r359466.
llvm-svn: 359611
Originally this code as added for 64-bit platform and was never changed.
Add static_assert to make sure that we have correct map on all platforms.
llvm-svn: 359269
In libdispatch, dispatch_apply is implemented in terms of
dispatch_apply_f. Unfortunately, this means that we can't implement the
interceptor for dispatch_apply_f by forwarding to the dispatch_apply
interceptor.
In the interceptor dispatch_apply_f, we can't use WRAP(dispatch_apply).
WRAP(dispatch_apply) -> REAL(dispatch_apply_f). Requires duplication of
some setup code.
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D59526
llvm-svn: 356467
When COMPILER_RT_INTERCEPT_LIBDISPATCH is ON the TSan runtime library
now has a dependency on the blocks runtime and libdispatch. Make sure we
set all the required linking options.
Also add cmake options for specifying additional library paths to
instruct the linker where to search for libdispatch and the blocks
runtime. This allows us to build TSan runtime with libdispatch support
without installing those libraries into default linker library paths.
`CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY` is necessary to avoid
aborting the build due to failing the link step in CMake's
check_c_compiler test.
Reviewed By: dvyukov, kubamracek
Differential Revision: https://reviews.llvm.org/D59334
llvm-svn: 356281
Including <dispatch/dispatch.h> and <Blocks.h> transitively pulls in
other system headers. Let's try to avoid that.
Blocks.h: compiler-rt already includes a blocks runtime. Just use the
header file that comes with it.
dispatch.h: Declare the bare minimum required for our implementation,
i.e., everything needed to define the interceptors, but not the
interceptors themselves. See tsan_dispatch_defs.h. I spotted a few other
places in compile-rt, where we declare libdispatch types. Maybe this
file can be moved to sanitizer_common if we deem it useful enough.
tsan_libdispatch.cc now compiles on Linux/Clang (requires support for
-fblocks). Linking still requires some manual configuration.
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D59145
llvm-svn: 356201
On Linux (and other non-Darwin platforms) we need to initialize
interceptors. Since tsan_libdispatch.cc is compiled optionally, add a
weak default implementation of `InitializeLibdispatchInterceptors`.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D59113
llvm-svn: 355717
Most libdispatch functions come in two variants: callbacks can be
specified via blocks or function pointers. Some of our interceptors for
the block variant actually forward to the function variant. However, on
Linux, `DECLARE_REAL(name)` has to appear before `REAL(name)`.
This patch reorders _f variant interceptors before _b variants
where possible and forward declares the _f variant in the remaining
cases (cyclic dependency between _f and _b interceptors).
Also rename macro to DISPATCH_INTERCEPT_ASYNC_F for better consistency.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D59067
llvm-svn: 355619
This is a new attempt for bringing TSan libdispatch support to Linux.
The main issue with the last patch (https://reviews.llvm.org/D53171) was
that we want to avoid building a separate library.
The updated plan is as follows:
1) Hide libdispatch support behind a flag: true on Darwin, false
elsewhere. If flag is specified, assume that libdispatch header and
-flbocks is available for building. This way we can directly include
the libdispatch header and rely on blocks runtime for our
implementation.
2) Optionally/weakly intercept libdispatch API functions.
This patch accomplishes 1). It compiles (without the flag enabled) on
Linux. Follow-up patches will provide 2) and enabling of tests on Linux.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D58935
llvm-svn: 355538
GetStackTrace is a implementation detail of BufferedStackTrace. Make it
a private method.
Reviewed By: vitalybuka
Differential-Revision: https://reviews.llvm.org/D58753
llvm-svn: 355232
Retrying without replacing call sites in sanitizer_common (which might
not have a symbol definition).
Add new Unwind API. This is the final envisioned API with the correct
abstraction level. It hides/slow fast unwinder selection from the caller
and doesn't take any arguments that would leak that abstraction (i.e.,
arguments like stack_top/stack_bottom).
GetStackTrace will become an implementation detail (private method) of
the BufferedStackTrace class.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58741
> llvm-svn: 355168
llvm-svn: 355172
Add new Unwind API. This is the final envisioned API with the correct
abstraction level. It hides/slow fast unwinder selection from the caller
and doesn't take any arguments that would leak that abstraction (i.e.,
arguments like stack_top/stack_bottom).
GetStackTrace will become an implementation detail (private method) of
the BufferedStackTrace class.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58741
llvm-svn: 355168
We already independently declare GetStackTrace in all (except TSan)
sanitizer runtime headers. Lets move it to sanitizer_stacktrace.h to
have one canonical way to fill in a BufferedStackFrame. Also enables us
to use it in sanitizer_common itself.
This patch defines GetStackTrace for TSan and moves the function from
ubsan_diag.cc to ubsan_diag_standalone.cc to avoid duplicate symbols
for the UBSan-ASan runtime.
Other than that this patch just moves the code out of headers and into
the correct namespace.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58651
llvm-svn: 355039
Dmitry Vyukov