... so that FreeBSD specific GetTls/glibc specific pthread_self code can be
removed. This also helps FreeBSD arm64/powerpc64 which don't have GetTls
implementation yet.
GetTls is the range of
* thread control block and optional TLS_PRE_TCB_SIZE
* static TLS blocks plus static TLS surplus
On glibc, lsan requires the range to include
`pthread::{specific_1stblock,specific}` so that allocations only referenced by
`pthread_setspecific` can be scanned.
This patch uses `dl_iterate_phdr` to collect TLS blocks. Find the one
with `dlpi_tls_modid==1` as one of the initially loaded module, then find
consecutive ranges. The boundaries give us addr and size.
This allows us to drop the glibc internal `_dl_get_tls_static_info` and
`InitTlsSize`. However, huge glibc x86-64 binaries with numerous shared objects
may observe time complexity penalty, so exclude them for now. Use the simplified
method with non-Android Linux for now, but in theory this can be used with *BSD
and potentially other ELF OSes.
This removal of RISC-V `__builtin_thread_pointer` makes the code compilable with
more compiler versions (added in Clang in 2020-03, added in GCC in 2020-07).
This simplification enables D99566 for TLS Variant I architectures.
Note: as of musl 1.2.2 and FreeBSD 12.2, dlpi_tls_data returned by
dl_iterate_phdr is not desired: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254774
This can be worked around by using `__tls_get_addr({modid,0})` instead
of `dlpi_tls_data`. The workaround can be shared with the workaround for glibc<2.25.
This fixes some tests on Alpine Linux x86-64 (musl)
```
test/lsan/Linux/cleanup_in_tsd_destructor.c
test/lsan/Linux/fork.cpp
test/lsan/Linux/fork_threaded.cpp
test/lsan/Linux/use_tls_static.cpp
test/lsan/many_tls_keys_thread.cpp
test/msan/tls_reuse.cpp
```
and `test/lsan/TestCases/many_tls_keys_pthread.cpp` on glibc aarch64.
The number of sanitizer test failures does not change on FreeBSD/amd64 12.2.
Differential Revision: https://reviews.llvm.org/D98926
This was reverted by f176803ef1 due to
Ubuntu 16.04 x86-64 glibc 2.23 problems.
This commit additionally calls `__tls_get_addr({modid,0})` to work around the
dlpi_tls_data==NULL issues for glibc<2.25
(https://sourceware.org/bugzilla/show_bug.cgi?id=19826)
GetTls is the range of
* thread control block and optional TLS_PRE_TCB_SIZE
* static TLS blocks plus static TLS surplus
On glibc, lsan requires the range to include
`pthread::{specific_1stblock,specific}` so that allocations only referenced by
`pthread_setspecific` can be scanned.
This patch uses `dl_iterate_phdr` to collect TLS blocks. Find the one
with `dlpi_tls_modid==1` as one of the initially loaded module, then find
consecutive ranges. The boundaries give us addr and size.
This allows us to drop the glibc internal `_dl_get_tls_static_info` and
`InitTlsSize` entirely. Use the simplified method with non-Android Linux for
now, but in theory this can be used with *BSD and potentially other ELF OSes.
This simplification enables D99566 for TLS Variant I architectures.
See https://reviews.llvm.org/D93972#2480556 for analysis on GetTls usage
across various sanitizers.
Differential Revision: https://reviews.llvm.org/D98926
Linux-only for now. Some mac bits stubbed out, but not tested.
Good enough for the tiny_race.c example at
https://clang.llvm.org/docs/ThreadSanitizer.html :
$ out/gn/bin/clang -fsanitize=address -g -O1 tiny_race.c
$ while true; do ./a.out || echo $? ; done
While here, also make `-fsanitize=address` work for .c files.
Differential Revision: https://reviews.llvm.org/D99795
GetTls is the range of
* thread control block and optional TLS_PRE_TCB_SIZE
* static TLS blocks plus static TLS surplus
On glibc, lsan requires the range to include
`pthread::{specific_1stblock,specific}` so that allocations only referenced by
`pthread_setspecific` can be scanned.
This patch uses `dl_iterate_phdr` to collect TLS ranges. Find the one
with `dlpi_tls_modid==1` as one of the initially loaded module, then find
consecutive ranges. The boundaries give us addr and size.
This allows us to drop the glibc internal `_dl_get_tls_static_info` and
`InitTlsSize` entirely. Use the simplified method with non-Android Linux for
now, but in theory this can be used with *BSD and potentially other ELF OSes.
In the future, we can move `ThreadDescriptorSize` code to lsan (and consider
intercepting `pthread_setspecific`) to avoid hacks in generic code.
See https://reviews.llvm.org/D93972#2480556 for analysis on GetTls usage
across various sanitizers.
Differential Revision: https://reviews.llvm.org/D98926
Make TSan runtime initialization and finalization hooks work
even if these hooks are not built in the main executable. When these
hooks are defined in another library that is not directly linked against
the TSan runtime (e.g., Swift runtime) we cannot rely on the "strong-def
overriding weak-def" mechanics and have to look them up via `dlsym()`.
Let's also define hooks that are easier to use from C-only code:
```
extern "C" void __tsan_on_initialize();
extern "C" int __tsan_on_finalize(int failed);
```
For now, these will call through to the old hooks. Eventually, we want
to adopt the new hooks downstream and remove the old ones.
This is part of the effort to support Swift Tasks (async/await and
actors) in TSan.
rdar://74256720
Reviewed By: vitalybuka, delcypher
Differential Revision: https://reviews.llvm.org/D98810
-mbranch-protection protects the LR on the stack with PAC.
When the frames are walked the LR need to be cleared.
This inline assembly later will be replaced with a new builtin.
Test: build with -DCMAKE_C_FLAGS="-mbranch-protection=standard".
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D98008
InternalScopedString uses InternalMmapVector internally
so it can be resized dynamically as needed.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D98751
size() is inconsistent with length().
In most size() use cases we can replace InternalScopedString with
InternalMmapVector.
Remove non-constant data() to avoid direct manipulations of internal
buffer. append() should be enought to modify InternalScopedString.
-mbranch-protection protects the LR on the stack with PAC.
When the frames are walked the LR need to be cleared.
This inline assembly later will be replaced with a new builtin.
Test: build with -DCMAKE_C_FLAGS="-mbranch-protection=standard".
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D98008
All check-tsan tests fail on aarch64-*-linux because HeapMemEnd() > ShadowBeg()
for the following code path:
```
#if defined(__aarch64__) && !HAS_48_BIT_ADDRESS_SPACE
ProtectRange(HeapMemEnd(), ShadowBeg());
```
Restore the behavior before D86377 for aarch64-*-linux.
Go requires 47 bits VA for tsan.
Go will run race_detector testcases unless tsan warns about "unsupported VMA range"
Author: mzh (Meng Zhuo)
Reviewed-in: https://reviews.llvm.org/D98238
Previously, on GLibc systems, the interceptor was calling __compat_regexec
(regexec@GLIBC_2.2.5) insead of the newer __regexec (regexec@GLIBC_2.3.4).
The __compat_regexec strips the REG_STARTEND flag but does not report an
error if other flags are present. This can result in infinite loops for
programs that use REG_STARTEND to find all matches inside a buffer (since
ignoring REG_STARTEND means that the search always starts from the first
character).
The underlying issue is that GLibc's dlsym(RTLD_NEXT, ...) appears to
always return the oldest versioned symbol instead of the default. This
means it does not match the behaviour of dlsym(RTLD_DEFAULT, ...) or the
behaviour documented in the manpage.
It appears a similar issue was encountered with realpath and worked around
in 77ef78a0a5.
See also https://sourceware.org/bugzilla/show_bug.cgi?id=14932 and
https://sourceware.org/bugzilla/show_bug.cgi?id=1319.
Fixes https://github.com/google/sanitizers/issues/1371
Reviewed By: #sanitizers, vitalybuka, marxin
Differential Revision: https://reviews.llvm.org/D96348
This reverts commit bde2e56071.
This patch produces a compile failure on linux amd64 environments, when
running:
ninja GotsanRuntimeCheck
I get various build errors:
../rtl/tsan_platform.h:608: error: use of undeclared identifier 'Mapping'
return MappingImpl<Mapping, Type>();
Here's a buildbot with the same failure during stage "check-tsan in gcc
build", there are other unrelated failures in there.
http://lab.llvm.org:8011/#/builders/37/builds/2831
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.
If CMAKE_C_FLAGS contains -Wno-unused-command-line-argument or
-Wno-error=unused-command-line-argument, the latter flag is passed into the
build script.
This is a re-commit. The previous version was reverted because of failing
tests.
Differential Revision: https://reviews.llvm.org/D96762
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.
This patch passes the CMAKE_C_FLAGS into the build script. Configuring with
-DCMAKE_C_FLAGS=-Wno-unused-command-line-argument allows successful testing.
Differential Revision: https://reviews.llvm.org/D96762
If an app mmaps lots of memory, a user mmap may end up
in the tsan region for traces. Shadow for this range
overlaps with shadow for other user regions.
This causes havok: from false positives to crashes.
Don't leave unmapped holes in the traces region.
Reviewed-in: https://reviews.llvm.org/D96697
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi lsan msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
Unsupported: 7
Passed : 65
Failed : 22
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Note: we need to place `_FILE_OFFSET_BITS` above `#include "sanitizer_platform.h"` to avoid `#define __USE_FILE_OFFSET64 1` in 32-bit ARM `features.h`
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
Suppress the warning:
```
'fake_shared_weak_count' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
```
The warning has been recently enabled [1], but the associated cleanup
missed this instance in Darwin code [2].
[1] 9c31e12609
[2] d48f2d7c02
Differential Revision: https://reviews.llvm.org/D94139
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
Unsupported: 7
Passed : 65
Failed : 22
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).
This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:
* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)
Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).
Many sanitizer features are available now.
```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
Unsupported : 185
Passed : 512
Expectedly Failed: 1
Failed : 12
% ninja check-ubsan check-ubsan-minimal check-memprof # all passed
% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
Unsupported : 264
Passed : 80
Expectedly Failed: 8
Failed : 32
% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
Unsupported : 6
Passed : 764
Expectedly Failed: 2
Failed : 58
% ninja check-tsan
Testing Time: 23.21s
Unsupported : 86
Passed : 295
Expectedly Failed: 1
Failed : 25
```
Used `ASAN_OPTIONS=verbosity=2` to verify no unneeded interceptors.
Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D93848
rL254966 added `--sysroot=.` to prevent accidental including system headers.
It caused hassle to FreeBSD (D17383)/NetBSD. The next problem is that
we want to include `features.h` (usually `/usr/include/features.h`) to detect `__GLIBC__`.
At this point it seems that `--sysroot=.` adds lots of inconvenience so we disable it for now.
If there is a better way preventing accidental system header inclusion we can consider it again.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D93921
Disable the test on old systems.
pthread_cond_clockwait is supported by glibc-2.30.
It also supported by Android api 30 even though we
do not run tsan on Android.
Fixes https://github.com/google/sanitizers/issues/1259
Reviewed By: dvyukov
As discussed in the review for D87120 (specifically at
https://reviews.llvm.org/D87120#inline-831939), clean up PrintModuleMap
and DumpProcessMap usage differences. The former is only implemented for
Mac OSX, whereas the latter is implemented for all OSes. The former is
called by asan and tsan, and the latter by hwasan and now memprof, under
the same option. Simply rename the PrintModuleMap implementation for Mac
to DumpProcessMap, remove other empty PrintModuleMap implementations,
and convert asan/tsan to new name. The existing posix DumpProcessMap is
disabled for SANITIZER_MAC.
Differential Revision: https://reviews.llvm.org/D89630
- Fixing VS compiler and other cases settings this time.
Reviewers: dmajor, hans
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D89759
Revert "Fix compiler-rt build on Windows after D89640"
This reverts commit a7acee89d6.
This reverts commit d09b08919c.
Reason: breaks Linux / x86_64 build.
Fangrui Song