If sanitizer cannot determine name of the module it
will use "<unknown module>". Then it can be suppressed
if needed.
Reviewed By: kda
Differential Revision: https://reviews.llvm.org/D121674
Fix darwin interface test after D121464. asan_rtl_x86_64.S is not
available on Darwin.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D121636
For ppc64 PIE, it seems that [0xa00000000000,0xc00000000000) may be occupied
which will lead to a segfault in certain kernel configurations
(clang-ppc64le-rhel). Use the `!kUsingConstantSpaceBeg` code path like Fuchsia.
Reviewed By: jsji
Differential Revision: https://reviews.llvm.org/D121257
.thumb_func was not switching mode until [1]
so it did not show up but now that .thumb_func (without argument) is
switching mode, its causing build failures on armv6 ( rpi0 ) even when
build is explicitly asking for this file to be built with -marm (ARM
mode), therefore use DEFINE_COMPILERRT_FUNCTION macro to add function
header which considers arch and mode from compiler cmdline to decide if
the function is built using thumb mode or arm mode.
[1] https://reviews.llvm.org/D101975
Note that it also needs https://reviews.llvm.org/D99282
Reviewed By: peter.smith, MaskRay
Differential Revision: https://reviews.llvm.org/D104183
This clarifies that this is an LLVM specific variable and avoids
potential conflicts with other projects.
Differential Revision: https://reviews.llvm.org/D119918
This is a part of optimized callback reverts. This is needed to export the callbacks from the rt-asan libraries.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D121464
At present compiler-rt cross compiles for armv6 ( -march=armv6 ) but includes
dmb instructions which are only available in armv7+ this causes SIGILL on
clang+compiler-rt compiled components on rpi0w platforms.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D99282
The false positive fixed by commit f831d6fc80
("tsan: fix false positive during fd close") still happens episodically
on the added more stressful test which does just open/close.
I don't have a coherent explanation as to what exactly happens
but the fix fixes the false positive on this test as well.
The issue may be related to lost writes during asynchronous MADV_DONTNEED.
I've debugged similar unexplainable false positive related to freed and
reused memory and at the time the only possible explanation I found is that
an asynchronous MADV_DONTNEED may lead to lost writes. That's why commit
302ec7b9bc ("tsan: add memory_limit_mb flag") added StopTheWorld around
the memory flush, but unfortunately the commit does not capture these findings.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D121363
Use the correct types for OFF_T, __sanitizer_time_t and
__sanitizer_dirent and forward time_t related functions
to fix using compiler-rt with 32-bit musl libc.
Also redirect the time_t functions that are affected by
https://musl.libc.org/time64.html to use their 64-bit
ABI names.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D119358
follow up to 0a4dec6cc2.
add unsupported for s390 (SEGV)
restore line that s390 complains, so following asserts work.
Differential Revision: https://reviews.llvm.org/D121326
FdClose is a subjet to the same atomicity problem as MemoryRangeFreed
(memory state is not "monotoic" wrt race detection).
So we need to lock the thread slot in FdClose the same way we do
in MemoryRangeFreed.
This fixes the modified stress.cpp test.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D121143
This is useful when building a complete toolchain to ensure that CRT
is built after builtins but before the rest of the compiler-rt.
Differential Revision: https://reviews.llvm.org/D120682
This fixes a warning about comparing mismatched types. Since `mmap()` already returns a `void *` use that as the pointer type for comparison.
Reviewed By: kyulee, zequanwu
Differential Revision: https://reviews.llvm.org/D120945
Pass LIBCXX_HAS_PTHREAD_LIB, LIBCXX_HAS_RT_LIB and LIBCXXABI_HAS_PTHREAD_LIB
through to the custom lib++ builds so that libfuzzer doesn't end up with a .deplibs section that
links against those libraries when the variables are set to false.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D120946
The upstream project ships CMake rules for building vanilla gtest/gmock which conflict with the names chosen by LLVM. Since LLVM's build rules here are quite specific to LLVM, prefixing them to avoid collision is the right thing (i.e. there does not appear to be a path to letting someone *replace* LLVM's googletest with one they bring, so co-existence should be the goal).
This allows LLVM to be included with testing enabled within projects that themselves have a dependency on an official gtest release.
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D120789
These are failing on our silent bot:
https://lab.llvm.org/staging/#/builders/162/builds/358
$ <run cmd>
main
foo
bar
baz
SanitizerCoverage: ./sanitizer_coverage_trace_pc_guard-dso.cpp.tmp.2122517.sancov: 2 PCs written
SanitizerCoverage: ./sanitizer_coverage_trace_pc_guard-dso.cpp.tmp_2.so.2122517.sancov: 1 PCs written
SanitizerCoverage: ./sanitizer_coverage_trace_pc_guard-dso.cpp.tmp_1.so.2122517.sancov: 1 PCs written
$ <sancov cmd>
ERROR: Coverage points in binary and .sancov file do not match.
Also reproduces if you build for Thumb on v8 hardware.
Doesn't fail when built with Arm only code so I guess the Thumb mode bit
in the PCs might be the issue.
Let `archive-aix-libatomic` accept additional argument to customize name of output atomic library.
Reviewed By: jsji
Differential Revision: https://reviews.llvm.org/D120534
See post-commit discussion on https://reviews.llvm.org/D120305.
This change breaks the clang-ppc64le-rhel buildbot, though
there is suspicion that it's an issue with the bot. The change
also had a larger than expected impact on compile-time and
code-size.
This reverts commit 3c4ed02698
and some followup changes.
This works with glibc crt1.o (its crt1.o is essentially Scrt1.o plus (if static
PIE is supported for the arch) _dl_relocate_static_pie) but looks wacky.
musl crt1.o is not built with an explicit -fno-pic/-fpie/-fpic. If it was built
with a non-default-pie GCC/Clang, the linker might complain about absolute
relocations referencing _init/_fini for the -pie link:
```
ld: error: relocation R_X86_64_32 cannot be used against symbol '_fini'; recompile with -fPIC
>>> defined in obj/crt/crt1.o
>>> referenced by crt1.c
>>> obj/crt/crt1.o:(_start_c)
```
Sanitizer coverage point should be the previous instruction PC of the
caller and the offset to the previous instruction might be different
on each CPU architecture.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D119233
Aligned new does not require size to be a multiple of alignment, so
memalign is the correct choice instead of aligned_alloc.
Fixes false reports for unaligned sizes.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D119161
The stack trace addresses may be odd (normally addresses should be even), but
seems a good compromise when the instruction length (2,4,6) cannot be detected
easily.
Reviewed By: uweigand
Differential Revision: https://reviews.llvm.org/D120432
There should be 1-bit unused field between tid field and is_atomic field of Shadow.
Reviewed By: dvyukov, vitalybuka
Differential Revision: https://reviews.llvm.org/D119417
There should be 1-bit unused field between tid field and is_atomic field of Shadow.
Reviewed By: dvyukov, vitalybuka
Differential Revision: https://reviews.llvm.org/D119417
clang-cl doesn't support -target <target>, instead it only supports
--target=<target> so building a RUNTIME configuration for clang-cl
ended up in never building builtins. Which in turn lead to clang-cl
not being able to find the runtime libraries because we depend
on the compiler_rt.builtins.lib being in the runtime dir for the
Driver to add it as a candidate.
I don't think this should have any downsides since most the code
these days are using --target=<target> instead of the old syntax.
Reviewed By: mstorsjo
Differential Revision: https://reviews.llvm.org/D120308
This patch extends compiler-rt's cmake config to build macOS builtins with both macOS and Mac Catalyst support.
This is done by telling the compiler to emit macho files with two build version load commands.
Differential Revision: https://reviews.llvm.org/D118875
Port the change to compiler-rt/lib/fuzzer/FuzzerTracePC.cpp .
Update RISCV to use PC-2: this is coarse (C extension may be disabled) but
sufficient for pure symbolization purpose.
The commit is separate from D120362 so that bisecting/reverting is easier.
x86 uses offset 1 while most RISC architectures use offset 4.
Check x86 first to prevent changes for new RISC architectures.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D120362
This is a restricted alternative to D91605
<https://reviews.llvm.org/D91605> which only works on Solaris 11.4 SRU 10+,
but would break the build on Solaris 11.3 and Illumos which lack
`dlpi_tls_modid`.
Apart from that, the patch is trivial. One caveat is that the
`sanitizer_common` and `asan` tests need to be linked explicitly with `ld
-z relax=transtls` on Solaris/amd64 since the archives with calls to
`__tls_get_addr` are linked in directly.
Tested on `amd64-pc-solaris2.11`, `sparcv9-sun-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D120048
This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.
The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.
This commit also includes the changes reviewed separately in D120093.
Differential Revision: https://reviews.llvm.org/D120103
TLS teardown is currently broken, as we unpoison the shadow a little bit
and to the right of the TLS section, rather than the full TLS section
itself. This currently breaks at -O0, and breaks with some upcoming
globals code that I have.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D120080
This reverts commit 85355a560a.
This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.
The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.
Differential Revision: https://reviews.llvm.org/D118653
Turns out the test was working by accident: we need to ensure
TSan instrumentation is not called from the fork() hook, otherwise the
tool will deadlock. Previously it worked because alloc_free_blocks() got
inlined into __tsan_test_only_on_fork(), but it cannot always be the
case.
Adding __attribute__((disable_sanitizer_instrumentation)) will prevent
TSan from instrumenting alloc_free_blocks().
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D120050
This reverts commit 910a642c0a.
There are serious correctness issues with the current approach: __sync_*
routines which are not actually atomic should not be enabled by default.
I'll continue discussion on the review.
ARMv5 and older architectures don’t support SMP and do not have atomic instructions. Still they’re in use in IoT world, where one has to stick to libgcc.
Reviewed By: mstorsjo
Differential Revision: https://reviews.llvm.org/D116088
Note: the term "libgcc" refers to the all of `libgcc.a`, `libgcc_eh.a`,
and `libgcc_s.so`.
Enabling libunwind as a replacement for libgcc on Linux has proven to be
challenging since libgcc_s.so is a required dependency in the [Linux
standard base][5]. Some software is transitively dependent on libgcc
because glibc makes hardcoded calls to functions in libgcc_s. For example,
the function `__GI___backtrace` eventually makes its way to a [hardcoded
dlopen to libgcc_s' _Unwind_Backtrace][1]. Since libgcc_{eh.a,s.so} and
libunwind have the same ABI, but different implementations, the two
libraries end up [cross-talking, which ultimately results in a
segfault][2].
To solve this problem, libunwind needs to build a “libgcc”. That is, link
the necessary functions from compiler-rt and libunwind into an archive
and shared object that advertise themselves as `libgcc.a`, `libgcc_eh.a`,
and `libgcc_s.so`, so that glibc’s baked calls are diverted to the
correct objects in memory. Fortunately for us, compiler-rt and libunwind
use the same ABI as the libgcc family, so the problem is solvable at the
llvm-project configuration level: no program source needs to be edited.
Thus, the end result is for a user to configure their LLVM build with a
flag that indicates they want to archive compiler-rt/unwind as libgcc.
We achieve this by compiling libunwind with all the symbols necessary
for compiler-rt to emulate the libgcc family, and then generate symlinks
named for our "libgcc" that point to their corresponding libunwind
counterparts.
We alternatively considered patching glibc so that the source doesn't
directly refer to libgcc, but rather _defaults_ to libgcc, so that a
system preferring compiler-rt/libunwind can point to these libraries
at the config stage instead. Even if we modified the Linux standard
base, this alternative won't work because binaries that are built using
libgcc will still end up having crosstalk between the differing
implementations.
This problem has been solved in this manner for [FreeBSD][3], and this
CL has been tested against [Chrome OS][4].
[1]: https://github.com/bminor/glibc/blob/master/sysdeps/arm/backtrace.c#L68
[2]: https://bugs.chromium.org/p/chromium/issues/detail?id=1162190#c16
[3]: https://github.com/freebsd/freebsd-src/tree/main/lib/libgcc_s
[4]: https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2945947
[5]: https://refspecs.linuxbase.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/libgcc-s.html
Differential Revision: https://reviews.llvm.org/D108416
This reverts commit e6999040f5.
Update test to fix signed int comparison warning, fix whitespace in
compiler-rt MIBEntryDef.inc file.
Differential Revision: https://reviews.llvm.org/D117256
This reverts commit 857ec0d01f.
Fixes -DLLVM_ENABLE_MODULES=On build by adding the new textual
header to the modulemap file.
Reviewed in https://reviews.llvm.org/D117722
This reverts commit 0f73fb18ca.
Use llvm/Profile/MIBEntryDef.inc instead of relative path.
Generated the raw profile data with `-mllvm
-enable-name-compression=false` so that builbots where the reader is
built without zlib do not fail.
Also updated the test build instructions.
This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.
The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.
Differential Revision: https://reviews.llvm.org/D118653
This patch refactors out the MemInfoBlock definition into a macro based
header which can be included to generate enums, structus and code for
each field recorded by the memprof profiling runtime.
Differential Revision: https://reviews.llvm.org/D117722
Third attempt to fix a bot failure from
634da7a1c6 on an Android bot:
https://lab.llvm.org/buildbot#builders/77/builds/14339
My last attempt used an approach from another test where chmod was not
working of using a bad character in the path name. But it looks like
this trick only works on Windows.
Instead, restore the original version of this test before my change at
634da7a1c6 and move the bad path test to
a new test file, marking it unsupported on Android.
Attempts to fix a bot failure from
634da7a1c6 on an Android bot:
https://lab.llvm.org/buildbot#builders/77/builds/14339
It appears that the chmod is not making the directory unwritable as
expected on this system for some reason. Adopt an approach used in
compiler-rt/test/fuzzer/fuzzer-dirs.test for systems with
non-functioning chmod by including illegal characters in directory.
Add a DirExists mechanism, modeled after FileExists. Use it to guard
creation of the report path directory.
This should avoid failures running the sanitizer in a sandbox where the
file creation attempt causes hard failures, even for an existing
directory. Problem reported on D109794 for ChromeOS in sandbox
(https://issuetracker.google.com/209296420).
Differential Revision: https://reviews.llvm.org/D119495
All platforms return the main executable as the first dl_phdr_info.
FreeBSD, NetBSD, Solaris, and Linux-musl place the executable name
in the dlpi_name field of this entry. It appears that only Linux-glibc
uses the empty string.
To make this work generically on all platforms, unconditionally skip the first
object (like is currently done for FreeBSD and NetBSD). This fixes first DSO
detection on Linux-musl with clang -shared-libsan/-shared-libasan and GCC's
default. It also would likely fix detection on Solaris/Illumos if it were to
gain PIE support (since dlpi_addr would not be NULL).
Additionally, only skip the Linux VDSO on linux.
Finally, use the empty string as the "seen first dl_phdr_info"
marker rather than (char *)-1. If there was no other object, we
would try to dereference it for a string comparison.
Reviewed By: MaskRay, vitalybuka
Differential Revision: https://reviews.llvm.org/D119515
Follow-up to 458ead66dc, which replaced the bespoke CMakeLists.txt
file for building a custom instrumented libc++ with an invocation of the
runtimes build.
In the the bespoke CMakeLists.txt, the LIBCXX_CXX_ABI setting was forced
to libcxxabi, but this was not done for the CMake invocation for the
runtimes build. This would cause CMake configuration issues on platforms
where the default LIBCXX_CXX_ABI setting is not libcxxabi, such as
FreeBSD.
Add `-DLIBCXX_CXX_ABI=libcxxabi` to that invocation, to make sure the
custom instrumented libc++ always uses the expected ABI.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D119554
An infinite loop without any effects is illegal C++ and can be optimized
away by the compiler.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D119575
This is a follow up to 4f3f4d6722
("sanitizer_common: fix __sanitizer_get_module_and_offset_for_pc signature mismatch")
which fixes a similar problem for msan build.
I am getting the following error compiling a unit test for code that
uses sanitizer_common headers and googletest transitively includes
sanitizer interface headers:
In file included from third_party/gwp_sanitizers/singlestep_test.cpp:3:
In file included from sanitizer_common/sanitizer_common.h:19:
sanitizer_interface_internal.h:41:5: error: typedef redefinition with different types
('struct __sanitizer_sandbox_arguments' vs 'struct __sanitizer_sandbox_arguments')
} __sanitizer_sandbox_arguments;
common_interface_defs.h:39:3: note: previous definition is here
} __sanitizer_sandbox_arguments;
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D119546
We run bots on a shared machine and under high load
this test sometimes segfaults.
https://lab.llvm.org/buildbot/#/builders/185/builds/1368
==1952234==XRay FDR init successful.
==1952234==XRay FDR: Not flushing to file, 'no_file_flush=true'.
<...>fdr-reinit.cpp.script: line 4: 1952234 Segmentation fault
XRAY_OPTIONS="verbosity=1" <...>/fdr-reinit.cpp.tmp
Looking at the printed output I think it's happening at:
// Finally, we should signal the sibling thread to stop.
keep_going.clear(std::memory_order_release);
Disabling the test while I try to reproduce.
We are moving away from building the runtimes with LLVM_ENABLE_PROJECTS,
however the documentation was largely outdated. This commit updates all
the documentation I could find to use LLVM_ENABLE_RUNTIMES instead of
LLVM_ENABLE_PROJECTS for building runtimes.
Note that in the near future, libcxx, libcxxabi and libunwind will stop
supporting being built with LLVM_ENABLE_PROJECTS altogether. I don't know
what the plans are for other runtimes like libc, openmp and compiler-rt,
so I didn't make any changes to the documentation that would imply
something for those projects.
Once this lands, I will also cherry-pick this on the release/14.x branch
to make sure that LLVM's documentation is up-to-date and reflects what
we intend to support in the future.
Differential Revision: https://reviews.llvm.org/D119351
Summary:
This is neccessary to support solaris/sparc9 where some userspace
addresses have all top bits set, as well as, potentially, kernel memory
on aarch64.
This change does not update the compiler side (HWASan IR pass) which
needs to be done separately for the affected targets.
Reviewers: ro, vitalybuka
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D91827
Fix passing the port and IP address with the wrong endianness
in get_sock_peer_name() that causes the connect() to fail inside
without an outgoing network interface (it's trying to connect
to 1.0.0.127 instead of 127.0.0.1).
Differential Revision: https://reviews.llvm.org/D119461
We don't need precise control over the low-level behavior of these testcases so
C should be preferred for readability.
The new testcases test (1) the base dlopen case (running initializers and
deinitializers), and (2) the serial case of dlopen; dlclose; dlopen; dlclose,
where we expect the initializers and deinitializers to be run twice.
D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to a more
typical casing style for non-macro symbols.
Reapplying with changes to the symbol names in various platform
specific code, which I missed previously.
Differential Revision: https://reviews.llvm.org/D118783
D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to a more
typical casing style for non-macro symbols.
Differential Revision: https://reviews.llvm.org/D118783
As reported in Issue #41838, `clang` doesn't correctly implement `long
double` on 32-bit Solaris/SPARC: the psABI requires this to be an 128-bit
type. Four sanitizer tests currently `FAIL` for this reason.
While there is a WIP patch to fix `clang` (D89130
<https://reviews.llvm.org/D89130>), it isn't complete yet and I've hit so
many brick walls while trying to finish it that I'm unsure if I ever will.
This patch therefore `XFAIL`s those tests in the meantime.
Tested on `sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D119016
Enabling `sanitizer_common` tests on Solaris (D91606
<https://reviews.llvm.org/D91606>) and SPARC (D91608
<https://reviews.llvm.org/D91608>) uncovered a sparcv9 failure
SanitizerCommon-Unit :: ./Sanitizer-sparcv9-Test/CompactRingBuffer.int64
like this:
[ RUN ] CompactRingBuffer.int64
==24576==ERROR: SanitizerTool failed to deallocate 0x2000 (8192) bytes at address 0xffffffff7f59b000
==24576==Sanitizer CHECK failed: /vol/llvm/src/llvm-project/local/compiler-rt/lib/sanitizer_common/sanitizer_posix.cpp:61 (("unable to unmap" && 0)) != (0) (0, 0)
The problem is that the original allocation via
`MmapAlignedOrDieOnFatalError` is for 4 kB, but the Solaris/sparcv9
pagesize is 8 kB. So the initial allocation is for 12 kB, rounded to a
multiple of the pagesize. Afterwards, the unneeded rest is unmapped again,
but this fails since the address is not pagesize-aligned.
This patch avoids this by aligning the end of the mapping to the pagesize.
With D91827 <https://reviews.llvm.org/D91827> added, the test `PASS`es on
`sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D91615
As described in Issue #53523, the
`DenseMapCustomTest.DefaultMinReservedSizeTest` test FAILs on Solaris/SPARC
(both 32 and 64-bit):
/vol/llvm/src/llvm-project/local/compiler-rt/lib/sanitizer_common/tests/sanitizer_dense_map_test.cpp:399:
Failure
Expected: (MemorySize) != (Map.getMemorySize()), actual: 8192 vs 8192
This happens because SPARC, unlike many other CPUs, uses an 8 kB pagesize.
Fixed by incorporating the pagesize into the calculations of
`ExpectedInitialBucketCount` and derived values.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D118771
The SANITIZER_GO code path reports an undefined symbol error for dlsym.
```
FAILED: projects/compiler-rt/lib/tsan/rtl/CMakeFiles/GotsanRuntimeCheck /tmp/RelA/projects/compiler-rt/lib/tsan/rtl/CMakeFiles/GotsanRuntimeCheck
```
This symbol has been exported (as an internal GLIBC_PRIVATE symbol) from libc.so.6 starting with glibc 2.34. glibc uses it internally for its libthread_db implementation to enable thread debugging on GDB, so it is unlikely to go away for now.
Fixes#52989.
Reviewed By: #sanitizers, MaskRay, vitalybuka
Differential Revision: https://reviews.llvm.org/D119007
This patch updates the MachO platform (both the ORC MachOPlatform class and the
ORC-Runtime macho_platform.* files) to use allocation actions, rather than EPC
calls, to transfer the initializer information scraped from each linked object.
Interactions between the ORC and ORC-Runtime sides of the platform are
substantially redesigned to accomodate the change.
The high-level changes in this patch are:
1. The MachOPlatform::setupJITDylib method now calls into the runtime to set up
a dylib name <-> header mapping, and a dylib state object (JITDylibState).
2. The MachOPlatformPlugin builds an allocation action that calls the
__orc_rt_macho_register_object_platform_sections and
__orc_rt_macho_deregister_object_platform_sections functions in the runtime
to register the address ranges for all "interesting" sections in the object
being allocated (TLS data sections, initializers, language runtime metadata
sections, etc.).
3. The MachOPlatform::rt_getInitializers method (the entry point in the
controller for requests from the runtime for initializer information) is
replaced by MachOPlatform::rt_pushInitializers. The former returned a data
structure containing the "interesting" section address ranges, but these are
now handled by __orc_rt_macho_register_object_platform_sections. The new
rt_pushInitializers method first issues a lookup to trigger materialization
of the "interesting" sections, then returns the dylib dependence tree rooted
at the requested dylib for dlopen to consume. (The dylib dependence tree is
returned by rt_pushInitializers, rather than being handled by some dedicated
call, because rt_pushInitializers can alter the dependence tree).
The advantage of these changes (beyond the performance advantages of using
allocation actions) is that it moves more information about the materialized
portions of the JITDylib into the executor. This tends to make the runtime
easier to reason about, e.g. the implementation of dlopen in the runtime is now
recursive, rather than relying on recursive calls in the controller to build a
linear data structure for consumption by the runtime. This change can also make
some operations more efficient, e.g. JITDylibs can be dlclosed and then
re-dlopened without having to pull all initializers over from the controller
again.
In addition to the high-level changes, there are some low-level changes to ORC
and the runtime:
* In ORC, at ExecutionSession teardown time JITDylibs are now destroyed in
reverse creation order. This is on the assumption that the ORC runtime will be
loaded into an earlier dylib that will be used by later JITDylibs. This is a
short-term solution to crashes that arose during testing when the runtime was
torn down before its users. Longer term we will likely destroy dylibs in
dependence order.
* toSPSSerializable(Expected<T> E) is updated to explicitly initialize the T
value, allowing it to be used by Ts that have explicit constructors.
* The ORC runtime now (1) attempts to track ref-counts, and (2) distinguishes
not-yet-processed "interesting" sections from previously processed ones. (1)
is necessary for standard dlopen/dlclose emulation. (2) is intended as a step
towards better REPL support -- it should enable future runtime calls that
run only newly registered initializers ("dlopen_more", "dlopen_additions",
...?).
Adds construction from std::string, an ostream &operator<< and std::hash
specialization. Also adds unit tests for each of these operations, as well as
tests for copy construction and assignment.
These new operations will be used in upcoming macho_platform patches.
Aligned new does not require size to be a multiple of alignment, so
memalign is the correct choice instead of aligned_alloc.
Fixes false reports for unaligned sizes.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D119161
Similar to 60cc1d3218 for NetBSD, add aliases and interceptors for the
following pthread related functions:
- pthread_cond_init(3)
- pthread_cond_destroy(3)
- pthread_cond_signal(3)
- pthread_cond_broadcast(3)
- pthread_cond_wait(3)
- pthread_mutex_init(3)
- pthread_mutex_destroy(3)
- pthread_mutex_lock(3)
- pthread_mutex_trylock(3)
- pthread_mutex_unlock(3)
- pthread_rwlock_init(3)
- pthread_rwlock_destroy(3)
- pthread_rwlock_rdlock(3)
- pthread_rwlock_tryrdlock(3)
- pthread_rwlock_wrlock(3)
- pthread_rwlock_trywrlock(3)
- pthread_rwlock_unlock(3)
- pthread_once(3)
- pthread_sigmask(3)
In FreeBSD's libc, a number of internal aliases of the pthread functions
are invoked, typically with an additional prefixed underscore, e.g.
_pthread_cond_init() and so on.
ThreadSanitizer needs to intercept these aliases too, otherwise some
false positive reports about data races might be produced.
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D119034
Fixes segfaults on x86_64 caused by instrumented code running before
shadow is set up.
Reviewed By: pcc
Differential Revision: https://reviews.llvm.org/D118171
This fixes the following error:
sanitizer_interface_internal.h:77:7: error: conflicting types for
'__sanitizer_get_module_and_offset_for_pc'
int __sanitizer_get_module_and_offset_for_pc(
common_interface_defs.h:349:5: note: previous declaration is here
int __sanitizer_get_module_and_offset_for_pc(void *pc, char *module_path,
I am getting it on a code that uses sanitizer_common (includes internal headers),
but also transitively gets includes of the public headers in tests
via an internal version of gtest.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D118910
We (Linaro) still have the occasional failure here due
to high load on a shared buildbot machine.
We are looking into general soloutions but perhaps this
will help in the meantime.
Instead of calling asan_report.* directly from assembly code they have been replaced with corresponding asan_report.*_asm function, which call asan_report.*. All asan_report.* are now undefined weak symbols, which allows DSOs to link when z defs is used.
Reviewed By: MaskRay, morehouse
Differential Revision: https://reviews.llvm.org/D118813
I was looking around and noticed that builtins for iossim, tvossim
and watchossim was missing arm64 builds, while apple's clang
toolchain ship with these. After a bit of searching around it just
seems like these are not listed correctly in CMake to be enabled.
I enabled just arm64 since I saw that Apple clang didn't include
arm64e.
Reviewed By: t.p.northover
Differential Revision: https://reviews.llvm.org/D118759
Unfortunately, the `sanitizer_common` tests are disabled on many targets
that are supported by `sanitizer_common`, making it easy to miss issues
with that support. This patch enables SPARC testing.
Beside the enabling proper, the patch fixes (together with D91607
<https://reviews.llvm.org/D91607>) the failures of the `symbolize_pc.cpp`,
`symbolize_pc_demangle.cpp`, and `symbolize_pc_inline.cpp` tests. They
lack calls to `__builtin_extract_return_addr`. When those are added, they
`PASS` when compiled with `gcc`. `clang` incorrectly doesn't implement a
non-default `__builtin_extract_return_addr` on several targets, SPARC
included.
Because `__builtin_extract_return_addr(__builtin_return_addr(0))` is quite
a mouthful and I'm uncertain if the code needs to compile with msvc which
appparently has it's own `_ReturnAddress`, I've introduced
`__sanitizer_return_addr` to hide the difference and complexity. Because
on 32-bit SPARC `__builtin_extract_return_addr` differs when the calling
function returns a struct, I've added a testcase for that.
There are a couple more tests failing on SPARC that I will deal with
separately.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D91608
setjmp can return twice, but PostDominatorTree is unaware of this. as
such, it overestimates postdominance, leaving some cases (see attached
compiler-rt) where memory does not get untagged on return. this causes
false positives later in the program execution.
this is a crude workaround to unblock use-after-scope for now, in the
longer term PostDominatorTree should bemade aware of returns_twice
function, as this may cause problems elsewhere.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D118647
These tests appear to be causing timeouts on our silent
Thumbv7 bot: https://lab.llvm.org/staging/#/builders/162/builds/260
It is possible they would complete given enough time. value-profile-switch
seems to take a long time even on a powerful Armv8 machine.
The definition of the MemInfoBlock is shared between the memprof
compiler-rt runtime and llvm/lib/ProfileData/. This change removes the
memprof_meminfoblock header and moves the struct to the shared include
file. To enable this sharing, the Print method is moved to the
memprof_allocator (the only place it is used) and the remaining uses are
updated to refer to the MemInfoBlock defined in the MemProfData.inc
file.
Also a couple of other minor changes which improve usability of the
types in MemProfData.inc.
* Update the PACKED macro to handle commas.
* Add constructors and equality operators.
* Don't initialize the buildid field.
Differential Revision: https://reviews.llvm.org/D116780
`__builtin_c[tl]z` accepts `unsigned int` argument that is not always the
same as uint32_t. For example, `unsigned int` is uint16_t on MSP430.
Reviewed By: aykevl
Differential Revision: https://reviews.llvm.org/D86547
Use s[iu]_int instead of `(unsigned) int` and d[ui]_int instead of
`(unsigned) long long` for LibCall arguments.
Note: the `*vfp` LibCall versions were NOT touched.
Reviewed By: aykevl
Differential Revision: https://reviews.llvm.org/D86546
As discussed in D118021 <https://reviews.llvm.org/D118021>, `clang -m32` on
Solaris/sparcv9 currently incorrectly doesn't inline atomics on 8-byte
operands, unlike `gcc`. With the workaround in that patch in place, we're
left with may undefined references to `__sync_val_compare_and_swap_8`,
which isn't provided by `libatomic`. This reference is due to the use of
`__sync_val_compare_and_swap` in `sanitizer_atomic_clang.h`'s
`atomic_compare_exchange_strong`. As is already done in
`scudo/standalone/atomic_helpers.h`, using `__atomic_compare_exchange`
instead avoids this problem.
Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.
Differential Revision: https://reviews.llvm.org/D118024
We previously had a few varied definitions of this floating around.
I had tried to make the one installed with LLVM handle all the cases, and then made the others use it, but this ran into issues with `HandleOutOfTreeLLVM` not working for compiler-rt, and also `CMAKE_EXE_LINKER_FLAGS` not working right without `CMP0056` set to the new behavior.
My compromise solution is this:
- No not completely deduplicate: the runtime libs will instead use a version that still exists as part of the internal and not installed common shared CMake utilities. This avoids `HandleOutOfTreeLLVM` or a workaround for compiler-rt.
- Continue to use `CMAKE_REQUIRED_FLAGS`, which effects compilation and linking. Maybe this is unnecessary, but it's safer to leave that as a future change. Also means we can avoid `CMP0056` for now, to try out later, which is good incrementality too.
- Call it `llvm_check_compiler_linker_flag` since it, in fact is about both per its implementation (before and after this patch), so there is no name collision.
In the future, we might still enable CMP0056 and make compiler-rt work with HandleOutOfTreeLLVM, which case we delete `llvm_check_compiler_flag` and go back to the old way (as these are, in fact, linking related flags), but that I leave for someone else as future work.
The original issue was reported to me in https://reviews.llvm.org/D116521#3248117 as
D116521 made clang and LLVM use the common cmake utils.
Reviewed By: sebastian-ne, phosek, #libunwind, #libc, #libc_abi, ldionne
Differential Revision: https://reviews.llvm.org/D117537
In glibc before 2.33, include/sys/stat.h defines fstat/fstat64 to
`__fxstat/__fxstat64` and provides `__fxstat/__fxstat64` in libc_nonshared.a.
The symbols are glibc specific and not needed on other systems.
Reviewed By: vitalybuka, #sanitizers
Differential Revision: https://reviews.llvm.org/D118423
Fixes a false positive that occurs when a user-implemented memmove is
instrumented by HWASan.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D118180
Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because
* We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
* We use a single byte per function rather than 8 bytes per block
The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.
When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).
[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116180
This is a follow up to D118101, that added bexpfull to the test on instrprof-get-filename-merge-mode.c AIX, in order to get the
necessary symbols exported. But unfortunately the extra dependent symbols this exports actually cause segfaults, which is why
this isn't really recommended in the first place, so just use an actual export list.
Differential Revision: https://reviews.llvm.org/D118310
Use the `llvm-profdata show` command to verify debug info for profile correlation using the `--debug-info` option.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D118181
The "asan/asan_mapping.h" header relies on sanitizer_platform.h
macros, but doesn't directly include the header. All the existing
uses until recently happened to be in places where some other header
had indirectly included sanitizer_platform.h first. The addition of
asan_rtl_x86_64.S was the first place to use "asan/asan_mapping.h"
alone. It so happens that its uses of the macros make having no
macros defined equivalent to SANITIZER_LINUX, so this did not affect
Linux builds. But the assembly constants in asan_rtl_x86_64.S were
wrong for Fuchsia when SANITIZER_FUCHSIA was not properly defined.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D118296
When using the in-tree libc++, we should be using the full path to
ensure that we're using the right library and not accidentally pick up
the system library.
Differential Revision: https://reviews.llvm.org/D118200
We have been relying on the logic for C++ ABI library for sanitizer
tests but that's incorrect since most tests require a full C++ library
and not just C++ ABI. This change tries to address this by using the
dependency on libc++ if available.
Differential Revision: https://reviews.llvm.org/D118163
There is no need to compress the names string when correlating with
debug info since InstrProfReader will immediately uncompress it anyway.
This also removes the dependency on zlib in this case.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D118176
The AIX linker doesn't export any symbols by default, so an export list is usually used. Since clang doesn't have the tools to auto-generate an export list yet, just pass the linker an extra opt to tell it to export everything. This is generally not recommended for real shared libs, but is fine for the purpose of this test.
Differential Revision: https://reviews.llvm.org/D118101
`_vfork` moved from libsystem_kernel.dylib to libsystem_c.dylib as part
of the below changes. The iOS simulator does not actually have
libsystem_kernel.dylib of its own, it only has the host Mac's. The
umbrella-nature of Libsystem makes this movement transparent to
everyone; except the simulator! So when we "back deploy", i.e., use the
current version of TSan with an older simulator runtime then this symbol
is now missing, when we run on the latest OS (but an older simulator
runtime).
Note we use `SANITIZER_IOS` because usage of vfork is forbidden on iOS
and the API is completely unavailable on watchOS and tvOS, even if this
problem is specific to the iOS simulator.
Caused by:
rdar://74818691 (Shim vfork() to fork syscall on iOS)
rdar://76762076 (Shim vfork() to fork syscall on macOS)
Radar-Id: rdar://8634734
This reverts commit 8c9f62ea90, which is causing build failures on
the bots because it inadvertently changes the output directory of the compiler-rt libs when
built as a runtime.
Differential Revision: https://reviews.llvm.org/D117815
We previously had a few varied definitions of this floating around. I made the one installed with LLVM handle all the cases, and then made the others use it.
This issue was reported to me in https://reviews.llvm.org/D116521#3248117 as
D116521 made clang and llvm use the common cmake utils.
Reviewed By: sebastian-ne, phosek, #libunwind, #libc, #libc_abi, ldionne
Differential Revision: https://reviews.llvm.org/D117537
This gives us the option of using CMake modules from LLVM, and other
things. We will use that to deduplicate code later.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D117815
https://reviews.llvm.org/D116179 introduced some changes to
`InstrProfData.inc` which broke some downstream builds. This commit
reverts those changes since they only changes two field names.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D117631
The kern.elf64.aslr.pie_enable and kern.elf32.aslr.pie_enable sysctls
control the default setting for PIE binary address randomization, but
it is possible to enable or disable ASLR on a per-process basis. So,
use procctl(2) to query whether ASLR is enabled.
(Note that with ASLR enabled but sysctl kern.elf64.aslr.pie_enable=0
a PIE binary will in effect have randomization disabled, and would be
functional with msan. This is not intended as as a user-facing control
though; proccontrol(1) should be used to disable aslr for the process.)
Reviewed By: devnexen
Differential Revision: https://reviews.llvm.org/D117521
1b1c8d changed `enable-noundef-analysis` flag to
`disable-noundef-analysis`. noundef_analysis.cpp was using old
`enable-noundef-analysis` flag and this patch fixes it.
This is the original patch in my GNUInstallDirs series, now last to merge as the final piece!
It arose as a new draft of D28234. I initially did the unorthodox thing of pushing to that when I wasn't the original author, but since I ended up
- Using `GNUInstallDirs`, rather than mimicking it, as the original author was hesitant to do but others requested.
- Converting all the packages, not just LLVM, effecting many more projects than LLVM itself.
I figured it was time to make a new revision.
I have used this patch series (and many back-ports) as the basis of https://github.com/NixOS/nixpkgs/pull/111487 for my distro (NixOS), which was merged last spring (2021). It looked like people were generally on board in D28234, but I make note of this here in case extra motivation is useful.
---
As pointed out in the original issue, a central tension is that LLVM already has some partial support for these sorts of things. Variables like `COMPILER_RT_INSTALL_PATH` have already been dealt with. Variables like `LLVM_LIBDIR_SUFFIX` however, will require further work, so that we may use `CMAKE_INSTALL_LIBDIR`.
These remaining items will be addressed in further patches. What is here is now rote and so we should get it out of the way before dealing more intricately with the remainder.
Reviewed By: #libunwind, #libc, #libc_abi, compnerd
Differential Revision: https://reviews.llvm.org/D99484
089acf2522 updated WrapperFunctionCall to carry arbitrary argument payloads
(rather than plain address ranges). This commit implements the corresponding
update for the ORC runtime.
Fixes:
[2587/4073] Building CXX object projects\compiler-rt\lib\sanitizer_common\CMakeFiles\RTSanitizerCommon.x86_64.dir\sanitizer_stoptheworld_win.cpp.obj
D:\git\llvm-project\compiler-rt\lib\sanitizer_common\sanitizer_stoptheworld_win.cpp(125,33): warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'int' [-Wsign-compare]
if (SuspendThread(thread) == -1) {
~~~~~~~~~~~~~~~~~~~~~ ^ ~~
1 warning generated.
This is the original patch in my GNUInstallDirs series, now last to merge as the final piece!
It arose as a new draft of D28234. I initially did the unorthodox thing of pushing to that when I wasn't the original author, but since I ended up
- Using `GNUInstallDirs`, rather than mimicking it, as the original author was hesitant to do but others requested.
- Converting all the packages, not just LLVM, effecting many more projects than LLVM itself.
I figured it was time to make a new revision.
I have used this patch series (and many back-ports) as the basis of https://github.com/NixOS/nixpkgs/pull/111487 for my distro (NixOS), which was merged last spring (2021). It looked like people were generally on board in D28234, but I make note of this here in case extra motivation is useful.
---
As pointed out in the original issue, a central tension is that LLVM already has some partial support for these sorts of things. Variables like `COMPILER_RT_INSTALL_PATH` have already been dealt with. Variables like `LLVM_LIBDIR_SUFFIX` however, will require further work, so that we may use `CMAKE_INSTALL_LIBDIR`.
These remaining items will be addressed in further patches. What is here is now rote and so we should get it out of the way before dealing more intricately with the remainder.
Reviewed By: #libunwind, #libc, #libc_abi, compnerd
Differential Revision: https://reviews.llvm.org/D99484
In C++20 compound assignment to volatile (here `LocalData[I]++`) is
deprecated, so `mutex_test.cpp` fails to compile.
Simply changing it to `LocalData[I] = LocalData[I] + 1` fixes it.
Differential Revision: https://reviews.llvm.org/D117359
Existing code tended to assume that counters had type `uint64_t` and
computed size from the number of counters. Fix this code to directly
compute the counters size in number of bytes where possible. When the
number of counters is needed, use `__llvm_profile_counter_entry_size()`
or `getCounterTypeSize()`. In a later diff these functions will depend
on the profile mode.
Change the meaning of `DataSize` and `CountersSize` to make them more clear.
* `DataSize` (`CountersSize`) - the size of the data (counter) section in bytes.
* `NumData` (`NumCounters`) - the number of data (counter) entries.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116179
If `pthread_create` is not available on a platform, we won't be able to check if interceptors work. Use `strcmp` instead.
Reviewed By: yln
Differential Revision: https://reviews.llvm.org/D116989
The test checks that an array of Obj-C literal integers (e.g. `@1`) gets a UBSan
warning when cast to an NSString, however the actual concrete Obj-C class of
literal integers doesn't always need to be __NSCFNumber. Let's relax the test
expectations to allow NSConstantIntegerNumber. Which exact subclass of NSNumber
is used is not actually important for the test (the test is just checking that
the invalid cast warning is thrown).
These tests are now green:
```
Trace.MultiPart
Trace.RestoreAccess
Trace.RestoreMutexLock
TraceAlloc.FinishedThreadReuse
TraceAlloc.FinishedThreadReuse2
TraceAlloc.SingleThread
```
rdar://82107856
D114294 <https://reviews.llvm.org/D114294> broke the Solaris buildbots:
/opt/llvm-buildbot/home/solaris11-amd64/clang-solaris11-amd64/llvm/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:613:29: error: use of undeclared identifier 'NT_GNU_BUILD_ID'
if (nhdr->n_type == NT_GNU_BUILD_ID && nhdr->n_namesz == kGnuNamesz) {
^
Like D107556 <https://reviews.llvm.org/D107556>, it forgot that
`NT_GNU_BUILD_ID` is an unportable GNU extension.
Fixed by making the code conditional on the definition of the macro.
Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D117051
indirect branch tracking(IBT) feature aiming to ensure the target address
of an indirect jump/call is not tampered.
When IBT is enabled, each function or target of any indirect jump/call will start
with an 'endbr32/64' instruction otherwise the program will crash during execution.
To build an application with CET enabled. we need to ensure:
1. build the source code with "-fcf-protection=full"
2. all the libraries linked with .o files must be CET enabled too
This patch aims to enable CET for compiler-rt builtins library, we add an option
"COMPILER_RT_ENABLE_CET" whose default value is OFF to enable CET for compiler-rt
in building time and when this option is "ON", "-fcf-protection=full" is added to
BUILTINS_CFLAG and the "endbr32/64" will be placed in the beginning of each assembly
function. We also enabled CET for crtbegin, crtend object files in this patch.
Reviewed by: MaskRay, compnerd, manojgupta, efriedma
Differential Revision: https://reviews.llvm.org/D109811
Signed-off-by: jinge90 <ge.jin@intel.com>
Inspired by LLVM_DEBUG, but using environment variables rather than command line
options.
Code can use ORC_RT_DEBUG(...) (if ORC_RT_DEBUG_TYPE is set), or
ORC_RT_DEBUG_WITH_TYPE(<type>, ...) (if ORC_RT_DEBUG_TYPE is not set. E.g. in
headers).
Debug logging is enabled in the executor by setting the ORC_RT_DEBUG environment
variable. Debug logging can be restricted by type by setting the
ORC_RT_DEBUG_TYPES environment variable to a comma separated list of types,
e.g. ORC_RT_DEBUG_TYPES=macho_platform,sps.
Differential Revision: https://reviews.llvm.org/D116139
When building libcxx, libcxxabi, and libunwind the build environment may
specify any number of sanitizers. For some build feature tests these
sanitizers must be disabled to prevent spurious linking errors. With
-fsanitize= this is straight forward with -fno-sanitize=all. With
-fsanitize-coverage= there is no -fno-sanitize-coverage=all, but there
is the equivalent undocumented but tested -fsanitize-coverage=0.
The current build rules fail to disable 'trace-pc-guard'. By disabling
all sanitize-coverage flags, including 'trace-pc-guard', possible
spurious linker errors are prevented. In particular, this allows libcxx,
libcxxabi, and libunwind to be built with HonggFuzz.
CMAKE_REQUIRED_FLAGS is extra compile flags when running CMake build
configuration steps (like check_cxx_compiler_flag). It does not affect
the compile flags for the actual build of the project (unless of course
these flags change whether or not a given source compiles and links or
not). So libcxx, libcxxabi, and libunwind will still be built with any
specified sanitize-coverage as before. The build configuration steps
(which are mostly checking to see if certain compiler flags are
available) will not try to compile and link "int main() { return 0;}"
(or other specified source) with sanitize-coverage (which can fail to
link at this stage in building, since the final compile flags required
are yet to be determined).
The change to LIBFUZZER_CFLAGS was done to keep it consistent with the
obvious intention of disabling all sanitize-coverage. This appears to
be intentional, preventing the fuzzer driver itself from showing up in
any coverage calculations.
Reviewed By: #libunwind, #libc, #libc_abi, ldionne, phosek
Differential Revision: https://reviews.llvm.org/D116050
Currently we use very common names for macros like ACQUIRE/RELEASE,
which cause conflicts with system headers.
Prefix all macros with SANITIZER_ to avoid conflicts.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116652
This allows DFSan to find tainted values used to control program behavior.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D116207
This allows their reuse across projects. The name of the module
is intentionally generic because we would like to move more platform
checks there.
Differential Revision: https://reviews.llvm.org/D115276
This reverts commit 640beb38e7.
That commit caused performance degradtion in Quicksilver test QS:sGPU and a functional test failure in (rocPRIM rocprim.device_segmented_radix_sort).
Reverting until we have a better solution to s_cselect_b64 codegen cleanup
Change-Id: Ibf8e397df94001f248fba609f072088a46abae08
Reviewed By: kzhuravl
Differential Revision: https://reviews.llvm.org/D115960
Change-Id: Id169459ce4dfffa857d5645a0af50b0063ce1105
This will allow linking in the callbacks directly instead of using PLT.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116182
A signal handler can alter ucontext_t to affect execution after
the signal returns. Check that the contents are initialized.
Restoring unitialized values in registers can't be good.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116209
ucontext_t can be larger than its static size if it contains
AVX state and YMM/ZMM registers.
Currently a signal handler that tries to access that state
can produce false positives with random origins on stack.
Account for the additional ucontext_t state.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116208
This is a segmentation fault in INTERCEPTOR function on a special edge
case of strstr libc call. When 'Haystack'(main string to be examined) is
NULL and 'needle'(sub-string to be searched in 'Haystack') is an empty
string then it hits a SEGV while using sanitizers and as a 'string not
found' case otherwise.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115919
In D116472 we created conditionally defined variables for the tools to
unbreak the legacy build where they are in `llvm/tools`.
The runtimes are not tools, so that flexibility doesn't matter. Still,
it might be nice to define (unconditionally) and use the variable for
the runtimes simply to make the code a bit clearer and document what is
going on.
Also, consistently put project dirs at the beginning, not end of `CMAKE_MODULE_PATH`. This ensures they will properly shadow similarly named stuff that happens to be later on the path.
Reviewed By: mstorsjo, #libunwind, #libc, #libc_abi, ldionne
Differential Revision: https://reviews.llvm.org/D116477
It is likely to become used again, if other projects want their own per-project
install directory variables. `install` is removed from the name since it is not inherently about installing.
Reviewed By: stephenneuendorffer
Differential Revision: https://reviews.llvm.org/D115746
When using debug info for profile correlation, avoid adding duplicate
functions in the synthetic Data section.
Before this patch, n duplicate function entries in the Data section would
cause counter values to be a factor of n larger. I built instrumented
clang with and without debug info correlation and got these summaries.
```
# With Debug Info Correlate
$ llvm-profdata show default.profdata
Instrumentation level: IR entry_first = 0
Total functions: 182530
Maximum function count: 52034
Maximum internal block count: 5763
# Without
$ llvm-profdata show default.profdata
Instrumentation level: IR entry_first = 0
Total functions: 183212
Maximum function count: 52034
Maximum internal block count: 5766
```
The slight difference in counts seem to be mostly from FileSystem and
Map functions and the difference in the number of instrumented functions
seems to come from missing debug info like destructors without source.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116051
This patch adds support to read all the PT_NOTE segments in the
executable to find the binary ids. Previously, it was only reading
the first PT_NOTE segment, and this was missing the cases where
binary id is in the following segments. As a result, binary-id.c
and binary-id-padding.c test were failing in the following cases:
1) sanitizer-x86_64-linux bot
https://lab.llvm.org/staging/#/builders/97
2) OpenSuse Tumbleweed
https://github.com/llvm/llvm-project/issues/52695
Differential Revision: https://reviews.llvm.org/D115830
This will allow linking in the callbacks directly instead of using PLT.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116182
This option is per process anyway. I'd like to add more options, but
having them as parameters of __sanitizer_symbolize_code looks
inconvenient.
Reviewed By: browneee
Differential Revision: https://reviews.llvm.org/D116201
This will allow linking in the callbacks directly instead of using PLT.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D116182
After D116148 the memccpy gets optimized away and the expected
uninitialized memory access does not occur.
Make sure the call does not get optimized away.
The new tsan runtime has 2x more compact shadow.
Adjust shadow ranges accordingly.
Depends on D112603.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D113751
If there are multiple processes, it's hard to understand
what output comes from what process.
VReport prepends pid to the output. Use it.
Depends on D113982.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113983
Update now after long operations so that we don't use
stale value in subsequent computations.
Depends on D113981.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113982
Creating threads after a multi-threaded fork is semi-supported,
we don't give particular guarantees, but we try to not fail
on simple cases and we have die_after_fork=0 flag that enables
not dying on creation of threads after a multi-threaded fork.
This flag is used in the wild:
23c052e3e3/SConstruct (L3599)
fork_multithreaded.cpp test started hanging in debug mode
after the recent "tsan: fix deadlock during race reporting" commit,
which added proactive ThreadRegistryLock check in SlotLock.
But the test broke earlier after "tsan: remove quadratic behavior in pthread_join"
commit which made tracking of alive threads based on pthread_t stricter
(CHECK-fail on 2 threads with the same pthread_t, or joining a non-existent thread).
When we start a thread after a multi-threaded fork, the new pthread_t
can actually match one of existing values (for threads that don't exist anymore).
Thread creation started CHECK-failing on this, but the test simply
ignored this CHECK failure in the child thread and "passed".
But after "tsan: fix deadlock during race reporting" the test started hanging dead,
because CHECK failures recursively lock thread registry.
Fix this purging all alive threads from thread registry on fork.
Also the thread registry mutex somehow lost the internal deadlock detector id
and was excluded from deadlock detection. If it would have the id, the CHECK
wouldn't hang because of the nested CHECK failure due to the deadlock.
But then again the test would have silently ignore this error as well
and the bugs wouldn't have been noticed.
Add the deadlock detector id to the thread registry mutex.
Also extend the test to check more cases and detect more bugs.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116091
If we miss both close of a file descriptor and a subsequent open
if the same file descriptor number, we report false positives
between operations on the old and on the new descriptors.
There are lots of ways to create new file descriptors, but for closing
there is mostly close call. So we try to handle at least it.
However, if the close happens in an ignored library, we miss it
and start reporting false positives.
Handle closing of file descriptors always, even in ignored libraries
(as we do for malloc/free and other critical functions).
But don't imitate memory accesses on close for ignored libraries.
FdClose checks validity of the fd (fd >= 0) itself,
so remove the excessive checks in the callers.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116095
We used to use u64 as mutex id because it was some
tricky identifier built from address and reuse count.
Now it's just the mutex index in the report (0, 1, 2...),
so use int to represent it.
Depends on D112603.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D113980
These callbacks are used for SSE vector accesses.
In some computational programs these accesses dominate.
Currently we do 2 uninlined 8-byte accesses to handle them.
Inline and optimize them similarly to unaligned accesses.
This reduces the vector access benchmark time from 8 to 3 seconds.
Depends on D112603.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114594
This also makes the sanitizer_stoptheworld_test cross-platform by using the STL, rather than pthread.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115204
This attempts to adjust the test to still exercise the expected codepath after D115904. This test is fundementally rather fragile.
Unfortunately, I have not been able to confirm this workaround either does, or does not, work. Attempting check-all with compiler-rt blows through an additional 30GB of disk space so my build config which exceeds my local disk space.
There is a small chance that the slot may be not queued in TraceSwitchPart.
This can happen if the slot has kEpochLast epoch and another thread
in FindSlotAndLock discovered that it's exhausted and removed it from
the slot queue. kEpochLast can happen in 2 cases: (1) if TraceSwitchPart
was called with the slot locked and epoch already at kEpochLast,
or (2) if we've acquired a new slot in SlotLock in the beginning
of the function and the slot was at kEpochLast - 1, so after increment
in SlotAttachAndLock it become kEpochLast.
If this happens we crash on ctx->slot_queue.Remove(thr->slot).
Skip the requeueing if the slot is not queued.
The slot is exhausted, so it must not be ctx->slot_queue.
The existing stress test triggers this with very small probability.
I am not sure how to make this condition more likely to be triggered,
it evaded lots of testing.
Depends on D116040.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116041
SlotPairLocker calls SlotLock under ctx->multi_slot_mtx.
SlotLock can invoke global reset DoReset if we are out of slots/epochs.
But DoReset locks ctx->multi_slot_mtx as well, which leads to deadlock.
Resolve the deadlock by removing SlotPairLocker/multi_slot_mtx
and only lock one slot for which we will do RestoreStack.
We need to lock that slot because RestoreStack accesses the slot journal.
But it's unclear why we need to lock the current slot.
Initially I did it just to be on the safer side (but at that time
we dit not lock the second slot, so it was easy just to lock the current slot).
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D116040
Profile merging is not supported when using debug info profile
correlation because the data section won't be in the binary at runtime.
Change the default profile name in this mode to `default_%p.proflite` so
we don't use profile merging.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115979
Previously we would crash in the TSan runtime if the user program passes
a pointer to `malloc_size()` that doesn't point into app memory.
In these cases, `malloc_size()` should return 0.
For ASan, we fixed a similar issue here:
https://reviews.llvm.org/D15008
Radar-Id: rdar://problem/86213149
Differential Revision: https://reviews.llvm.org/D115947
There was a build failure on the `instrprof-debug-info-correlate.c` test
because zlib was missing so we need to require it to run the test.
Differential Revision: https://reviews.llvm.org/D115970
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).
This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
This was first landed in https://reviews.llvm.org/D114566 but had to be reverted due to build errors.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115915
This reverts commit 3f5f687e2e.
That commit broke building for mingw, where the sanitizers are
built with -nostdinc++, while the added source file includes
the C++ standard library's <algorithm>.
Additionally, the new code fails to build for i386, as it
unconditionally uses the CONTEXT member Rsp.
AARCH64_GET_REG() is used to initialize uptrs, and after D79132
the ptrauth branch of its implementation explicitly casts to uptr.
The non-ptrauth branch returns ucontext->uc_mcontext->__ss.__fp (etc),
which has either type void* or __uint64_t (ref usr/include/mach/arm/_structs.h)
where __uint64_t is a unsigned long long (ref usr/include/arm/_types.h).
uptr is an unsigned long (ref
compiler-rt/lib/sanitizer_common/sanitizer_internal_defs.h). So explicitly
cast to uptr in this branch as well, so that AARCH64_GET_REG() has a
well-defined type.
Then change DUMPREGA64() tu use %lx instead of %llx since that's the right type
for uptr. (Most other places in compiler-rt print uptrs as %p and cast the arg
to (void*), but there are explicit 0x%016 format strings in the surroundings,
so be locally consistent with that.)
No behavior change, in the end it's just 64-bit unsigneds by slightly different
names.
This also makes the sanitizer_stoptheworld_test cross-platform by using the STL, rather than pthread.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115204
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).
This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D114566
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.
Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
The original diff https://reviews.llvm.org/D114565 was reverted because of the `Instrumentation/InstrProfiling/debug-info-correlate.ll` test, which is fixed in this commit.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115693
There were changes made to the linux version of this test that were not made for darwin
(see https://reviews.llvm.org/D115837) and this caused downstream failures.
Adding comment to this test to remind people to edit interface_symbols_darwin.cpp.
There is the reverse of this comment in the darwin file to remind us to edit the linux version already.
Differential Revision: https://reviews.llvm.org/D115899
LSan (`ASAN_OPTIONS=detect_leaks=1`) is supported on macOS, but disabled
by default on Darwin (`SANITIZER_MAC`):
```
COMMON_FLAG(bool, detect_leaks, !SANITIZER_MAC, "Enable memory leak detection.")
```
We enable it here for ASan tests to prevent regressions (per comment).
However, LSan is not supported for the iOS simulator and the tests fail
when it is enabled.
Make this "Is macOS?" check more precise since the current one (`Darwin
&& x86_64`) has two issues:
* Includes the simulators
* Excludes macOS on Apple Silicon
This will allow us to (re)enable simulator testing on Green dragon to
give open source better feedback about sanitizer changes:
https://green.lab.llvm.org
rdar://86529234
Differential Revision: https://reviews.llvm.org/D115816
This test would hang when the system ran out of resources and we fail to
create all 300 threads.
Differential Revision: https://reviews.llvm.org/D115845
In the new TSan runtime refactoring this line was changed:
```
ProtectRange(MetaShadowEnd(), TraceMemBeg());
-->
ProtectRange(MetaShadowEnd(), HeapMemBeg());
```
But for `MappingAppleAarch64` the app heap comes before the shadow,
resulting in:
```
CHECK failed: tsan_platform_posix.cpp:83 "((beg)) <= ((end))" (0xe00000000, 0x200000000)
```
rdar://86521924
Differential Revision: https://reviews.llvm.org/D115834
A while ago we added some code to the sanitizer runtimes for iOS
simulators to allow `atos` (external process) to inspect the sanitized
process during report generation to enable symbolication. This was done
by setting the `__check_mach_ports_lookup` env var early during process
startup which came with a couple of complications.
This workaround is not required anymore and removing it fixes TSan in
the iOS simulator after the new TSan runtime landed.
(https://reviews.llvm.org/D112603)
Relevant/reverted revisions:
https://reviews.llvm.org/D78178https://reviews.llvm.org/D78179https://reviews.llvm.org/D78525
rdar://86472733
Differential Revision: https://reviews.llvm.org/D115767
Adds -L<search-path> and -l<library> options that are analogous to ld's
versions.
Each instance of -L<search-path> or -l<library> will apply to the most recent
-jd option on the command line (-jd <name> creates a JITDylib with the given
name). Library names will match against JITDylibs first, then llvm-jitlink will
look through the search paths for files named <search-path>/lib<library>.dylib
or <search-path>/lib<library>.a.
The default "main" JITDylib will link against all JITDylibs created by -jd
options, and all JITDylibs will link against the process symbols (unless
-no-process-symbols is specified).
The -dlopen option is renamed -preload, and will load dylibs into the JITDylib
for the ORC runtime only.
The effect of these changes is to make it easier to describe a non-trivial
program layout to llvm-jitlink for testing purposes. E.g. the following
invocation describes a program consisting of three JITDylibs: "main" (created
implicitly) containing main.o, "Foo" containing foo1.o and foo2.o, and linking
against library "bar" (not a JITDylib, so it must be a .dylib or .a on disk)
and "Baz" (which is a JITDylib), and "Baz" containing baz.o.
llvm-jitlink \
main.o \
-jd Foo foo1.o foo2.o -L${HOME}/lib -lbar -lBaz
-jd Baz baz.o
This is present in our assembly files. It should fix decorate_proc_maps.cpp failures because of shadow memory being allocated as executable.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115552
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
This reverts commit 800bf8ed29.
The `Instrumentation/InstrProfiling/debug-info-correlate.ll` test was
failing because I forgot the `llc` commands are architecture specific.
I'll follow up with a fix.
Differential Revision: https://reviews.llvm.org/D115689
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.
Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D114565
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Updates compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh to include Debuginfod library to fix sanitizer-x86_64-linux breakage.
Reviewed By: jhenderson, vitalybuka
Differential Revision: https://reviews.llvm.org/D113717
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Currently the test calls dlclose in the thread
concurrently with the main thread calling a function
from the dynamic library. This is not good.
Wait for the main thread to call the function
before calling dlclose.
Depends on D115612.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D115613
The test contains a race and checks that it's detected.
But the race may not be detected since we are doing aggressive flushes
and if the state flush happens between racing accesses, tsan won't
detect the race). So return 1 to make the test deterministic
regardless of the race.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D115612
This is present in our assembly files. It should fix decorate_proc_maps.cpp failures because of shadow memory being allocated as executable.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115552
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
Adds x-ray support for hexagon to llvm codegen, clang driver,
compiler-rt libs.
Differential Revision: https://reviews.llvm.org/D113638
Reapplying this after 543a9ad7c4,
which fixes the leak introduced there.
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Updates compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh to include Debuginfod library to fix sanitizer-x86_64-linux breakage.
Reviewed By: jhenderson, vitalybuka
Differential Revision: https://reviews.llvm.org/D113717
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Adds new symbolizer symbols to `global_symbols.txt`.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D113717
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
The test has been flaky for years, and I think we should remove it to
eliminate noise on the buildbot.
Neither me nor dokyungs have been able to fully deflake the test, and it
tests a non-default Entropic flag.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115453
This removes the last use of StackDepot from StopTheWorld.
Depends on D115284.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D115319
StackDepot locks some stuff. As is there is small probability to
deadlock if we stop thread which locked the Depot.
We need either Lock/Unlock StackDepot for StopTheWorld, or don't
interact with StackDepot from there.
This patch does not run LeakReport under StopTheWorld. LeakReport
contains most of StackDepot access.
As a bonus this patch will help to resolve kMaxLeaksConsidered FIXME.
Depends on D114498.
Reviewed By: morehouse, kstoimenov
Differential Revision: https://reviews.llvm.org/D115284
This reverts commit e5c2a46c5e as this
change introduced a linker error when building sanitizer runtimes:
ld.lld: error: undefined symbol: __sanitizer::internal_start_thread(void* (*)(void*), void*)
>>> referenced by sanitizer_stackdepot.cpp:133 (compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp:133)
>>> compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stackdepot.cpp.obj:(__sanitizer::(anonymous namespace)::CompressThread::NewWorkNotify())
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Fork the current version of tsan runtime before commiting
rewrite of the runtime (D112603). The old runtime can be
enabled with TSAN_USE_OLD_RUNTIME option.
This is a temporal measure for emergencies and is required
for Chromium rollout (for context see http://crbug.com/1275581).
The old runtime is supposed to be deleted soon.
Reviewed By: thakis
Differential Revision: https://reviews.llvm.org/D115223
This reverts commit 5c27740238.
Reverting due to Windows build issue:
sanitizer_stackdepot.cpp.obj : error LNK2005: "void __cdecl __sanitizer::StackDepotStopBackgroundThread(void)" (?StackDepotStopBackgroundThread@__sanitizer@@YAXXZ) already defined in sanitizer_common_libcdep.cpp.obj
LINK : fatal error LNK1181: cannot open input file 'projects\compiler-rt\lib\asan\CMakeFiles\RTAsan_dynamic.x86_64.dir\asan_rtl_x86_64.S.obj'
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114558
Some of the compiler-rt runtimes use custom instrumented libc++ build.
Use the runtimes build for building this custom libc++.
Differential Revision: https://reviews.llvm.org/D114922
We don't run tests or benchmarks from this build anyway.
Benchmarks in custom libc++ break my local build.
Reviewed By: ldionne
Differential Revision: https://reviews.llvm.org/D115375
kIgnored didn't exist when the code was added, but it should be
equivalent to kReachable.
The goal is to refactor MarkInvalidPCCb to avoid StackDepotGet
in StopTheWorld.
Some of the compiler-rt runtimes use custom instrumented libc++ build.
Use the runtimes build for building this custom libc++.
Differential Revision: https://reviews.llvm.org/D114922
It is required for the [Leak Sanitizer port to Windows](https://reviews.llvm.org/D115103).
The currently used `unsigned long` type is 64 bits wide on UNIX like systems but only 32 bits wide on Windows.
Because of that, the literal `8UL << 30` causes an integer overflow on Windows.
By changing the type of the literals to `unsigned long long`, we have consistent behavior and no overflows on all Platforms.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D115186
That way, the build rules are closer to the source files they describe.
No intended behavior change.
Differential Revision: https://reviews.llvm.org/D115155
Building xray with recent clang on a 64-bit system results in a number
of -Wformat warnings:
compiler-rt/lib/xray/xray_allocator.h:70:11: warning: format specifies type 'int' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
RoundedSize, B);
^~~~~~~~~~~
compiler-rt/lib/xray/xray_allocator.h:119:11: warning: format specifies type 'int' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
RoundedSize, B);
^~~~~~~~~~~
Since `__sanitizer::uptr` has the same size as `size_t`, these can be
fixed by using the printf specifier `%zu`.
compiler-rt/lib/xray/xray_basic_logging.cpp:348:46: warning: format specifies type 'int' but the argument has type '__sanitizer::tid_t' (aka 'unsigned long long') [-Wformat]
Report("Cleaned up log for TID: %d\n", GetTid());
~~ ^~~~~~~~
%llu
compiler-rt/lib/xray/xray_basic_logging.cpp:353:62: warning: format specifies type 'int' but the argument has type '__sanitizer::tid_t' (aka 'unsigned long long') [-Wformat]
Report("Skipping buffer for TID: %d; Offset = %llu\n", GetTid(),
~~ ^~~~~~~~
%llu
Since `__sanitizer::tid_t` is effectively declared as `unsigned long
long`, these can be fixed by using the printf specifier `%llu`.
compiler-rt/lib/xray/xray_basic_logging.cpp:354:14: warning: format specifies type 'unsigned long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
TLD.BufferOffset);
^~~~~~~~~~~~~~~~
Since `BufferOffset` is declared as `size_t`, this one can be fixed by
using `%zu` as a printf specifier.
compiler-rt/lib/xray/xray_interface.cpp:172:50: warning: format specifies type 'int' but the argument has type 'uint64_t' (aka 'unsigned long') [-Wformat]
Report("Unsupported sled kind '%d' @%04x\n", Sled.Address, int(Sled.Kind));
~~ ^~~~~~~~~~~~
%lu
Since ``xray::SledEntry::Address` is declared as `uint64_t`, this one
can be fixed by using `PRIu64`, and adding `<cinttypes>`.
compiler-rt/lib/xray/xray_interface.cpp:308:62: warning: format specifies type 'long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
Report("System page size is not a power of two: %lld\n", PageSize);
~~~~ ^~~~~~~~
%zu
compiler-rt/lib/xray/xray_interface.cpp:359:64: warning: format specifies type 'long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
Report("Provided page size is not a power of two: %lld\n", PageSize);
~~~~ ^~~~~~~~
%zu
Since `PageSize` is declared as `size_t`, these can be fixed by using
`%zu` as a printf specifier.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114469
It's very simple, fast and efficient for the stack depot compression if used on entire pointers.
Reviewed By: morehouse, kstoimenov
Differential Revision: https://reviews.llvm.org/D114918
Using `_mkdir` of CRT in Asan Init leads to launch failure and hanging in Windows.
You can trigger it by calling:
> set ASAN_OPTIONS=log_path=a/a/a
> .\asan_program.exe
And their crash dump shows the following stack trace:
```
_guard_dispatch_icall_nop()
__acrt_get_utf8_acp_compatibility_codepage()
_mkdir(const char * path)
```
I guess there could be a cfg guard in CRT, which may lead to calling uninitialized cfg guard function address. Also, `_mkdir` supports UTF-8 encoding of the path and calls _wmkdir, but that's not necessary for this case since other file apis in sanitizer_win.cpp assumes only ANSI code case, so it makes sense to use CreateDirectoryA matching other file api calls in the same file.
Reviewed By: tejohnson
Differential Revision: https://reviews.llvm.org/D114760
Google-signed apexes appear on Android build servers' symbol files as
being under /apex/com.google.android.<foo>/. In reality, the apexes are
always installed as /apex/com.android.<foo>/ (note the lack of
'google'). In order for local symbolization under hwasan_symbolize to
work correctly, we also try the 'google' directory.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D114919
According comments on D44404, something like that was the goal.
Reviewed By: morehouse, kstoimenov
Differential Revision: https://reviews.llvm.org/D114991
The goal is to identify the bot and try to fix it.
SetSoftRssLimitExceededCallback is AsanInitInternal as I assume
that only MaybeStartBackgroudThread needs to be delayed to constructors.
Later I want to move MaybeStartBackgroudThread call into sanitizer_common.
If it needs to be reverted please provide to more info, like bot, or details about setup.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D114934
Compress by factor 4x, takes about 10ms per 8 MiB block.
Depends on D114498.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114503
It should be NFC, as they already intercept pthread_create.
This will let us to fix BackgroundThread for these sanitizerts.
In in followup patches I will fix MaybeStartBackgroudThread for them
and corresponding tests.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D114935
We call UnmapShadow before the actual munmap, at that point we don't yet
know if the provided address/size are sane. We can't call UnmapShadow
after the actual munmap becuase at that point the memory range can
already be reused for something else, so we can't rely on the munmap
return value to understand is the values are sane.
While calling munmap with insane values (non-canonical address, negative
size, etc) is an error, the kernel won't crash. We must also try to not
crash as the failure mode is very confusing (paging fault inside of the
runtime on some derived shadow address).
Such invalid arguments are observed on Chromium tests:
https://bugs.chromium.org/p/chromium/issues/detail?id=1275581
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114944
The added test demonstrates loading a dynamic library with static TLS.
Such static TLS is a hack that allows a dynamic library to have faster TLS,
but it can be loaded only iff all threads happened to allocate some excess
of static TLS space for whatever reason. If it's not the case loading fails with:
dlopen: cannot load any more object with static TLS
We used to produce a false positive because dlopen will write into TLS
of all existing threads to initialize/zero TLS region for the loaded library.
And this appears to be racing with initialization of TLS in the thread
since we model a write into the whole static TLS region (we don't what part
of it is currently unused):
WARNING: ThreadSanitizer: data race (pid=2317365)
Write of size 1 at 0x7f1fa9bfcdd7 by main thread:
0 memset
1 init_one_static_tls
2 __pthread_init_static_tls
[[ this is where main calls dlopen ]]
3 main
Previous write of size 8 at 0x7f1fa9bfcdd0 by thread T1:
0 __tsan_tls_initialization
Fix this by ignoring accesses during dlopen.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114953
Broke the build on Windows, where MprotectReadOnly() isn't defined, see comment
on the code review.
> Compress by factor 4x, takes about 10ms per 8 MiB block.
>
> Depends on D114498.
>
> Reviewed By: morehouse
>
> Differential Revision: https://reviews.llvm.org/D114503
This reverts commit 1d8f295759.
Compress by factor 4x, takes about 10ms per 8 MiB block.
Depends on D114498.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114503
The first 8b of each raw profile section need to be aligned to 8b since
the first item in each section is a u64 count of the number of items in
the section.
Summary of changes:
* Assert alignment when reading counts.
* Update test to check alignment, relax some size checks to allow padding.
* Update raw binary inputs for llvm-profdata tests.
Differential Revision: https://reviews.llvm.org/D114826
Add Compression::Test type which just pretends packing,
but does nothing useful. It's only called from test for now.
Depends on D114493.
Reviewed By: kstoimenov
Differential Revision: https://reviews.llvm.org/D114494
We would like to use TLS to store the ThreadState object (or at least a
reference ot it), but on Darwin accessing TLS via __thread or manually
by using pthread_key_* is problematic, because there are several places
where interceptors are called when TLS is not accessible (early process
startup, thread cleanup, ...).
Previously, we used a "poor man's TLS" implementation, where we use the
shadow memory of the pointer returned by pthread_self() to store a
pointer to the ThreadState object.
The problem with that was that certain operations can populate shadow
bytes unbeknownst to TSan, and we later interpret these non-zero bytes
as the pointer to our ThreadState object and crash on when dereferencing
the pointer.
This patch changes the storage location of our reference to the
ThreadState object to "real" TLS. We make this work by artificially
keeping this reference alive in the pthread_key destructor by resetting
the key value with pthread_setspecific().
This change also fixes the issue were the ThreadState object is
re-allocated after DestroyThreadState() because intercepted functions
can still get called on the terminating thread after the
THREAD_TERMINATE event.
Radar-Id: rdar://problem/72010355
Reviewed By: dvyukov
Differential Revision: https://reviews.llvm.org/D110236
The memprof unit tests use an older version of gmock (included in the
repo) which does not build cleanly with -pedantic:
https://github.com/google/googletest/issues/2650
For now just silence the warning by disabling pedantic and add the
appropriate flags for gcc and clang.
This commit adds initial support to llvm-profdata to read and print
summaries of raw memprof profiles.
Summary of changes:
* Refactor shared defs to MemProfData.inc
* Extend show_main to display memprof profile summaries.
* Add a simple raw memprof profile reader.
* Add a couple of tests to tools/llvm-profdata.
Differential Revision: https://reviews.llvm.org/D114286
In multi-threaded application concurrent StackStore::Store may
finish in order different from assigned Id. So we can't assume
that after we switch writing the next block the previous is done.
The workaround is to count exact number of uptr stored into the block,
including skipped tail/head which were not able to fit entire trace.
Depends on D114490.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114493
Larger blocks are more convenient for compressions.
Blocks are allocated with MmapNoReserveOrDie to save some memory.
Also it's 15% faster on StackDepotBenchmarkSuite
Depends on D114464.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D114488
Sometimes stacks for at_exit callbacks don't include any of the user functions/files.
For example, a race with a global std container destructor will only contain
the container type name and our at_exit_wrapper function. No signs what global variable
this is.
Remember and include in reports the function that installed the at_exit callback.
This should give glues as to what variable is being destroyed.
Depends on D114606.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D114607
Add a test for a common C++ bug when a global object is destroyed
while background threads still use it.
Depends on D114604.
Reviewed By: vitalybuka, melver
Differential Revision: https://reviews.llvm.org/D114605
[NFC] As part of using inclusive language within the llvm project, this patch
replaces master and slave with primary and secondary respectively in
`sanitizer_mac.cpp`.
Reviewed By: ZarkoCA
Differential Revision: https://reviews.llvm.org/D114255
This change switches tsan to the new runtime which features:
- 2x smaller shadow memory (2x of app memory)
- faster fully vectorized race detection
- small fixed-size vector clocks (512b)
- fast vectorized vector clock operations
- unlimited number of alive threads/goroutimes
Depends on D112602.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D112603
Linux/fork_deadlock.cpp currently hangs in debug mode in the following stack.
Disable memory access handling in OnUserAlloc/Free around fork.
1 0x000000000042c54b in __sanitizer::internal_sched_yield () at sanitizer_linux.cpp:452
2 0x000000000042da15 in __sanitizer::StaticSpinMutex::LockSlow (this=0x57ef02 <__sanitizer::internal_allocator_cache_mu>) at sanitizer_mutex.cpp:24
3 0x0000000000423927 in __sanitizer::StaticSpinMutex::Lock (this=0x57ef02 <__sanitizer::internal_allocator_cache_mu>) at sanitizer_mutex.h:32
4 0x000000000042354c in __sanitizer::GenericScopedLock<__sanitizer::StaticSpinMutex>::GenericScopedLock (this=this@entry=0x7ffcabfca0b8, mu=0x1) at sanitizer_mutex.h:367
5 0x0000000000423653 in __sanitizer::RawInternalAlloc (size=size@entry=72, cache=cache@entry=0x0, alignment=8, alignment@entry=0) at sanitizer_allocator.cpp:52
6 0x00000000004235e9 in __sanitizer::InternalAlloc (size=size@entry=72, cache=0x1, cache@entry=0x0, alignment=4, alignment@entry=0) at sanitizer_allocator.cpp:86
7 0x000000000043aa15 in __sanitizer::SymbolizedStack::New (addr=4802655) at sanitizer_symbolizer.cpp:45
8 0x000000000043b353 in __sanitizer::Symbolizer::SymbolizePC (this=0x7f578b77a028, addr=4802655) at sanitizer_symbolizer_libcdep.cpp:90
9 0x0000000000439dbe in __sanitizer::(anonymous namespace)::StackTraceTextPrinter::ProcessAddressFrames (this=this@entry=0x7ffcabfca208, pc=4802655) at sanitizer_stacktrace_libcdep.cpp:36
10 0x0000000000439c89 in __sanitizer::StackTrace::PrintTo (this=this@entry=0x7ffcabfca2a0, output=output@entry=0x7ffcabfca260) at sanitizer_stacktrace_libcdep.cpp:109
11 0x0000000000439fe0 in __sanitizer::StackTrace::Print (this=0x18) at sanitizer_stacktrace_libcdep.cpp:132
12 0x0000000000495359 in __sanitizer::PrintMutexPC (pc=4802656) at tsan_rtl.cpp:774
13 0x000000000042e0e4 in __sanitizer::InternalDeadlockDetector::Lock (this=0x7f578b1ca740, type=type@entry=2, pc=pc@entry=4371612) at sanitizer_mutex.cpp:177
14 0x000000000042df65 in __sanitizer::CheckedMutex::LockImpl (this=<optimized out>, pc=4) at sanitizer_mutex.cpp:218
15 0x000000000042bc95 in __sanitizer::CheckedMutex::Lock (this=0x600001000000) at sanitizer_mutex.h:127
16 __sanitizer::Mutex::Lock (this=0x600001000000) at sanitizer_mutex.h:165
17 0x000000000042b49c in __sanitizer::GenericScopedLock<__sanitizer::Mutex>::GenericScopedLock (this=this@entry=0x7ffcabfca370, mu=0x1) at sanitizer_mutex.h:367
18 0x000000000049504f in __tsan::TraceSwitch (thr=0x7f578b1ca980) at tsan_rtl.cpp:656
19 0x000000000049523e in __tsan_trace_switch () at tsan_rtl.cpp:683
20 0x0000000000499862 in __tsan::TraceAddEvent (thr=0x7f578b1ca980, fs=..., typ=__tsan::EventTypeMop, addr=4499472) at tsan_rtl.h:624
21 __tsan::MemoryAccessRange (thr=0x7f578b1ca980, pc=4499472, addr=135257110102784, size=size@entry=16, is_write=true) at tsan_rtl_access.cpp:563
22 0x000000000049853a in __tsan::MemoryRangeFreed (thr=thr@entry=0x7f578b1ca980, pc=pc@entry=4499472, addr=addr@entry=135257110102784, size=16) at tsan_rtl_access.cpp:487
23 0x000000000048f6bf in __tsan::OnUserFree (thr=thr@entry=0x7f578b1ca980, pc=pc@entry=4499472, p=p@entry=135257110102784, write=true) at tsan_mman.cpp:260
24 0x000000000048f61f in __tsan::user_free (thr=thr@entry=0x7f578b1ca980, pc=4499472, p=p@entry=0x7b0400004300, signal=true) at tsan_mman.cpp:213
25 0x000000000044a820 in __interceptor_free (p=0x7b0400004300) at tsan_interceptors_posix.cpp:708
26 0x00000000004ad599 in alloc_free_blocks () at fork_deadlock.cpp:25
27 __tsan_test_only_on_fork () at fork_deadlock.cpp:32
28 0x0000000000494870 in __tsan::ForkBefore (thr=0x7f578b1ca980, pc=pc@entry=4904437) at tsan_rtl.cpp:510
29 0x000000000046fcb4 in syscall_pre_fork (pc=1) at tsan_interceptors_posix.cpp:2577
30 0x000000000046fc9b in __sanitizer_syscall_pre_impl_fork () at sanitizer_common_syscalls.inc:3094
31 0x00000000004ad5f5 in myfork () at syscall.h:9
32 main () at fork_deadlock.cpp:46
Depends on D114595.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114597
We currently use a wrong value for heap block
(only works for C++, but not for Java).
Use the correct value (we already computed it before, just forgot to use).
Depends on D114593.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114595
Add a basic test that checks races between vector/non-vector
read/write accesses of different sizes/offsets in different orders.
This gives coverage of __tsan_read/write16 callbacks.
Depends on D114591.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114592
Vector SSE accesses make compiler emit __tsan_[unaligned_]read/write16 callbacks.
Make it possible to test these.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114591
Building cfi with recent clang on a 64-bit system results in the
following warnings:
compiler-rt/lib/cfi/cfi.cpp:233:64: warning: format specifies type 'void *' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
VReport(1, "Can not handle: symtab > strtab (%p > %zx)\n", symtab, strtab);
~~ ^~~~~~
%lu
compiler-rt/lib/sanitizer_common/sanitizer_common.h:231:46: note: expanded from macro 'VReport'
if ((uptr)Verbosity() >= (level)) Report(__VA_ARGS__); \
^~~~~~~~~~~
compiler-rt/lib/cfi/cfi.cpp:253:59: warning: format specifies type 'void *' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
VReport(1, "Can not handle: symtab %p, strtab %zx\n", symtab, strtab);
~~ ^~~~~~
%lu
compiler-rt/lib/sanitizer_common/sanitizer_common.h:231:46: note: expanded from macro 'VReport'
if ((uptr)Verbosity() >= (level)) Report(__VA_ARGS__); \
^~~~~~~~~~~
Since `__sanitizer::uptr` has the same size as `size_t`, consistently
use `%z` as a printf specifier.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114466
Now that we lock the internal allocator around fork,
it's possible it will create additional deadlocks.
Add a fake mutex that substitutes the internal allocator
for the purposes of deadlock detection.
Depends on D114531.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114532
There is a small chance that the internal allocator is locked
during fork and then the new process is created with locked
internal allocator and any attempts to use it will deadlock.
For example, if detected a suppressed race in the parent during fork
and then another suppressed race after the fork.
This becomes much more likely with the new tsan runtime
as it uses the internal allocator for more things.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114531
The test tries to provoke internal allocator to be locked during fork
and then force the child process to use the internal allocator.
This test sometimes deadlocks with the new tsan runtime.
Depends on D114514.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114515
It was introduced in:
9cffc9550b tsan: allow to force use of __libc_malloc in sanitizer_common
and used in:
512a18e518 tsan: add standalone deadlock detector
and later used for Go support.
But now both uses are gone. Nothing defines SANITIZER_USE_MALLOC.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D114514
Test size larger than clear_shadow_mmap_threshold,
which is handled differently.
Depends on D114348.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D114366
Nothing uses more than 8bit now. So the rest of the headers can store other data.
kStackTraceMax is 256 now, but all sanitizers by default store just 20-30 frames here.
Verified no diff exist between previous version, new version python 2, and python 3 for an example stack.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D114404
Tavian Barnes