Commit Graph

15031 Commits

Author SHA1 Message Date
Julian Lettner f643afa25f Revert "Mark tsan cxa_guard_acquire test as unsupported on Darwin"
This reverts commit 593275c93c.

This test now passes again.
2021-11-03 19:07:56 -07:00
Lang Hames 870fc844d1 [ORC-RT] Add SPS serialization for span<const char> / SPSSequence<char>. 2021-11-03 13:43:49 -07:00
Vitaly Buka 91f0a6ad4e [asan] Disable test on Android Arm 32bit
Caused by D111703.
2021-11-03 13:12:56 -07:00
Vitaly Buka e28c64ecb1 [sanitizer] Disable new test on Android
Test added with D113055
2021-11-03 11:57:04 -07:00
Tamir Duberstein f639882be8 [sanitizer] Allow getsockname with NULL addrlen
This is already permitted in getpeername, and returns EFAULT
on Linux (does not crash the program).

Fixes https://github.com/google/sanitizers/issues/1451.

Differential Revision: https://reviews.llvm.org/D113055
2021-11-03 10:23:01 -07:00
Tamir Duberstein 33d9b7b4b2 [sanitizer] Mark before deref in PosixSpawnImpl
Read each pointer in the argv and envp arrays before dereferencing
it; this correctly marks an error when these pointers point into
memory that has been freed.

Differential Revision: https://reviews.llvm.org/D113046
2021-11-03 10:18:06 -07:00
Rich Lowe de6f7252da [sanitizer_common] Fix readlink error handling in sanitizer_procmaps_solaris.cpp
As pointed out in Bug 52371, the Solaris version of
`MemoryMappingLayout::Next` completely failed to handle `readlink` errors
or properly NUL-terminate the result.

This patch fixes this.  Originally provided in the PR with slight
formatting changes.

Tested on `amd64-pc-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D112998
2021-11-02 22:06:17 +01:00
Matt Morehouse 498a4c2fd7 [HWASan] Print short tags in tag mismatch description.
I recently spent some extra time debugging a false positive because I
didn't realize the "real" tag was in the short granule.  Adding the
short tag here makes it more obvious that we could be dealing with a
short granule.

Reviewed By: hctim, eugenis

Differential Revision: https://reviews.llvm.org/D112949
2021-11-02 08:00:33 -07:00
Matt Morehouse 8c60e0b632 [HWASan] Print short tags in __hwasan_print_shadow.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D112959
2021-11-02 07:59:25 -07:00
David Carlier 56513e3e42 [sanitizers] remove extra comma for sha2 interceptors 2021-11-02 10:35:44 +00:00
Vitaly Buka 08e0b9235b Revert "[NFC][sanitizer] Disable a test with large stderr output"
This reverts commit 6eb38e5171.

The test was not the reason of sanitizer-x86_64-linux-android failures.
2021-11-02 01:25:46 -07:00
Vitaly Buka 869989dd71 [sanitizer] Restore trivial posix_spawn init in test
It's still needed on Darwin.
2021-11-01 23:58:18 -07:00
Vitaly Buka 57d34cb223 Revert "[sanitizer] Fix compilation of the test"
Revert "[sanitizer] Fix PosixSpawnImpl which fails exitcode test"

This reverts commit 8cabd8f0d2 and part of 9213202abd.

It is not enough for Android.
2021-11-01 22:25:26 -07:00
Vitaly Buka 8cabd8f0d2 [sanitizer] Fix compilation of the test 2021-11-01 21:50:50 -07:00
Vitaly Buka 15361a98aa [sanitizer] Add test for 9213202abd 2021-11-01 21:14:11 -07:00
Vitaly Buka 9213202abd [sanitizer] Fix PosixSpawnImpl which fails exitcode test 2021-11-01 21:03:02 -07:00
Vitaly Buka 0d59efbba4 [sanitizer] Fix test include on Darwin 2021-11-01 16:49:50 -07:00
Matt Morehouse 4d8b0aa5c0 [HWASan] Apply TagMaskByte to every global tag.
Previously we only applied it to the first one, which could allow
subsequent global tags to exceed the valid number of bits.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D112853
2021-11-01 06:31:44 -07:00
Matt Morehouse b298d6d162 [sanitizer] Disable posix_spawn.c test on Android. 2021-11-01 06:22:48 -07:00
David Carlier b4fb42300e [sanitizer] msan build fix on non linux platforms 2021-11-01 07:18:43 +00:00
Vitaly Buka 948b91a08e [NFC][sanitizer] Atomix relaxed in TwoLevelMap
This is NOOP in x86_64.
On arch64 it avoids Data Memory Barrier with visible improvements on micro benchmarks.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D112391
2021-10-31 12:18:03 -07:00
Vitaly Buka e1240745ef [sanitizer] Add posix_spawn interceptor
To make pid initialized for Msan.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D112784
2021-10-31 12:10:54 -07:00
Lang Hames 81a6eabf71 [ORC-RT] Add missing file that was left out of ff846fcb64. 2021-10-31 10:53:17 -07:00
Lang Hames ff846fcb64 [ORC][ORC-RT] Switch MachO EH/TLV registration from EPC-calls to alloc actions.
MachOPlatform used to make an EPC-call (registerObjectSections) to register the
eh-frame and thread-data sections for each linked object with the ORC runtime.

Now that JITLinkMemoryManager supports allocation actions we can use these
instead of an EPC call. This saves us one EPC-call per object linked, and
manages registration/deregistration in the executor, rather than the controller
process. In the future we may use this to allow JIT'd code in the executor to
outlive the controller object while still being able to be cleanly destroyed.

Since the code for allocation actions must be available when the actions are
run, and since the eh-frame registration code lives in the ORC runtime itself,
this change required that MachO eh-frame support be split out of
macho_platform.cpp and into its own macho_ehframe_registration.cpp file that has
no other dependencies. During bootstrap we start by forcing emission of
macho_ehframe_registration.cpp so that eh-frame registration is guaranteed to be
available for the rest of the bootstrap process. Then we load the rest of the
MachO-platform runtime support, erroring out if there is any attempt to use
TLVs. Once the bootstrap process is complete all subsequent code can use all
features.
2021-10-31 10:27:40 -07:00
Dmitry Vyukov d31b2dc235 tsan: add debugging code for ptrace test failures
Debugging of crashes on powerpc after commit:
c80604f7a3 ("tsan: remove real func check from interceptors")
Somehow replacing if with DCHECK leads to strange failures in:
SanitizerCommon-tsan-powerpc64le-Linux :: Linux/ptrace.cpp
https://lab.llvm.org/buildbot/#/builders/105
https://lab.llvm.org/buildbot/#/builders/121
https://lab.llvm.org/buildbot/#/builders/57

The hypothesis is that something writes out-of-bounds
into pt_regs on stack and that corrupts internal tsan state.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112703
2021-10-29 17:37:02 +02:00
Dmitry Vyukov 014c6b0736 sanitizer_common: fix ParseUnixMemoryProfile test linking on Mac
The ParseUnixMemoryProfile function is defined only for a subset
of platforms. Define the test for the same set of platforms.

Also disable the test for 32-bit platforms b/c the pointer
values used in the test are 64-bit and don't fit into 32-bit uptr.

Reported-by: Jan Svoboda (jansvoboda11)

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112815
2021-10-29 16:26:24 +02:00
Rainer Orth 2f353c040f [sanitizer_common] Fix Posix/mmap_write_exec.cpp on 32-bit Solaris
The new `Posix/mmap_write_exec.cpp` test FAILs on 32-bit Solaris/x86.  This
happens because only `mmap` is intercepted, but not `mmap64` which is used
for largefile support.

Fixed by also intercepting `mmap64`.

Tested on `amd64-pc-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D112810
2021-10-29 14:29:41 +02:00
Dmitry Vyukov af0a8b8317 sanitizer_common: bump default file max size to 256MB
I am hitting some cases where /proc/self/maps does not fit into 64MB.
256MB is lots of memory, but it's not radically more than the current 64MB.
Ideally we should read/parse these huge files incrementally,
but that's lots of work for a debugging/introspection interface.
So for now just bump the limit.

Depends on D112793.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112794
2021-10-29 14:03:26 +02:00
Dmitry Vyukov e8861fa6c3 sanitizer_common: fix crashes in parsing of memory profiles
ParseUnixMemoryProfile assumes well-formed input with \n at the end, etc.
It can over-read the input and crash on basically every line
in the case of malformed input.
ReadFileToBuffer has cap the max file size (64MB) and returns
truncated contents if the file is larger. Thus even if kernel behaves,
ParseUnixMemoryProfile crashes on too large /proc/self/smaps.
Fix input over-reading in ParseUnixMemoryProfile.

Depends on D112792.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112793
2021-10-29 14:03:12 +02:00
Dmitry Vyukov 4acad5df33 sanitizer_common: add memory profile parsing test
Depends on D112790.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112792
2021-10-29 14:02:50 +02:00
Dmitry Vyukov ae833dbb81 sanitizer_common: remove unnecessary GetMemoryProfile arg
stats_size argument is unnecessary in GetMemoryProfile and in the callback.
It just clutters code. The callback knowns how many stats to expect.

Depends on D112789.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112790
2021-10-29 13:44:34 +02:00
Dmitry Vyukov d4602f759a sanitizer_common: make parsing of smaps testable (NFC)
Move parsing of /proc/self/smaps into a separate function
so that it can be tested.

Depends on D112788.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112789
2021-10-29 13:44:28 +02:00
Dmitry Vyukov a871c34e8d sanitizer_common: fix up onprint.cpp test more
D112630 ("sanitizer_common: fix up onprint.cpp test")
added O_CREAT, but we also need O_TRUNC b/c the file
may not exist, or may exist as well.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112788
2021-10-29 13:44:04 +02:00
Lang Hames dc8e5e1dc0 [ORC-RT] Add a WrapperFunctionCall utility.
WrapperFunctionCall represents a call to a wrapper function as a pair of a
target function (as an ExecutorAddr), and an argument buffer range (as an
ExecutorAddrRange). WrapperFunctionCall instances can be serialized via
SPS to send to remote machines (only the argument buffer address range is
copied, not any buffer content).

This utility will simplify the implementation of JITLinkMemoryManager
allocation actions in the ORC runtime.
2021-10-28 17:07:28 -07:00
Vitaly Buka 6eb38e5171 [NFC][sanitizer] Disable a test with large stderr output 2021-10-28 15:15:20 -07:00
David CARLIER c9174f63b6 [compiler-rt] fix asan buildbot failure on unit test for darwin 2021-10-28 18:48:54 +01:00
David CARLIER bb168f3207 [compiler-rt] update detect_write_exec option for apple devices.
Reviewed By: yln, vitalybuka

Differential Revision: https://reviews.llvm.org/D111390
2021-10-28 17:08:23 +01:00
Dmitry Vyukov d736002e90 tsan: move memory access functions to a separate file
tsan_rtl.cpp is huge and does lots of things.
Move everything related to memory access and tracing
to a separate tsan_rtl_access.cpp file.
No functional changes, only code movement.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D112625
2021-10-28 13:31:10 +02:00
Florian Mayer dd943ebc6d [hwasan] print exact mismatch offset for short granules.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D104463
2021-10-28 03:31:11 +01:00
Kai Luo 6ea2431d3f [clang][compiler-rt][atomics] Add `__c11_atomic_fetch_nand` builtin and support `__atomic_fetch_nand` libcall
Add `__c11_atomic_fetch_nand` builtin to language extensions and support `__atomic_fetch_nand` libcall in compiler-rt.

Reviewed By: theraven

Differential Revision: https://reviews.llvm.org/D112400
2021-10-28 02:18:43 +00:00
Petr Hosek 22acda48ff [CMake] Cache the compiler-rt library search results
There's a lot of duplicated calls to find various compiler-rt libraries
from build of runtime libraries like libunwind, libc++, libc++abi and
compiler-rt. The compiler-rt helper module already implemented caching
for results avoid repeated Clang invocations.

This change moves the compiler-rt implementation into a shared location
and reuses it from other runtimes to reduce duplication and speed up
the build.

Differential Revision: https://reviews.llvm.org/D88458
2021-10-27 17:53:03 -07:00
Ben Langmuir beb3d48262 [ORC-RT] Fix objc selector corruption
We were writing a pointer to a selector string into the contents of a
string instead of overwriting the pointer to the string, leading to
corruption. This was causing non-deterministic failures of the
'trivial-objc-methods' test case.

Differential Revision: https://reviews.llvm.org/D112671
2021-10-27 16:02:52 -07:00
Ben Langmuir 3d13ee2891 [ORC][ORC-RT] Enable the MachO platform for arm64
Enables the arm64 MachO platform, adds basic tests, and implements the
missing TLV relocations and runtime wrapper function. The TLV
relocations are just handled as GOT accesses.

rdar://84671534

Differential Revision: https://reviews.llvm.org/D112656
2021-10-27 13:36:03 -07:00
Dmitry Vyukov 910aeed77e sanitizer_common: fix up onprint.cpp test
Commit D112602 ("sanitizer_common: tighten on_print hook test")
changed fopen to open in this test. fopen created the file
if if does not exist, but open does not. This was unnoticed
during local testing because lit is not hermetic and reuses
files from previous runs, but it started failing on bots.
Fix the open call.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112630
2021-10-27 18:17:15 +02:00
Dmitry Vyukov eae047afe0 sanitizer_common: tighten on_print hook test
The new tsan runtime does not support arbitrary forms
of recursing into the runtime from hooks.
Disable instrumentation of the hook and use write instead
of fwrite (calls malloc internally).
The new version still recurses (write is intercepted),
but does not fail now (the issue at hand was malloc).

Depends on D112601.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112602
2021-10-27 15:11:43 +02:00
Dmitry Vyukov f50cee2f4b tsan: switch sync test from EXPECT to CHECK
Gtest's EXPECT calls whole lot of libc functions
(mem*, malloc) even when EXPECT does not fail.
This does not play well with tsan runtime unit tests
b/c e.g. we call some EXPECTs with runtime mutexes locked.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112601
2021-10-27 15:11:31 +02:00
Dmitry Vyukov abdefea6a2 tsan: tidy up SCOPED_INTERCEPTOR_RAW
Don't leak caller_pc var from the macro
(it's not supposed to be used by interceptors).
Use UNUSED instead of (void) cast.

Depends on D112540.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112541
2021-10-27 08:47:46 +02:00
Dmitry Vyukov c80604f7a3 tsan: remove real func check from interceptors
If the real function is not intercepted,
we are going to crash one way or another.
The question is just in the failure mode:
error message vs NULL deref. But the message
costs us a check in every interceptor and
they are not observed to be failing in real life
for a long time, also other sanitizers don't
have this check as well (also crash on
NULL deref if that happens).
Remove the check from non-debug mode.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112540
2021-10-27 08:47:27 +02:00
Julian Lettner c79d484ab1 [TSan] Ensure test uses tagged pointer
This is a test-only failure. The test wrongly assumes that this gets us
a tagged pointer:
```
NSObject* num1 = @7;
assert(isTaggedPtr(num1));
```

However, on newer deployment targets that have “const data support” we
get a “normal” pointer to constant object.

Radar-Id: rdar://problem/83217293
2021-10-26 11:53:02 -07:00
Dmitry Vyukov 65e795c9ca Revert "tsan: turn off COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED"
This reverts commit 5ec832269e.

It broke a number of bots, e.g.:
https://lab.llvm.org/buildbot/#/builders/52/builds/11811

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112563
2021-10-26 19:53:07 +02:00
Matt Morehouse 082d2ad015 [libFuzzer] Improve corpus replacement unit test.
Verify that SecondII is not replaced.
2021-10-26 08:09:11 -07:00
Dmitry Vyukov 5ec832269e tsan: turn off COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED
All tsan interceptors check for initialization and/or initialize things
as necessary lazily, so we can pretend everything is initialized in the
COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED check to avoid double-checking
for initialization (this is only necessary for sanitizers that don't
handle initialization on common grounds).

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112446
2021-10-26 16:13:03 +02:00
Dmitry Vyukov d53abf834c sanitizer_common: fix deadlock detector output
Print PC of the previous lock, not the current one.
The current one will be printed during unwind.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112533
2021-10-26 16:11:52 +02:00
Julian Lettner b3980b5b68 [Sanitizer] Strip PAC from PC in StackTrace::GetNextInstructionPc()
In TSan, we use the a function reference (`__tsan_stack_initialization`)
in a call to `StackTrace::GetNextInstructionPc(uptr pc)`.  We sign
function pointers, so we need to strip the signature from this function
pointer.

Caused by: https://reviews.llvm.org/D111147

Radar-Id: rdar://problem/83940546
2021-10-25 14:48:09 -07:00
Dmitry Vyukov 1b348902ea tsan: add DynamicMutexSet helper
MutexSet is too large to be allocated on stack.
But we need local MutexSet objects in few places
and use various hacks to allocate them.
Add DynamicMutexSet helper that simplifies allocation
of such objects.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112449
2021-10-25 19:45:06 +02:00
Dmitry Vyukov d7b34d0b0d msan: disble CHECK test on powerpc64
It seems that CHECK terminates the process with 0 status on powerpc64:
https://reviews.llvm.org/D112440#3084251
Disable the test for now.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112455
2021-10-25 19:38:27 +02:00
Dmitry Vyukov b0a926adac sanitizer_common: trap on CHECK failure
Trapping on CHECK failure makes it more convinient to use with gdb
(no need to set a breakpoint each time). Without a debugger attached
trap should terminate the program as well.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112440
2021-10-25 14:51:41 +02:00
Dmitry Vyukov f5a32f5a82 tsan: fix free_race3.c test
PPC64 bot failed with the following error.
The buildbot output is not particularly useful,
but looking at other similar tests, it seems
that there is something broken in free stacks on PPC64.
Use the same hack as other tests use to expect
an additional stray frame.

/home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:28:11: error: CHECK: expected string not found in input
// CHECK: Previous write of size 4 at {{.*}} by thread T1{{.*}}:
          ^
<stdin>:13:9: note: scanning from here
 #1 main /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:17:3 (free_race3.c.tmp+0x1012fab8)
        ^
<stdin>:17:2: note: possible intended match here
ThreadSanitizer: reported 1 warnings
 ^

Input file: <stdin>
Check file: /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            .
            .
            .
            8:  Previous write of size 4 at 0x7ffff4d01ab0 by thread T1:
            9:  #0 Thread /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:8:10 (free_race3.c.tmp+0x1012f9dc)
           10:
           11:  Thread T1 (tid=3222898, finished) created by main thread at:
           12:  #0 pthread_create /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1001:3 (free_race3.c.tmp+0x100b9040)
           13:  #1 main /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:17:3 (free_race3.c.tmp+0x1012fab8)
check:28'0             X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
           14:
check:28'0     ~
           15: SUMMARY: ThreadSanitizer: data race /home/buildbots/ppc64le-clang-lnt-test/clang-ppc64le-lnt/llvm/compiler-rt/test/tsan/free_race3.c:19:3 in main
check:28'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           16: ==================
check:28'0     ~~~~~~~~~~~~~~~~~~~
           17: ThreadSanitizer: reported 1 warnings
check:28'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:28'1      ?                                    possible intended match
>>>>>>

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112444
2021-10-25 14:51:24 +02:00
Dmitry Vyukov 269aa74aed tsan: add another use-after-free race test
Add a test where a race with free is called during the free itself
(we only have tests where a race with free is caught during the other memory acces).

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112433
2021-10-25 14:06:55 +02:00
Dmitry Vyukov 75a08b1ba4 tsan: don't use symbol binding 10
Building Go programs with the current runtime fails with:

loadelf: race_linux_amd64: malformed elf file:
_ZZN6__tsan15RestoreAddrImpl5ApplyINS_11MappingGo48EEEmmE6ranges: invalid symbol binding 10

Go linker does not understand ELF in all its generality.
Don't use static const data in inline methods.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112434
2021-10-25 14:05:15 +02:00
Vitaly Buka fb1c515d3b Revert "[NFC][sanitizer] constexpr a few functions"
This reverts a part of commit 8cd51a69e5
and 5bf24f0581 to fix Windows.
2021-10-24 19:26:55 -07:00
Vitaly Buka 8cd51a69e5 [NFC][sanitizer] Use power of two in TwoLevelMap
Using divisions by non power of two makes
a difference on x86_64 and aarch64 benchmarks.
2021-10-24 18:37:23 -07:00
Vitaly Buka 384ec7dc8c [NFC][sanitizer] DCHECKs in hot code 2021-10-24 18:37:23 -07:00
Vitaly Buka 5bf24f0581 [NFC][sanitizer] constexpr a few functions 2021-10-24 18:37:23 -07:00
Vitaly Buka 850217686e [sanitizer] Remove tag from StackDepotNode
And share storage with size.

Depends on D111615.

Differential Revision: https://reviews.llvm.org/D111616
2021-10-24 13:38:22 -07:00
Vitaly Buka e5859afa9f [sanitizer] Remove use_count from StackDepotNode
This is msan/dfsan data which does not need waste cache
of other sanitizers.

Depends on D111614.

Differential Revision: https://reviews.llvm.org/D111615
2021-10-24 13:38:21 -07:00
Tom Stellard 05c21f54a4 compiler-rt: Fix arch detection for ppc64le
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D110377
2021-10-21 16:12:13 -07:00
David Blaikie aee4925507 Recommit: Compress formatting of array type names (int [4] -> int[4])
Based on post-commit review discussion on
2bd8493847 with Richard Smith.

Other uses of forcing HasEmptyPlaceHolder to false seem OK to me -
they're all around pointer/reference types where the pointer/reference
token will appear at the rightmost side of the left side of the type
name, so they make nested types (eg: the "int" in "int *") behave as
though there is a non-empty placeholder (because the "*" is essentially
the placeholder as far as the "int" is concerned).

This was originally committed in 277623f4d5

Reverted in f9ad1d1c77 due to breakages
outside of clang - lldb seems to have some strange/strong dependence on
"char [N]" versus "char[N]" when printing strings (not due to that name
appearing in DWARF, but probably due to using clang to stringify type
names) that'll need to be addressed, plus a few other odds and ends in
other subprojects (clang-tools-extra, compiler-rt, etc).
2021-10-21 11:34:43 -07:00
Pirama Arumuga Nainar ab3d5d0533 Revert "[compiler-rt/profile] Hide __llvm_profile_raw_version"
This reverts commit 69708477be to unblock
instrprof-darwin-exports.c failure on MacOS bots.
2021-10-21 10:56:43 -07:00
Petr Hosek ba4920e98e Revert "[CMake] Cache the compiler-rt library search results"
This reverts commit 0eed292fba, there
are compiler-rt build failures that appear to have been introduced
by this change.
2021-10-21 10:32:01 -07:00
Ben Langmuir 92a6dd6e50 [test][ORC-RT] Disable x86_64 tests when target arch does not match
When cross-compiling, these tests will fail. For now leave the host arch
check that was already there since I don't know why it was added.
2021-10-21 10:01:23 -07:00
Ben Langmuir b8da594750 Reapply [ORC-RT] Configure the ORC runtime for more architectures and platforms
Reapply 5692ed0cce, but with the ORC runtime disabled explicitly on
CrossWinToARMLinux to match the other compiler-rt runtime libraries.

Differential Revision: https://reviews.llvm.org/D112229

---

Enable building the ORC runtime for 64-bit and 32-bit ARM architectures,
and for all Darwin embedded platforms (iOS, tvOS, and watchOS). This
covers building the cross-platform code, but does not add TLV runtime
support for the new architectures, which can be added independently.

Incidentally, stop building the Mach-O TLS support file unnecessarily on
other platforms.

Differential Revision: https://reviews.llvm.org/D112111
2021-10-21 09:00:18 -07:00
Lang Hames 5c72323141 [ORC-RT] Remove stray printf debugging output.
These were accidentally picked up in an earlier commit.
2021-10-21 08:22:58 -07:00
Nikita Malyavin 4e1a6c0705 [msan] Add stat-family interceptors on Linux
Add following interceptors on Linux: stat, lstat, fstat, fstatat.

This fixes use-of-uninitialized value on platforms with GLIBC 2.33+.
In particular: Arch Linux, Ubuntu hirsute/impish.

The tests should have also been failing during the release on the mentioned platforms, but I cannot find any related discussion.

Most likely, the regression was introduced by glibc commit [[ 8ed005daf0 | 8ed005daf0ab03e14250032 ]]:
all stat-family functions are now exported as shared functions.

Before, some of them (namely stat, lstat, fstat, fstatat) were provided as a part of libc_noshared.a and called their __xstat dopplegangers. This is still true for Debian Sid and earlier Ubuntu's. stat interceptors may be safely provided for them, no problem with that.

Closes https://github.com/google/sanitizers/issues/1452.
See also https://jira.mariadb.org/browse/MDEV-24841

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D111984
2021-10-20 17:14:43 -07:00
Ben Langmuir f3671de550 Revert "[ORC-RT] Configure the ORC runtime for more architectures and platforms"
Broke on aarch64-linux. Reverting while I investigate.

This reverts commit 5692ed0cce.
2021-10-20 15:32:06 -07:00
Ben Langmuir 5692ed0cce [ORC-RT] Configure the ORC runtime for more architectures and platforms
Enable building the ORC runtime for 64-bit and 32-bit ARM architectures,
and for all Darwin embedded platforms (iOS, tvOS, and watchOS). This
covers building the cross-platform code, but does not add TLV runtime
support for the new architectures, which can be added independently.

Incidentally, stop building the Mach-O TLS support file unnecessarily on
other platforms.

Differential Revision: https://reviews.llvm.org/D112111
2021-10-20 11:21:18 -07:00
Sterling Augustine 6b6564fcf9 Fix unused variable warning. 2021-10-20 09:59:16 -07:00
PZ Read 9e7b7303f1 [libFuzzer] Update InputInfo.TimeOfUnit when replacing it in the corpus.
Previously, when the fuzzing loop replaced an input in the corpus, it didn't update the execution time of the input. Therefore, some schedulers (e.g. Entropic) would adjust weights based on the incorrect execution time.

This patch updates the execution time of the input when replacing it.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D111479
2021-10-20 06:52:57 -07:00
Lang Hames b574c52db5 [ORC-RT] Add MethodWrapperHandler utility for WrapperFunction.
This is an ORC runtime counterpart to a01f772d19, which introduced the same
functionality into LLVM.
2021-10-19 18:38:01 -07:00
Lang Hames 21369d4b9d [ORC-RT] Use function pointer type (rather than reference) for traits class.
Aligns this template with the corresponding one in LLVM.
2021-10-19 18:38:01 -07:00
Lang Hames ea9826ff77 [ORC-RT] Avoid switching out-of-band error value into __orc_rt::Error and back.
WrapperFunctionResult can already convey serialization errors as out-of-band
error values, so there's no need to wrap it in an Expected here. Removing the
wrapper simplifies the plumbing and call sites.
2021-10-19 18:38:01 -07:00
Matt Morehouse e1e2635327 [HWASan] Use tagged-globals feature on x86.
Allows us to use the small code model when we disable relocation
relaxation.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D111344
2021-10-19 05:56:50 -07:00
Petr Hosek 0eed292fba [CMake] Cache the compiler-rt library search results
There's a lot of duplicated calls to find various compiler-rt libraries
from build of runtime libraries like libunwind, libc++, libc++abi and
compiler-rt. The compiler-rt helper module already implemented caching
for results avoid repeated Clang invocations.

This change moves the compiler-rt implementation into a shared location
and reuses it from other runtimes to reduce duplication and speed up
the build.

Differential Revision: https://reviews.llvm.org/D88458
2021-10-18 14:44:07 -07:00
Dmitry Vyukov 27969c4e00 tsan: refactor trace tests
Instead of creating real threads for trace tests
create a new ThreadState in the main thread.
This makes the tests more unit-testy and will also
help with future trace tests that will need
more than 1 thread. Creating more than 1 real thread and
dispatching test actions across multiple threads in the
required deterministic order is painful.

This is resubmit of reverted D110546 with 2 changes:
1. The previous version patched ImitateTlsWrite to not
expect ThreadState to be allocated in TLS (the CHECK
failed for the fake test threads).
This added an ugly hack into production code and was still
logically wrong because we imitated write to the main
thread TLS/stack when we started the fake test thread
(which has nothing to do with the main thread TLS/stack).
This version uses ThreadType::Fiber instead of ThreadType::Regular
for the fake threads. This naturally makes ThreadStart skip
obtaining stack/tls and imitating writes to them.

2. This version still skips the tests on Darwin and PowerPC
to be on the safer side. Build bots reported failures for PowerPC
for the previous version.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D111156
2021-10-17 12:38:56 +02:00
David Blaikie 6176fda3f9 Fix a few warnings (signed/unsigned comparison in gtest, and missing field initializers) 2021-10-16 22:43:57 -07:00
Vitaly Buka 746dd6a700 [NFC][sanitizer] Add StackDepotTestOnlyUnmap 2021-10-16 13:47:57 -07:00
Vitaly Buka c0b1b52a28 [NFC][sanitizer] Rename stack depot tests 2021-10-16 13:38:09 -07:00
Pirama Arumuga Nainar 69708477be [compiler-rt/profile] Hide __llvm_profile_raw_version
Hide __llvm_profile_raw_version so as not to resolve reference from a
dependent shared object.  Since libclang_rt.profile is added later in
the command line, a definition of __llvm_profile_raw_version is not
included if it is provided from an earlier object, e.g.  from a shared
dependency.

This causes an extra dependence edge where if libA.so depends on libB.so
and both are coverage-instrumented, libA.so uses libB.so's definition of
__llvm_profile_raw_version.  This leads to a runtime link failure if the
libB.so available at runtime does not provide this symbol (but provides
the other dependent symbols).  Such a scenario can occur in Android's
mainline modules.
E.g.:
  ld -o libB.so libclang_rt.profile-x86_64.a
  ld -o libA.so -l B libclang_rt.profile-x86_64.a

libB.so has a global definition of __llvm_profile_raw_version.  libA.so
uses libB.so's definition of __llvm_profile_raw_version.  At runtime,
libB.so may not be coverage-instrumented (i.e. not export
__llvm_profile_raw_version) so runtime linking of libA.so will fail.

Marking this symbol as hidden forces each binary to use the definition
of __llvm_profile_raw_version from libclang_rt.profile.

Differential Revision: https://reviews.llvm.org/D111759
2021-10-15 11:56:16 -07:00
Vitaly Buka e0f3a3b228 [ubsan] Remove REQUIRED from some TestCases
It's not obvious why they are needed, and tests pass.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D111859
2021-10-15 10:20:34 -07:00
Dan Liew 2232a68b2d Fix another test not using the standard separator for sanitizer options.
rdar://83637067
2021-10-15 00:20:34 -07:00
Vitaly Buka a1e78aee66 [NFC][asan] Speedup uar_signals.cpp test
It was the slowest test:
--------------------------------------------------------------------------
41.77s: AddressSanitizer-x86_64-linux :: TestCases/Linux/uar_signals.cpp
26.64s: AddressSanitizer-i386-linux :: TestCases/Linux/uar_signals.cpp
14.82s: AddressSanitizer-x86_64-linux :: TestCases/Posix/current_allocated_bytes.cpp
14.79s: AddressSanitizer-i386-linux :: TestCases/Posix/current_allocated_bytes.cpp
11.55s: AddressSanitizer-x86_64-linux :: TestCases/scariness_score_test.cpp
10.15s: AddressSanitizer-x86_64-linux :: TestCases/Posix/stack-use-after-return.cpp
2021-10-14 17:05:19 -07:00
Vitaly Buka 882ce178b3 [NFC][sanitizer] Remove %stdcxx11
-std=c++14 is a default for a while.

Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D111848
2021-10-14 16:47:43 -07:00
Vitaly Buka 1274a07da0 [NFC][asan] Use more common socket type in test 2021-10-14 16:46:07 -07:00
Evgenii Stepanov 439e00a25b [scudo] Fix running tests under hwasan.
When built with hwasan, assume that the target architecture does not
support TBI. HWASan uses that byte for its own purpose, and changing it
breaks things.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D111842
2021-10-14 14:56:58 -07:00
Evgenii Stepanov 7a3fb71c3c [hwasan] Fix TestCases/thread-uaf.c.
On newer glibc, this test detects an extra match somewhere under
pthread_getattr_np. This results in Thread: lines getting spread out in
the report and failing to match the CHECKs.

Fix the CHECKs to allow this possibility.

Reviewed By: fmayer

Differential Revision: https://reviews.llvm.org/D111841
2021-10-14 14:56:57 -07:00
Evgenii Stepanov 039096ee71 [hwasan] Add default "/" prefix.
Add a default "/" prefix to the symbol search path in the
symbolization script. Without this, the binary itself is not considered
a valid source of symbol info.

Differential Revision: https://reviews.llvm.org/D111840
2021-10-14 14:56:55 -07:00
Collin Baker 8c66d78172 [test] Fix asan dynamic unit tests with per-target runtime dirs
When LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=on
Asan-i386-calls-Dynamic-Test and Asan-i386-inline-Dynamic-Test fail to
run on a x86_64 host. This is because asan's unit test lit files are
configured once, rather than per target arch as with the non-unit
tests. LD_LIBRARY_PATH ends up incorrect, and the tests try linking
against the x86_64 runtime which fails.

This changes the unit test CMake machinery to configure the default
and dynamic unit tests once per target arch, similar to the other asan
tests. Then the fix from https://reviews.llvm.org/D108859 is adapted
to the unit test Lit files with some modifications.

Fixes PR52158.

Differential Revision: https://reviews.llvm.org/D111756
2021-10-14 16:47:25 -04:00
Vitaly Buka 8282024a74 [sanitizer] Move out stack trace pointer from header StackDepot
Trace pointers accessed very rarely and don't need to
be in hot data.

Depends on D111613.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111614
2021-10-14 09:23:04 -07:00
Jinsong Ji 4fee8a1691 [NFC][compiler-rt][profile] Remove non-Posix -h option from test
We are running `ls -lh` in gcov-execlp.c test in Posix folder.
However `-h` is not a POSIX option,ls on some POSIX system (eg: AIX)
may not support it.

This patch remove this option to avoid break.

Reviewed By: anhtuyen

Differential Revision: https://reviews.llvm.org/D111807
2021-10-14 15:08:38 +00:00
Vitaly Buka 8383e49b53 [sanitizer] Cleanup benchmark 2021-10-13 13:58:28 -07:00
Vitaly Buka 43bae7ae26 [sanitizer] Add trivial StackDepot benchmark 2021-10-13 12:03:13 -07:00
Jinsong Ji 666accf283 [compiler-rt][profile] Enable profile tests for AIX
This patch enable profile test for supported options on AIX.

Reviewed By: w2yehia

Differential Revision: https://reviews.llvm.org/D110945
2021-10-13 03:00:32 +00:00
Gulfem Savrun Yeniceri 9a70eb918e [profile] Remove emitting symbolizer markup
Fuchsia Clang code coverage pipeline started to use binary ids that are
embedded in profiles. This patch removes emitting symbolizer markup,
which is not necessary in the coverage pipeline anymore.

Differential Revision: https://reviews.llvm.org/D111674
2021-10-13 01:43:07 +00:00
Vitaly Buka ca0036df7d [sanitizer] Remove StackDepotReverseMap
Now StackDepotGet can retrive the stack in O(1).

Depends on D111612.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111613
2021-10-12 15:59:27 -07:00
Vitaly Buka ce7f8c8474 [sanitizer] Remove id and replace link with u32
This lets us reduce size of Node, similar to D111183 proposal.

Depends on D111610.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111612
2021-10-12 15:53:28 -07:00
Vitaly Buka f815c2ccda [sanitizer] Fix test on Windows 2021-10-12 15:38:37 -07:00
Vitaly Buka 3ec4d4dc99 [NFC][sanitizer] Add a few consts
Depends on D111609.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111610
2021-10-12 15:38:37 -07:00
Vitaly Buka d402fb02b9 [NFC][sanitizer] Clang-format a line
Depends on D111608.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111609
2021-10-12 14:11:26 -07:00
Vitaly Buka 67905bcaf3 [sanitizer] Re-enable test on Windows
It's likely missdiagnosed issue fixed with 336e88636a
2021-10-12 14:10:46 -07:00
Vitaly Buka d80a5d54e1 [NFC][sanitizer] Move consts into on top of the class 2021-10-12 13:59:16 -07:00
Vitaly Buka 38f121cd84 [sanitizer] Switch StackDepot to TwoLevelMap
Depends on D111607.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111608
2021-10-12 13:57:30 -07:00
Vitaly Buka 884d290427 [sanitizer] Remove SpinMutexLock from stat function
Mapped size is more appropriate for most users.

Depends on D111605.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111607
2021-10-12 13:49:10 -07:00
Vitaly Buka 9939e562f7 [NFC][sanitizer] Refactor Maps into templates
Depends on D111599.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111605
2021-10-12 13:37:44 -07:00
Vitaly Buka 336e88636a [sanitizer] Fix test on Windows
UL suffix on MSVC is 32bit.
2021-10-12 13:24:40 -07:00
Vitaly Buka 0e0d3b3326 [sanitizer] Fix typo in test 2021-10-12 13:24:40 -07:00
Vitaly Buka b4db2a500d [sanitizer] Fix StackDepotPrintAll
unlock corrupted backets by using s set by loop to nullptr.
Also StackDepot supports iterating without locking.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111599
2021-10-12 10:57:40 -07:00
Kostya Kortchinsky 56a9effc42 [scudo] Skip AllocAfterFork test on machines with low max_map_count
Reducing the number of iterations in that test with D111342 helped,
but the failure still occured flakily when the test is ran as part
of a large test suite.

Reducing further the number of iterations might not be good enough,
so we will skip the test if the `max_map_count` variable can be
read, and if lower than a given threshold.

Differential Revision: https://reviews.llvm.org/D111465
2021-10-11 10:33:47 -07:00
Andrew Browne 50a08e2c6d [DFSan] Fix flakey release_shadow_space.c accounting for Origin chains.
Test sometimes fails on buildbot (after two non-Origins executions):

/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 317992, after fixed map: 10792, after another mmap+set label: 317992, after munmap: 10792
release_shadow_space.c.tmp: /b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/dfsan/release_shadow_space.c:91: int main(int, char **): Assertion `after_fixed_mmap <= before + delta' failed.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D111522
2021-10-11 00:35:12 -07:00
Vitaly Buka 9ccb6024a0 [NFC][sanitizer] Add a few consts 2021-10-10 22:59:43 -07:00
Vitaly Buka 982bfec8f0 [NFC][sanitizer] Clang-format sanitizer_flat_map.h 2021-10-10 22:23:49 -07:00
Vitaly Buka eff6b369bf [NFC][sanitizer] Add constexpr to FlatMap::size 2021-10-10 22:23:48 -07:00
Vitaly Buka 76b7784bcd [NFC][sanitizer] Rename ByteMap to Map 2021-10-10 22:23:48 -07:00
Vitaly Buka 74277e254c [NFC] Allow to include sanitizer_allocator_bytemap.h 2021-10-10 22:23:48 -07:00
luxufan 590326382d [Orc] Support atexit in Orc(JITLink)
There is a bug reported at https://bugs.llvm.org/show_bug.cgi?id=48938

After looking through the glibc, I found the `atexit(f)` is the same as `__cxa_atexit(f, NULL, NULL)`. In orc runtime, we identify different JITDylib by their dso_handle value, so that a NULL dso_handle is invalid. So in this patch, I added a `PlatformJDDSOHandle` to ELFNixRuntimeState, and functions which are registered by atexit will be registered at PlatformJD.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D111413
2021-10-09 12:25:47 +08:00
Vitaly Buka 6800222068 [NFC][sanitizer] Add const to ChainedOriginDepotNode methods 2021-10-08 14:27:05 -07:00
Vitaly Buka df43d419de [NFC][sanitizer] Remove includes from header 2021-10-08 14:27:05 -07:00
Vitaly Buka d1aaef4296 [NFC][sanitizer] Parametrize PersistentAllocator with type 2021-10-08 14:07:05 -07:00
Vitaly Buka d2aa34e8d9 [NFC][sanitizer] Move ChainedOriginDepotNode into cpp file 2021-10-08 13:43:29 -07:00
Vitaly Buka 05d46f627c [NFC][sanitizer] Remove sanitizer_persistent_allocator.cpp
We need to make it a template
2021-10-08 13:43:28 -07:00
Andrew Browne 61ec2148c5 [DFSan] Remove -dfsan-args-abi support in favor of TLS.
ArgsABI was originally added in https://reviews.llvm.org/D965

Current benchmarking does not show a significant difference.
There is no need to maintain both ABIs.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D111097
2021-10-08 11:18:36 -07:00
Fangrui Song b3024ac084 [sanitizer] Use one #if instead of 3 nested #if after D111185 2021-10-08 10:31:57 -07:00
H.J. Lu c960c8c339 Reland [sanitizer] Support Intel CET
1. Include <cet.h> in sanitizer_common/sanitizer_asm.h, if it exists, to
mark Intel CET support when Intel CET is enabled.
2. Define _CET_ENDBR as empty if it isn't defined.
3. Add _CET_ENDBR to function entries in assembly codes so that ENDBR
instruction will be generated when Intel CET is enabled.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D111185
2021-10-08 10:22:39 -07:00
David Spickett 30677a043c [compiler-rt][fuzzer] Re-enable flags test on AArch64 Linux
This is now passing after bots were upgraded to Ubuntu Focal,
which comes with ld 2.34.
2021-10-08 08:55:22 +00:00
Arthur Eubanks d4c1f222f2 Revert "[sanitizer] Support Intel CET"
This reverts commit fdf4c03522.

Breaks macOS bots, e.g. https://crbug.com/1257863.
Still figuring out if this is actually supported on macOS. Other places
that include <cet.h> only do so on Linux.
2021-10-07 21:03:12 -07:00
Vitaly Buka ef85ea9a4f [msan] Print both shadow and user address
before:
00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
Shadow map of [0x211000000005, 0x21100000012e), 297 bytes:
now:
0x2f60d213ac10[0x7f60d213ac10]  00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
Shadow map [0x211000000005, 0x21100000012e) of [0x711000000005, 0x711000000135), 297 bytes:

Differential Revision: https://reviews.llvm.org/D111261
2021-10-07 17:56:46 -07:00
Kostya Kortchinsky 6727832c32 [scudo] Reduce the scope of AllocAfterFork
`ScudoWrappersCppTest.AllocAfterFork` was failing obscurely sometimes.
Someone pointed us to Linux's `vm.max_map_count` that can be
significantly lower on some machines than others. It turned out that
on a machine with that setting set to 65530, some `ENOMEM` errors
would occur with `mmap` & `mprotect` during that specific test.

Reducing the number of times we fork, and the maximum size allocated
during that test makes it pass on those machines.

Differential Revision: https://reviews.llvm.org/D111342
2021-10-07 14:01:58 -07:00
Vitaly Buka 0332d5d14d [NFC][sanitizer] Annotate a few branches in StackDepot 2021-10-07 13:54:02 -07:00
Vitaly Buka c86e7ec42c [sanitizer] Remove traces from the header
This will simplify removing id proposed by @dvyukov on D111183
Also now we have more flexiliby for traces compressio they
are not interleaving with uncompressable headers.

Depends on D111256.

Differential Revision: https://reviews.llvm.org/D111274
2021-10-07 13:54:01 -07:00
Vitaly Buka 8f3e52538d [NFC][sanitizer] Remove global PersistentAllocator
This way is easier to track memory usage and do other
incremental refactorings.

Differential Revision: https://reviews.llvm.org/D111256
2021-10-07 13:54:01 -07:00
Vitaly Buka 78c5754813 [sanitizer] Uninline slow path of PersistentAllocator::alloc 2021-10-07 13:54:01 -07:00
Leonard Chan 7afd956e0f [compiler-rt][memprof] Disambiguate checks for __tls_get_addr in output
TestCases/stress_dtls.c was failing when we ran memprof tests for the first
time. The test checks that __tls_get_addr is not in the output for the last
run when it is possible for the interceptor __interceptor___tls_get_addr to
be in the output from stack dumps. The test actually intends to check that
the various __tls_get_addr reports don't get emitted when intercept_tls_get_addr=0.
This updates the test to also check for the following `:` and preceding `==`
which should ignore the __interceptor___tls_get_addr interceptor.

Differential Revision: https://reviews.llvm.org/D111192
2021-10-06 13:54:42 -07:00
Dan Liew 9ed6b1cd97 Disable SANITIZER_CHECK_DEADLOCKS on Darwin platforms.
Although THREADLOCAL variables are supported on Darwin they cannot be
used very early on during process init (before dyld has set it up).

Unfortunately the checked lock is used before dyld has setup TLS leading
to an abort call (`_tlv_boostrap()` is never supposed to be called at
runtime).

To avoid this problem `SANITIZER_CHECK_DEADLOCKS` is now disabled on
Darwin platforms. This fixes running TSan tests (an possibly other
Sanitizers) when `COMPILER_RT_DEBUG=ON`.

For reference the crashing backtrace looks like this:

```
* thread #1, stop reason = signal SIGABRT
  * frame #0: 0x00000002044da0ae dyld`__abort_with_payload + 10
    frame #1: 0x00000002044f01af dyld`abort_with_payload_wrapper_internal + 80
    frame #2: 0x00000002044f01e1 dyld`abort_with_payload + 9
    frame #3: 0x000000010c989060 dyld_sim`abort_with_payload + 26
    frame #4: 0x000000010c94908b dyld_sim`dyld4::halt(char const*) + 375
    frame #5: 0x000000010c988f5c dyld_sim`abort + 16
    frame #6: 0x000000010c96104f dyld_sim`dyld4::APIs::_tlv_bootstrap() + 9
    frame #7: 0x000000010cd8d6d2 libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::CheckedMutex::LockImpl(this=<unavailable>, pc=<unavailable>) at sanitizer_mutex.cpp:218:58 [opt]
    frame #8: 0x000000010cd8a0f7 libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::Mutex::Lock() [inlined] __sanitizer::CheckedMutex::Lock(this=0x000000010d733c90) at sanitizer_mutex.h:124:5 [opt]
    frame #9: 0x000000010cd8a0ee libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::Mutex::Lock(this=0x000000010d733c90) at sanitizer_mutex.h:162:19 [opt]
    frame #10: 0x000000010cd8a0bf libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::GenericScopedLock<__sanitizer::Mutex>::GenericScopedLock(this=0x000000030c7479a8, mu=<unavailable>) at sanitizer_mutex.h:364:10 [opt]
    frame #11: 0x000000010cd89819 libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::GenericScopedLock<__sanitizer::Mutex>::GenericScopedLock(this=0x000000030c7479a8, mu=<unavailable>) at sanitizer_mutex.h:363:67 [opt]
    frame #12: 0x000000010cd8985b libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::LibIgnore::OnLibraryLoaded(this=0x000000010d72f480, name=0x0000000000000000) at sanitizer_libignore.cpp:39:8 [opt]
    frame #13: 0x000000010cda7aaa libclang_rt.tsan_iossim_dynamic.dylib`__tsan::InitializeLibIgnore() at tsan_interceptors_posix.cpp:219:16 [opt]
    frame #14: 0x000000010cdce0bb libclang_rt.tsan_iossim_dynamic.dylib`__tsan::Initialize(thr=0x0000000110141400) at tsan_rtl.cpp:403:3 [opt]
    frame #15: 0x000000010cda7b8e libclang_rt.tsan_iossim_dynamic.dylib`__tsan::ScopedInterceptor::ScopedInterceptor(__tsan::ThreadState*, char const*, unsigned long) [inlined] __tsan::LazyInitialize(thr=0x0000000110141400) at tsan_rtl.h:665:5 [opt]
    frame #16: 0x000000010cda7b86 libclang_rt.tsan_iossim_dynamic.dylib`__tsan::ScopedInterceptor::ScopedInterceptor(this=0x000000030c747af8, thr=0x0000000110141400, fname=<unavailable>, pc=4568918787) at tsan_interceptors_posix.cpp:247:3 [opt]
    frame #17: 0x000000010cda7bb9 libclang_rt.tsan_iossim_dynamic.dylib`__tsan::ScopedInterceptor::ScopedInterceptor(this=0x000000030c747af8, thr=<unavailable>, fname=<unavailable>, pc=<unavailable>) at tsan_interceptors_posix.cpp:246:59 [opt]
    frame #18: 0x000000010cdb72b7 libclang_rt.tsan_iossim_dynamic.dylib`::wrap_strlcpy(dst="\xd2", src="0xd1d398d1bb0a007b", size=20) at sanitizer_common_interceptors.inc:7386:3 [opt]
    frame #19: 0x0000000110542b03 libsystem_c.dylib`__guard_setup + 140
    frame #20: 0x00000001104f8ab4 libsystem_c.dylib`_libc_initializer + 65
    ...
```

rdar://83723445

Differential Revision: https://reviews.llvm.org/D111243
2021-10-06 12:05:33 -07:00
Leonard Chan 77d5ccdc6f [compiler-rt][test] Add shared_unwind requirement
When using a static libunwind, the check_memcpy.c can fail because it checks
that tsan intercepted all memcpy/memmoves in the final binary. Though if the
static libunwind is not instrumented, then this will fail because it may contain
regular memcpy/memmoves.

This adds a new REQUIRES check for ensuring that this test won't run unless a
dynamic libunwind.so is provided.

Differential Revision: https://reviews.llvm.org/D111194
2021-10-06 11:10:36 -07:00
Vitaly Buka b5b1b3aef1 [sanitizer] Switch to StackDepotNode to 64bit hash
Now we can avoid scanning the stack on fast path.
The price is the false stack trace with probability of the hash collision.
This increase performance of lsan by 6% and pre-requirement for stack compression.

Depends on D111182.

Reviewed By: morehouse, dvyukov

Differential Revision: https://reviews.llvm.org/D111183
2021-10-06 10:45:11 -07:00
H.J. Lu fdf4c03522 [sanitizer] Support Intel CET
1. Include <cet.h> in sanitizer_common/sanitizer_asm.h to mark Intel CET
support when Intel CET is enabled.
2. Add _CET_ENDBR to function entries in assembly codes so that ENDBR
instruction will be generated when Intel CET is enabled.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111185
2021-10-06 10:12:50 -07:00
David Spickett f8f1bb7462 [compiler-rt][lsan] Add backup AArch64 register for use_registers test
On Ubuntu Focal x13 is used by something in the process of calling
sched_yield. Causing the test to fail depending on when the thread
is stopped.

Adding x14 works around this and the test passes consistently.

Not switching to only x14 because that could make other platforms
fail. With both we'll always find at least one and even if both
values are present we'll only get one report.

Reviewed By: oontvoo, vitalybuka

Differential Revision: https://reviews.llvm.org/D110931
2021-10-06 09:45:53 +01:00
Dmitry Vyukov 24af1ba605 tsan: don't instrument runtime callbacks in tests
These runtime callbacks are supposed to be non-instrumented,
we can't handle runtime recursion well, nor can we afford
explicit recursion checks in the hot functions (memory access,
function entry/exit).
It used to work (not crash), but it won't work with the new runtime.
Mark all runtime callbacks as non-instrumented.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D111157
2021-10-06 08:44:04 +02:00
David Carlier 18a7ebda99 [Sanitizers] intercept md5 and sha* apis on FreeBSD.
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110989
2021-10-06 06:01:50 +01:00
Vitaly Buka 3129aa5caf [NFC][sanitizers] Add StackDepotBase Node::hash_type
Depends on D111177.

Differential Revision: https://reviews.llvm.org/D111182
2021-10-05 20:54:06 -07:00
Vitaly Buka 5ae9a3e4bf [NFC][sanitizer] Add MurMur2Hash64Builder
Depends on D111176.

Differential Revision: https://reviews.llvm.org/D111177
2021-10-05 20:54:06 -07:00