Commit Graph

15031 Commits

Author SHA1 Message Date
Vitaly Buka 43bae7ae26 [sanitizer] Add trivial StackDepot benchmark 2021-10-13 12:03:13 -07:00
Jinsong Ji 666accf283 [compiler-rt][profile] Enable profile tests for AIX
This patch enable profile test for supported options on AIX.

Reviewed By: w2yehia

Differential Revision: https://reviews.llvm.org/D110945
2021-10-13 03:00:32 +00:00
Gulfem Savrun Yeniceri 9a70eb918e [profile] Remove emitting symbolizer markup
Fuchsia Clang code coverage pipeline started to use binary ids that are
embedded in profiles. This patch removes emitting symbolizer markup,
which is not necessary in the coverage pipeline anymore.

Differential Revision: https://reviews.llvm.org/D111674
2021-10-13 01:43:07 +00:00
Vitaly Buka ca0036df7d [sanitizer] Remove StackDepotReverseMap
Now StackDepotGet can retrive the stack in O(1).

Depends on D111612.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111613
2021-10-12 15:59:27 -07:00
Vitaly Buka ce7f8c8474 [sanitizer] Remove id and replace link with u32
This lets us reduce size of Node, similar to D111183 proposal.

Depends on D111610.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111612
2021-10-12 15:53:28 -07:00
Vitaly Buka f815c2ccda [sanitizer] Fix test on Windows 2021-10-12 15:38:37 -07:00
Vitaly Buka 3ec4d4dc99 [NFC][sanitizer] Add a few consts
Depends on D111609.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111610
2021-10-12 15:38:37 -07:00
Vitaly Buka d402fb02b9 [NFC][sanitizer] Clang-format a line
Depends on D111608.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111609
2021-10-12 14:11:26 -07:00
Vitaly Buka 67905bcaf3 [sanitizer] Re-enable test on Windows
It's likely missdiagnosed issue fixed with 336e88636a
2021-10-12 14:10:46 -07:00
Vitaly Buka d80a5d54e1 [NFC][sanitizer] Move consts into on top of the class 2021-10-12 13:59:16 -07:00
Vitaly Buka 38f121cd84 [sanitizer] Switch StackDepot to TwoLevelMap
Depends on D111607.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111608
2021-10-12 13:57:30 -07:00
Vitaly Buka 884d290427 [sanitizer] Remove SpinMutexLock from stat function
Mapped size is more appropriate for most users.

Depends on D111605.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111607
2021-10-12 13:49:10 -07:00
Vitaly Buka 9939e562f7 [NFC][sanitizer] Refactor Maps into templates
Depends on D111599.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111605
2021-10-12 13:37:44 -07:00
Vitaly Buka 336e88636a [sanitizer] Fix test on Windows
UL suffix on MSVC is 32bit.
2021-10-12 13:24:40 -07:00
Vitaly Buka 0e0d3b3326 [sanitizer] Fix typo in test 2021-10-12 13:24:40 -07:00
Vitaly Buka b4db2a500d [sanitizer] Fix StackDepotPrintAll
unlock corrupted backets by using s set by loop to nullptr.
Also StackDepot supports iterating without locking.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111599
2021-10-12 10:57:40 -07:00
Kostya Kortchinsky 56a9effc42 [scudo] Skip AllocAfterFork test on machines with low max_map_count
Reducing the number of iterations in that test with D111342 helped,
but the failure still occured flakily when the test is ran as part
of a large test suite.

Reducing further the number of iterations might not be good enough,
so we will skip the test if the `max_map_count` variable can be
read, and if lower than a given threshold.

Differential Revision: https://reviews.llvm.org/D111465
2021-10-11 10:33:47 -07:00
Andrew Browne 50a08e2c6d [DFSan] Fix flakey release_shadow_space.c accounting for Origin chains.
Test sometimes fails on buildbot (after two non-Origins executions):

/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 209424, after fixed map: 4624, after another mmap+set label: 209424, after munmap: 4624
/usr/bin/ld: warning: Cannot export local symbol 'dfsan_flush'
RSS at start: 4620, after mmap: 107020, after mmap+set label: 317992, after fixed map: 10792, after another mmap+set label: 317992, after munmap: 10792
release_shadow_space.c.tmp: /b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/dfsan/release_shadow_space.c:91: int main(int, char **): Assertion `after_fixed_mmap <= before + delta' failed.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D111522
2021-10-11 00:35:12 -07:00
Vitaly Buka 9ccb6024a0 [NFC][sanitizer] Add a few consts 2021-10-10 22:59:43 -07:00
Vitaly Buka 982bfec8f0 [NFC][sanitizer] Clang-format sanitizer_flat_map.h 2021-10-10 22:23:49 -07:00
Vitaly Buka eff6b369bf [NFC][sanitizer] Add constexpr to FlatMap::size 2021-10-10 22:23:48 -07:00
Vitaly Buka 76b7784bcd [NFC][sanitizer] Rename ByteMap to Map 2021-10-10 22:23:48 -07:00
Vitaly Buka 74277e254c [NFC] Allow to include sanitizer_allocator_bytemap.h 2021-10-10 22:23:48 -07:00
luxufan 590326382d [Orc] Support atexit in Orc(JITLink)
There is a bug reported at https://bugs.llvm.org/show_bug.cgi?id=48938

After looking through the glibc, I found the `atexit(f)` is the same as `__cxa_atexit(f, NULL, NULL)`. In orc runtime, we identify different JITDylib by their dso_handle value, so that a NULL dso_handle is invalid. So in this patch, I added a `PlatformJDDSOHandle` to ELFNixRuntimeState, and functions which are registered by atexit will be registered at PlatformJD.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D111413
2021-10-09 12:25:47 +08:00
Vitaly Buka 6800222068 [NFC][sanitizer] Add const to ChainedOriginDepotNode methods 2021-10-08 14:27:05 -07:00
Vitaly Buka df43d419de [NFC][sanitizer] Remove includes from header 2021-10-08 14:27:05 -07:00
Vitaly Buka d1aaef4296 [NFC][sanitizer] Parametrize PersistentAllocator with type 2021-10-08 14:07:05 -07:00
Vitaly Buka d2aa34e8d9 [NFC][sanitizer] Move ChainedOriginDepotNode into cpp file 2021-10-08 13:43:29 -07:00
Vitaly Buka 05d46f627c [NFC][sanitizer] Remove sanitizer_persistent_allocator.cpp
We need to make it a template
2021-10-08 13:43:28 -07:00
Andrew Browne 61ec2148c5 [DFSan] Remove -dfsan-args-abi support in favor of TLS.
ArgsABI was originally added in https://reviews.llvm.org/D965

Current benchmarking does not show a significant difference.
There is no need to maintain both ABIs.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D111097
2021-10-08 11:18:36 -07:00
Fangrui Song b3024ac084 [sanitizer] Use one #if instead of 3 nested #if after D111185 2021-10-08 10:31:57 -07:00
H.J. Lu c960c8c339 Reland [sanitizer] Support Intel CET
1. Include <cet.h> in sanitizer_common/sanitizer_asm.h, if it exists, to
mark Intel CET support when Intel CET is enabled.
2. Define _CET_ENDBR as empty if it isn't defined.
3. Add _CET_ENDBR to function entries in assembly codes so that ENDBR
instruction will be generated when Intel CET is enabled.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D111185
2021-10-08 10:22:39 -07:00
David Spickett 30677a043c [compiler-rt][fuzzer] Re-enable flags test on AArch64 Linux
This is now passing after bots were upgraded to Ubuntu Focal,
which comes with ld 2.34.
2021-10-08 08:55:22 +00:00
Arthur Eubanks d4c1f222f2 Revert "[sanitizer] Support Intel CET"
This reverts commit fdf4c03522.

Breaks macOS bots, e.g. https://crbug.com/1257863.
Still figuring out if this is actually supported on macOS. Other places
that include <cet.h> only do so on Linux.
2021-10-07 21:03:12 -07:00
Vitaly Buka ef85ea9a4f [msan] Print both shadow and user address
before:
00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
Shadow map of [0x211000000005, 0x21100000012e), 297 bytes:
now:
0x2f60d213ac10[0x7f60d213ac10]  00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
Shadow map [0x211000000005, 0x21100000012e) of [0x711000000005, 0x711000000135), 297 bytes:

Differential Revision: https://reviews.llvm.org/D111261
2021-10-07 17:56:46 -07:00
Kostya Kortchinsky 6727832c32 [scudo] Reduce the scope of AllocAfterFork
`ScudoWrappersCppTest.AllocAfterFork` was failing obscurely sometimes.
Someone pointed us to Linux's `vm.max_map_count` that can be
significantly lower on some machines than others. It turned out that
on a machine with that setting set to 65530, some `ENOMEM` errors
would occur with `mmap` & `mprotect` during that specific test.

Reducing the number of times we fork, and the maximum size allocated
during that test makes it pass on those machines.

Differential Revision: https://reviews.llvm.org/D111342
2021-10-07 14:01:58 -07:00
Vitaly Buka 0332d5d14d [NFC][sanitizer] Annotate a few branches in StackDepot 2021-10-07 13:54:02 -07:00
Vitaly Buka c86e7ec42c [sanitizer] Remove traces from the header
This will simplify removing id proposed by @dvyukov on D111183
Also now we have more flexiliby for traces compressio they
are not interleaving with uncompressable headers.

Depends on D111256.

Differential Revision: https://reviews.llvm.org/D111274
2021-10-07 13:54:01 -07:00
Vitaly Buka 8f3e52538d [NFC][sanitizer] Remove global PersistentAllocator
This way is easier to track memory usage and do other
incremental refactorings.

Differential Revision: https://reviews.llvm.org/D111256
2021-10-07 13:54:01 -07:00
Vitaly Buka 78c5754813 [sanitizer] Uninline slow path of PersistentAllocator::alloc 2021-10-07 13:54:01 -07:00
Leonard Chan 7afd956e0f [compiler-rt][memprof] Disambiguate checks for __tls_get_addr in output
TestCases/stress_dtls.c was failing when we ran memprof tests for the first
time. The test checks that __tls_get_addr is not in the output for the last
run when it is possible for the interceptor __interceptor___tls_get_addr to
be in the output from stack dumps. The test actually intends to check that
the various __tls_get_addr reports don't get emitted when intercept_tls_get_addr=0.
This updates the test to also check for the following `:` and preceding `==`
which should ignore the __interceptor___tls_get_addr interceptor.

Differential Revision: https://reviews.llvm.org/D111192
2021-10-06 13:54:42 -07:00
Dan Liew 9ed6b1cd97 Disable SANITIZER_CHECK_DEADLOCKS on Darwin platforms.
Although THREADLOCAL variables are supported on Darwin they cannot be
used very early on during process init (before dyld has set it up).

Unfortunately the checked lock is used before dyld has setup TLS leading
to an abort call (`_tlv_boostrap()` is never supposed to be called at
runtime).

To avoid this problem `SANITIZER_CHECK_DEADLOCKS` is now disabled on
Darwin platforms. This fixes running TSan tests (an possibly other
Sanitizers) when `COMPILER_RT_DEBUG=ON`.

For reference the crashing backtrace looks like this:

```
* thread #1, stop reason = signal SIGABRT
  * frame #0: 0x00000002044da0ae dyld`__abort_with_payload + 10
    frame #1: 0x00000002044f01af dyld`abort_with_payload_wrapper_internal + 80
    frame #2: 0x00000002044f01e1 dyld`abort_with_payload + 9
    frame #3: 0x000000010c989060 dyld_sim`abort_with_payload + 26
    frame #4: 0x000000010c94908b dyld_sim`dyld4::halt(char const*) + 375
    frame #5: 0x000000010c988f5c dyld_sim`abort + 16
    frame #6: 0x000000010c96104f dyld_sim`dyld4::APIs::_tlv_bootstrap() + 9
    frame #7: 0x000000010cd8d6d2 libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::CheckedMutex::LockImpl(this=<unavailable>, pc=<unavailable>) at sanitizer_mutex.cpp:218:58 [opt]
    frame #8: 0x000000010cd8a0f7 libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::Mutex::Lock() [inlined] __sanitizer::CheckedMutex::Lock(this=0x000000010d733c90) at sanitizer_mutex.h:124:5 [opt]
    frame #9: 0x000000010cd8a0ee libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::Mutex::Lock(this=0x000000010d733c90) at sanitizer_mutex.h:162:19 [opt]
    frame #10: 0x000000010cd8a0bf libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::GenericScopedLock<__sanitizer::Mutex>::GenericScopedLock(this=0x000000030c7479a8, mu=<unavailable>) at sanitizer_mutex.h:364:10 [opt]
    frame #11: 0x000000010cd89819 libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::GenericScopedLock<__sanitizer::Mutex>::GenericScopedLock(this=0x000000030c7479a8, mu=<unavailable>) at sanitizer_mutex.h:363:67 [opt]
    frame #12: 0x000000010cd8985b libclang_rt.tsan_iossim_dynamic.dylib`__sanitizer::LibIgnore::OnLibraryLoaded(this=0x000000010d72f480, name=0x0000000000000000) at sanitizer_libignore.cpp:39:8 [opt]
    frame #13: 0x000000010cda7aaa libclang_rt.tsan_iossim_dynamic.dylib`__tsan::InitializeLibIgnore() at tsan_interceptors_posix.cpp:219:16 [opt]
    frame #14: 0x000000010cdce0bb libclang_rt.tsan_iossim_dynamic.dylib`__tsan::Initialize(thr=0x0000000110141400) at tsan_rtl.cpp:403:3 [opt]
    frame #15: 0x000000010cda7b8e libclang_rt.tsan_iossim_dynamic.dylib`__tsan::ScopedInterceptor::ScopedInterceptor(__tsan::ThreadState*, char const*, unsigned long) [inlined] __tsan::LazyInitialize(thr=0x0000000110141400) at tsan_rtl.h:665:5 [opt]
    frame #16: 0x000000010cda7b86 libclang_rt.tsan_iossim_dynamic.dylib`__tsan::ScopedInterceptor::ScopedInterceptor(this=0x000000030c747af8, thr=0x0000000110141400, fname=<unavailable>, pc=4568918787) at tsan_interceptors_posix.cpp:247:3 [opt]
    frame #17: 0x000000010cda7bb9 libclang_rt.tsan_iossim_dynamic.dylib`__tsan::ScopedInterceptor::ScopedInterceptor(this=0x000000030c747af8, thr=<unavailable>, fname=<unavailable>, pc=<unavailable>) at tsan_interceptors_posix.cpp:246:59 [opt]
    frame #18: 0x000000010cdb72b7 libclang_rt.tsan_iossim_dynamic.dylib`::wrap_strlcpy(dst="\xd2", src="0xd1d398d1bb0a007b", size=20) at sanitizer_common_interceptors.inc:7386:3 [opt]
    frame #19: 0x0000000110542b03 libsystem_c.dylib`__guard_setup + 140
    frame #20: 0x00000001104f8ab4 libsystem_c.dylib`_libc_initializer + 65
    ...
```

rdar://83723445

Differential Revision: https://reviews.llvm.org/D111243
2021-10-06 12:05:33 -07:00
Leonard Chan 77d5ccdc6f [compiler-rt][test] Add shared_unwind requirement
When using a static libunwind, the check_memcpy.c can fail because it checks
that tsan intercepted all memcpy/memmoves in the final binary. Though if the
static libunwind is not instrumented, then this will fail because it may contain
regular memcpy/memmoves.

This adds a new REQUIRES check for ensuring that this test won't run unless a
dynamic libunwind.so is provided.

Differential Revision: https://reviews.llvm.org/D111194
2021-10-06 11:10:36 -07:00
Vitaly Buka b5b1b3aef1 [sanitizer] Switch to StackDepotNode to 64bit hash
Now we can avoid scanning the stack on fast path.
The price is the false stack trace with probability of the hash collision.
This increase performance of lsan by 6% and pre-requirement for stack compression.

Depends on D111182.

Reviewed By: morehouse, dvyukov

Differential Revision: https://reviews.llvm.org/D111183
2021-10-06 10:45:11 -07:00
H.J. Lu fdf4c03522 [sanitizer] Support Intel CET
1. Include <cet.h> in sanitizer_common/sanitizer_asm.h to mark Intel CET
support when Intel CET is enabled.
2. Add _CET_ENDBR to function entries in assembly codes so that ENDBR
instruction will be generated when Intel CET is enabled.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D111185
2021-10-06 10:12:50 -07:00
David Spickett f8f1bb7462 [compiler-rt][lsan] Add backup AArch64 register for use_registers test
On Ubuntu Focal x13 is used by something in the process of calling
sched_yield. Causing the test to fail depending on when the thread
is stopped.

Adding x14 works around this and the test passes consistently.

Not switching to only x14 because that could make other platforms
fail. With both we'll always find at least one and even if both
values are present we'll only get one report.

Reviewed By: oontvoo, vitalybuka

Differential Revision: https://reviews.llvm.org/D110931
2021-10-06 09:45:53 +01:00
Dmitry Vyukov 24af1ba605 tsan: don't instrument runtime callbacks in tests
These runtime callbacks are supposed to be non-instrumented,
we can't handle runtime recursion well, nor can we afford
explicit recursion checks in the hot functions (memory access,
function entry/exit).
It used to work (not crash), but it won't work with the new runtime.
Mark all runtime callbacks as non-instrumented.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D111157
2021-10-06 08:44:04 +02:00
David Carlier 18a7ebda99 [Sanitizers] intercept md5 and sha* apis on FreeBSD.
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110989
2021-10-06 06:01:50 +01:00
Vitaly Buka 3129aa5caf [NFC][sanitizers] Add StackDepotBase Node::hash_type
Depends on D111177.

Differential Revision: https://reviews.llvm.org/D111182
2021-10-05 20:54:06 -07:00
Vitaly Buka 5ae9a3e4bf [NFC][sanitizer] Add MurMur2Hash64Builder
Depends on D111176.

Differential Revision: https://reviews.llvm.org/D111177
2021-10-05 20:54:06 -07:00
Vitaly Buka dc603b0e53 [NFC][sanitizer] Add basic hash test
Differential Revision: https://reviews.llvm.org/D111176
2021-10-05 20:54:06 -07:00
Zequan Wu 4e8efff53e [Profile] Add missing fflush in __llvm_profile_set_file_object 2021-10-05 15:41:51 -07:00
Vitaly Buka 84afd02525 [sanitizer] Fix Android bot
We don't need to check for equality, we need to check
that storage is large enough.
2021-10-05 13:08:16 -07:00
Vitaly Buka 6fab808f6f [NFC][sanitizer] Combine MSAN data in single field
Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D111118
2021-10-05 12:34:02 -07:00
Petr Hosek 24c615fa6b [InstrProfData] Bump the raw profile version to 8
This is to account for the change that made CountersPtr in __profd_
relative which landed in a1532ed275.
That change hasn't updated the raw profile version, and while the
profile layout stayed the same, profiles generated by tip-of-tree
LLVM are incompatible with 13.x tooling.

Differential Revision: https://reviews.llvm.org/D111123
2021-10-05 09:57:56 -07:00
Dmitry Vyukov c483140f3c tsan: improve detection of stack/tls races
Print meaningful stack frames for stack/tls races
(instead of PC 1/2 that don't symbolize).

Imitate stack/tls writes after we create and initialize
the new thread, otherwise the races are not detected.

This is re-submit of the following reverted commits,
but without tests as they failed on a number of OSes/arches:
"tsan: fix and test detection of TLS races"
"tsan: fix tls_race3 test on darwin"
"tsan: print a meaningful frame for stack races"

Differential Revision: https://reviews.llvm.org/D111147
2021-10-05 15:32:39 +02:00
Dmitry Vyukov a0ed71ff29 tsan: make cur_thread_init return cur_thread
Whenever we call cur_thread_init, we call cur_thread on the next line.
So make cur_thread_init return the current thread directly.
Makes code a bit shorter, does not affect codegen.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D110384
2021-10-05 15:24:52 +02:00
Leonard Chan 993555beb8 [compiler-rt][scudo] Check for failing prctl call
A bunch of MTE tests like ./ScudoUnitTest-aarch64-Test/MemtagTest.StoreTags
can fail on aarch64-linux if the kernel doesn't support the tagged address ABI. It looks like
the call to prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0) can return -1, which
casted to an unsigned int and masked will return a value not equal to
PR_MTE_TCF_NONE, meaning systemDetectsMemoryTagFaultsTestOnly can return an incorrect value.

This updates the check to account for a failing prctl call.

Differential Revision: https://reviews.llvm.org/D110888
2021-10-04 13:14:20 -07:00
Amy Kwan 83539d73f9 Fix msan/tests/msan_test.cpp due to -Wbitwise-instead-of-logical
The LE Power sanitizer bot fails when testing standalone compiler-rt due to
an MSAN test warning introduced by -Wbitwise-instead-of-logical. As this option
along with -Werror is enabled on the bot, the test failure occurs.
This patch updates msan_test.cpp to fix the warning introduced by the
-Wbitwise-instead-of-logical.
2021-10-04 12:58:12 -05:00
Hans Wennborg c7bd643599 [libFuzzer] Use octal instead of hex escape sequences in PrintASCII
Previously, PrintASCII would print the string "\ta" as "\x09a". However,
in C/C++ those strings are not the same: the trailing 'a' is part of the
escape sequence, which means it's equivalent to "\x9a". This is an
annoying quirk of the standard. (See
https://eel.is/c++draft/lex.ccon#nt:hexadecimal-escape-sequence)

To fix this, output three-digit octal escape sequences instead. Since
octal escapes are limited to max three digits, this avoids the problem
of subsequent characters unintentionally becoming part of the escape
sequence.

Dictionary files still use the non-C-compatible hex escapes, but I
believe we can't change the format since it comes from AFL, and
libfuzzer never writes such files, it only has to read them, so they're
not affected by this change.

Differential revision: https://reviews.llvm.org/D110920
2021-10-04 11:29:54 +02:00
Dan Liew d6a4294d13 Use standard separator for TSan options in `stress.cpp` test case.
Use of space as a separator for options is problematic for wrapper
scripts (i.e. implementations of `%run`) that have to marshall
environment variables to target different than the host.

Rather than requiring every implementation of `%run` to support spaces
in `TSAN_OPTIONS` it is simpler to fix this single test case.

rdar://83637067

Differential Revision: https://reviews.llvm.org/D110967
2021-10-02 21:11:18 -07:00
Amy Kwan 103c1bd118 Revert "tsan: fix and test detection of TLS races"
This reverts commit b4c1e5cb73.

Reverting this as it contains a test that is currently failing on the PPC BE bots.
2021-10-01 16:42:31 -05:00
Amy Kwan 8b1984bb8c Revert "tsan: fix tls_race3 test on darwin"
This reverts commit ade5023c54.

Reverting this commit as it is dependent on a test breaking the PPC BE bots.
2021-10-01 16:42:31 -05:00
Amy Kwan 2df1019576 Revert "tsan: print a meaningful frame for stack races"
This reverts commit ccc83ac7c5.

Reverting this commit as it is dependent on additional commits breaking the
PPC BE bots.
2021-10-01 16:42:30 -05:00
Zequan Wu ab694cd845 [Profile] Add a warning when lock file failed in __llvm_profile_set_file_object with continuous mode 2021-10-01 14:37:09 -07:00
ZijunZhao 0e8862901c revert tsan part for investigation 2021-10-01 18:52:36 +00:00
Hans Wennborg 369ac75853 [libFuzzer] Fix two typos 2021-10-01 13:15:43 +02:00
Teresa Johnson d047368149 [MemProf] Loosen matching of profile data to avoid bot flakes
Allow for the allocations to have migrated cpus, assuming they wouldn't
is causing some bot flakiness, e.g.:
https://lab.llvm.org/buildbot/#/builders/37/builds/7197
2021-09-30 21:22:40 -07:00
Vitaly Buka d2df5ce294 [NFC][asan] Remove redundant functions 2021-09-30 19:38:23 -07:00
Vitaly Buka 051d766bae [NFC][lsan] Expand use StackDepotReverseMap
Before StackDepotReverseMap was used only by ProcessPC.
2021-09-30 19:26:47 -07:00
Vitaly Buka 548aa9022e [NFC][sanitizer] Lazy init in StackDepotReverseMap 2021-09-30 19:26:34 -07:00
ZijunZhao 91bfccf837 add tsan shared library 2021-10-01 00:19:35 +00:00
Vitaly Buka 5c3568d01f [NFC][sanitizer] Add const into method 2021-09-30 17:16:34 -07:00
Teresa Johnson 0d8bdc1786 [MemProf] Record accesses for all words touched in mem intrinsic
Previously for mem* intrinsics we only incremented the access count for
the first word in the range. However, after thinking it through I think
it makes more sense to record an access for every word in the range.
This better matches the behavior of inlined memory intrinsics, and also
allows better analysis of utilization at a future date.

Differential Revision: https://reviews.llvm.org/D110799
2021-09-30 15:07:55 -07:00
Gwen Mittertreiner 72e7e15a12 [compiler-rt] Add -fno-omit-frame-pointer check to builtins
rG210d72e9d6b4a8e7633921d0bd7186fd3c7a2c8c moved the check from
builtin-config-ix to config-ix so that the check would be made even when
the builtins are not built. However, now the check is no longer made
when the builtins are built standalone which causes the builtins to fail
to build.

Add the check back to builtins-config-ix so that the check gets
performed both when the builtins are not built, and when they are built
standalone.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D110879
2021-09-30 13:53:13 -07:00
Andrew Browne d81723c99b [DFSan] Optimize code for writing to shadow. Move SetShadow to namespace.
Writing zeros to shadow (including checking for existing zero) is now ~2x
faster on one example.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D110733
2021-09-30 12:42:21 -07:00
Dan Liew cac25e4042 Adapt `tsan/flush_memory.cpp` to run on non-local platforms.
ad890aa232 landed a test without
using the `%run` prefix which means the test fails to run for
platforms that need it (e.g. iOS simulators).

This patch adds the `%run` prefix. While we're here also split
the single `RUN` line into two to make debugging easier.

rdar://83637296

Differential Revision: https://reviews.llvm.org/D110734
2021-09-29 10:39:38 -07:00
Lang Hames 1380eae590 [ORC-RT] Add target dependencies to ORC-RT regression tests.
check-orc-rt had no cmake target dependency on orc or llvm-jitlink, which
could lead to regression test failures in compiler-rt. This patch should
fix the issue.

Patch by Jack Andersen (jackoalan@gmail.com). Thanks Jack!

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D110659
2021-09-28 22:08:47 -07:00
Vitaly Buka 7c1128f3bb [NFC][sanitizer] Return StackDepotStats by value
Differential Revision: https://reviews.llvm.org/D110644
2021-09-28 15:42:21 -07:00
Leonard Chan b9f547e8e5 [llvm][profile] Add padding after binary IDs
Some tests with binary IDs would fail with error: no profile can be merged.
This is because raw profiles could have unaligned headers when emitting binary
IDs. This means padding should be emitted after binary IDs are emitted to
ensure everything else is aligned. This patch adds padding after each binary ID
to ensure the next binary ID size is 8-byte aligned. This also adds extra
checks to ensure we aren't reading corrupted data when printing binary IDs.

Differential Revision: https://reviews.llvm.org/D110365
2021-09-28 11:50:50 -07:00
Vitaly Buka bfa50250b6 [NFC][sanitizer] Clang-format some code 2021-09-28 11:10:15 -07:00
Dmitry Vyukov ccc83ac7c5 tsan: print a meaningful frame for stack races
Depends on D110631.

Differential Revision: https://reviews.llvm.org/D110632
2021-09-28 17:08:00 +02:00
Dmitry Vyukov ade5023c54 tsan: fix tls_race3 test on darwin
Darwin also needs to use __tsan_tls_initialization
to pass the test.

Differential Revision: https://reviews.llvm.org/D110631
2021-09-28 17:07:51 +02:00
Dmitry Vyukov f3932ae1a0 tsan: fix cur_thread alignment
Commit 354ded67b3 ("tsan: align ThreadState to cache line")
did an incomplete thing. It marked ThreadState as cache line
aligned, but the thread local ThreadState instance is declared
as an aligned char array with hard-coded 64-byte alignment.
On PowerPC cache line size is 128 bytes, so the hard-coded
64-byte alignment is not enough.
Use cache line alignment consistently.

Differential Revision: https://reviews.llvm.org/D110629
2021-09-28 16:49:44 +02:00
Ahsan Saghir 4f6a6ba126 Revert "tsan: fix trace tests on darwin"
This reverts commit 94ea36649e.

Reverting due to errors on buildbots.
2021-09-27 20:17:17 -05:00
Kevin Athey b345952ad4 Revert "tsan: add a test for stack init race"
This reverts commit b72176b9bc.

Broke bot: https://lab.llvm.org/buildbot/#/builders/70/builds/12193
2021-09-27 15:31:23 -07:00
Kostya Kortchinsky 04f5913395 [gwp-asan] Initialize AllocatorVersionMagic at runtime
GWP-ASan's `AllocatorState` was recently extended with a
`AllocatorVersionMagic` structure required so that GWP-ASan bug reports
can be understood by tools at different versions.

On Fuchsia, this in included in the `scudo::Allocator` structure, and
by having non-zero initializers, this effectively moved the static
allocator structure from the `.bss` segment to the `.data` segment, thus
increasing (significantly) the size of the libc.

This CL proposes to initialize the structure with its magic numbers at
runtime, allowing for the allocator to go back into the `.bss` segment.

I will work on adding a test on the Scudo side to ensure that this type
of changes get detected early on. Additional work is also needed to
reduce the footprint of the (large) memory-tagging related structures
that are currently part of the allocator.

Differential Revision: https://reviews.llvm.org/D110575
2021-09-27 13:49:55 -07:00
Dmitry Vyukov 94ea36649e tsan: fix trace tests on darwin
The trace tests crashed on darwin because of some thread
initialization issues (thread initialization is somewhat
different on darwin).
Instead of starting real threads, create a new ThreadState
in the main thread. This makes the tests more unit-testy
and hopefully won't crash on darwin (there is almost no
platform-specific code involved now).
This will also help with future trace tests that will need
more than 1 thread. Creating more than 1 real thread and
dispatching test actions across multiple threads in the
required deterministic order is painful.

Depends on D110539.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110546
2021-09-27 16:40:57 +02:00
Dmitry Vyukov b72176b9bc tsan: add a test for stack init race
Depends on D110538.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110539
2021-09-27 16:40:17 +02:00
Dmitry Vyukov b4c1e5cb73 tsan: fix and test detection of TLS races
Currently detection of races with TLS/stack initialization
is broken because we imitate the write before thread initialization,
so it's modelled with a wrong thread/epoch.
Fix that and add a test.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110538
2021-09-27 16:40:08 +02:00
Dmitry Vyukov 1455b552b7 tsan: de-hardcode MemCount const
Use MemCount instead of hard-coded value 7.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110532
2021-09-27 16:11:49 +02:00
Dmitry Vyukov 354ded67b3 tsan: align ThreadState to cache line
There are 2 reasons to do this:
1. We place hot data in the first cache line of ThreadState,
this assumed that it's cache-line-aligned but we never actually
enforced it (or it was lost at some point).
2. The new vector clock uses vector instructions and requires
data alignment. Later the new vector clock will be embedded in
ThreadState, then ensuring vector clock alignment will be
impossible w/o ThreadState alignment.

Depends on D110519.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110520
2021-09-27 12:54:09 +02:00
Dmitry Vyukov ed7f3f5bc9 tsan: move shadow stack into ThreadState
Currently the shadow stack is located in the trace memory mapping.
The new tsan runtime will remove the trace memory mapping.
Move the shadow stack into ThreadState as a preparation step.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110519
2021-09-27 12:53:02 +02:00
Lang Hames 897eb579c6 [ORC-RT] ExecutorAddrDiff ergonomic improvements; contains and overlaps methods
Renames StartAddress and EndAddress members to Start and End.

Adds contains and overlap methods.

Adds a constructor from an address and size.

These changes are counterparts to LLVM commits ef391df2b6, c0d889995e, and
37f1b7a3f3.
2021-09-25 12:01:42 -07:00
Dmitry Vyukov b02938439d tsan: uninline RacyStacks::operator==
It's only used during race reporting.
There is no point in polluting the main header file with it.

Reviewed By: xgupta

Differential Revision: https://reviews.llvm.org/D110470
2021-09-25 12:08:51 +02:00
Petr Hosek d893692024 [CMake] Pass through CMAKE_READELF to subbuilds
This matches handling of other CMake variables.

Differential Revision: https://reviews.llvm.org/D110463
2021-09-24 18:20:30 -07:00
Leonard Chan f8da95cd7e [compiler-rt] Add shared_cxxabi requirement to some tests
This adds REQUIRES: shared_cxxabi to a bunch of tests that would fail if this
weak reference in sanitizer common was undefined. This is necessary in cases
where libc++abi.a is statically linked in. Because there is no strong reference
to __cxa_demangle in compiler-rt, then if libc++abi is linked in via a static
archive, then the linker will not extract the archive member that would define
that weak symbol. This causes a handful of tests to fail because this leads to
the symbolizer printing mangled symbols where tests expect them demangled.

Technically, this feature is WAI since sanitizer runtimes shouldn't fail if
this symbol isn't resolved, and linking statically means you wouldn't need to
link in all of libc++abi. As a workaround, we can simply make it a requirement
that these tests use shared libc++abis.

Differential Revision: https://reviews.llvm.org/D109639
2021-09-24 11:51:26 -07:00
Dmitry Vyukov 34412cea5c tsan: don't use pipe2 in tests
MacOS buildbots failed:
stress.cpp:57:7: error: use of undeclared identifier 'pipe2'
https://green.lab.llvm.org/green//job/clang-stage1-RA/24209/consoleFull#-3468768778254eaf0-7326-4999-85b0-388101f2d404

Fix the test to not use pipe2.

Differential Revision: https://reviews.llvm.org/D110423
2021-09-24 17:52:16 +02:00
Dmitry Vyukov 124fcd7e9d tsan: add a stress test
The stress test does various assorted things
(memory accesses, function calls, atomic operations,
thread creation/join, intercepted libc calls)
in multiple threads just to stress various parts
of the runtime.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110416
2021-09-24 16:46:13 +02:00
Dmitry Vyukov ad890aa232 tsan: add a test for flushing memory
Add a test for __tsan_flush_memory() and for background
flushing of the runtime memory.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110409
2021-09-24 15:59:19 +02:00
Frederic Cambus 626e2a6c62 [compiler-rt] Use portable "#!/usr/bin/env bash" shebang for tests.
In build_symbolizer.sh we can safely remove the -eu argument from the shebang (which is an unportable construct), as the scripts sets **-e** and **-u** already.

Differential Revision: https://reviews.llvm.org/D110039
2021-09-24 19:10:07 +05:30
Lang Hames 0820fbab99 [ORC-RT] Rename ExecutorAddress to ExecutorAddr.
This is an ORC-runtime counterpart to LLVM commit ef391df2b6, and the
motivation is the same: to move to a shorter name to improve the ergonomics of
this type before it's more widely adopted.
2021-09-23 21:42:23 -07:00
David Carlier 3675e147a1 [Sanitizers] intercept ttyent api on FreeBSD.
and ttyentpath separately on NetBSD.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D109843
2021-09-24 04:26:05 +01:00
Leonard Chan c579c658cd [compiler-rt][profile] Make corrupted-profile.c more robust
This test specifically checks that profiles are not mergeable if there's a
change in the CounterPtr in the profile header. The test manually changes
CounterPtr by explicitly calling memset on some offset into the profile file.
This test would fail if binary IDs were emitted because the offset calculation
does not take into account the binary ID sizes.

This patch updates the test to use types provided in profile/InstrProfData.inc
to make it more resistant to profile layout changes.

Differential Revision: https://reviews.llvm.org/D110277
2021-09-23 17:16:47 -07:00
Leonard Chan eb115aa6c8 Revert "[compiler-rt][profile] Add padding after binary IDs"
This reverts commit 6bc9c8dfe3.

Reverted because this broke some PPC buildbots.
2021-09-23 15:48:59 -07:00
Roland McGrath 80b92db02c [profile][fuchsia] Don't include extra NUL in log messages
Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D110361
2021-09-23 15:16:31 -07:00
Leonard Chan 6bc9c8dfe3 [compiler-rt][profile] Add padding after binary IDs
Some tests with binary IDs would fail with error: no profile can be merged.
This is because raw profiles could have unaligned headers when emitting binary
IDs. This means padding should be emitted after binary IDs are emitted to
ensure everything else is aligned. This patch accounts for that padding in
__llvm_write_binary_ids.

Differential Revision: https://reviews.llvm.org/D110188
2021-09-23 10:29:24 -07:00
Dmitry Vyukov 7faf1285f2 tsan: remove expected race leftover
Remove nmissed_expected variable.
It's a leftover from removed "expected race" feature and is never incremented.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110321
2021-09-23 14:13:26 +02:00
Dmitry Vyukov 702cb7afe9 tsan: move shadow declaration into a separate header file (NFC)
tsan_rtl.h is very huge and contains too many things.
Move FastState and Shadow types into a new tsan_shadow.h file.
This also allows to use FastState/Shadow in other header files
without creating circular dependencies (which most likely will
happen today).

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110272
2021-09-23 11:04:43 +02:00
Dmitry Vyukov 20d3e51801 tsan: use RawShadow instead of u64 more
Fix few remaining cases where we use u64 instead of the new RawShadow type.

Depends on D110265.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110266
2021-09-23 10:46:07 +02:00
Dmitry Vyukov dcc6db22d8 tsan: add another deep stack test
Add a test for a trace corner case that lead to a bug
in experimental runtime replacement.
Since it passes with the current runtime it makes sense
to submit it on its own.

Depends on D110264.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110265
2021-09-23 10:45:17 +02:00
Dmitry Vyukov 356ecd9bd1 tsan: remove DontDumpShadow from Go build
DontDumpShadow is used only in InitializeShadowMemory which is Go-only.

Depends on D110263.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110264
2021-09-23 10:45:05 +02:00
Dmitry Vyukov af8b14c278 tsan: remove unnecessary enum values (NFC)
Remove unnecessary enum values in the memory profiler.
There is no point in spelling them, it can only lead to bugs
and larger diffs when values are added/removed.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110263
2021-09-23 10:43:58 +02:00
Leonard Chan ac191bcc99 [compiler-rt][test] Add REQUIRES for checking static libc++abi
intercept-rethrow-exception.cc fails when running runtimes tests if linking in
a hermetic libc++abi. This is because if libc++abi is used, then asan expects
to intercept __cxa_rethrow_primary_exception on linux, which should unpoison the
stack. If we statically link in libc++abi though, it will contain a strong
definition for __cxa_rethrow_primary_exception which wins over the weakly
defined interceptor provided by asan, causing the test to fail by not unpoisoning
the stack on the exception being thrown.

It's likely no one has encountered this before and possible that upstream tests
opt for dynamically linking where the interceptor can work properly. An ideal
long term solution would be to update the interceptor and libc++[abi] APIs to
work for this case, but that will likely take a long time to work out. In the
meantime, since the test isn't necessarily broken, we can just add another
REQUIRES check to make sure that it's only run if we aren't statically linking
in libc++abi.

Differential Revision: https://reviews.llvm.org/D109938
2021-09-22 15:25:05 -07:00
Matt Morehouse 1aedf77ece [HWASan] Use a single .weak binding in asm.
Specifying .global and .weak causes a compiler warning:

  warning: __sigsetjmp changed binding to STB_WEAK

Specifying only .weak should have the same effect without causing a
warning.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D110178
2021-09-22 09:35:09 -07:00
Teresa Johnson 1864976c96 [Sanitizer] Add Windows header for _mkdir
This will hopefully fix the sanitizer_windows bot failure after D109794:
https://lab.llvm.org/buildbot/#/builders/127/builds/17222
2021-09-22 08:05:43 -07:00
Dmitry Vyukov 0ee77d6db3 tsan: write uptime in mem profile
Write uptime in real time seconds for every mem profile record.
Uptime is useful to make more sense out of the profile,
compare random lines, etc.

Depends on D110153.

Reviewed By: melver, vitalybuka

Differential Revision: https://reviews.llvm.org/D110154
2021-09-22 10:19:58 +02:00
Dmitry Vyukov ae6d57ca5a tsan: remove stale comment
We do query it every 100ms now.
(GetRSS was fixed to not be dead slow IIRC)

Depends on D110152.

Reviewed By: melver, vitalybuka

Differential Revision: https://reviews.llvm.org/D110153
2021-09-22 10:18:58 +02:00
Dmitry Vyukov e8101f2149 tsan: move mem profile initialization into separate function
BackgroundThread function is quite large,
move mem profile initialization into a separate function.

Depends on D110151.

Reviewed By: melver, vitalybuka

Differential Revision: https://reviews.llvm.org/D110152
2021-09-22 10:18:08 +02:00
Dmitry Vyukov b8aa9b0c37 tsan: include internal allocator info in mem profile
We allocate things from the internal allocator,
it's useful to know how much it consumes.

Depends on D110150.

Reviewed By: melver, vitalybuka

Differential Revision: https://reviews.llvm.org/D110151
2021-09-22 10:17:01 +02:00
Dmitry Vyukov 58a157cd3b tsan: make mem profile data more consistent
We currently query number of threads before reading /proc/self/smaps.
But reading /proc/self/smaps can take lots of time for huge processes
and it's retries several times with different buffer sizes.
Overall it can take tens of seconds. This can make number of threads
significantly inconsistent with the rest of the stats.
So query it after reading /proc/self/smaps.

Depends on D110149.

Reviewed By: melver, vitalybuka

Differential Revision: https://reviews.llvm.org/D110150
2021-09-22 10:16:15 +02:00
Dmitry Vyukov eefef56ece tsan: include MBlock/SyncObj stats into mem profile
Include info about MBlock/SyncObj memory consumption in the memory profile.

Depends on D110148.

Reviewed By: melver, vitalybuka

Differential Revision: https://reviews.llvm.org/D110149
2021-09-22 10:14:33 +02:00
Dmitry Vyukov 608ffc98c3 tsan: account for mid app range in mem profile
We account low and high ranges, but forgot abount the mid range.
Account mid range as well.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110148
2021-09-22 10:13:31 +02:00
Dmitry Vyukov 4986959eb2 tsan: prepare for trace mapping removal
Don't test for presence of the trace mapping,
it will be removed soon.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110194
2021-09-22 07:26:37 +02:00
Dmitry Vyukov 82e593cf90 tsan: uninline Enable/DisableIgnores
ScopedInterceptor::Enable/DisableIgnores is only used for some special cases.
Unline them from the common interceptor handling.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110157
2021-09-22 07:25:14 +02:00
Dmitry Vyukov db2f870fe3 tsan: reset destination range in Java heap move
Switch Java heap move to the new scheme required for the new tsan runtime.
Instead of copying the shadow we reset the destination range.
The new v3 trace contains addresses of accesses, so we cannot simply copy the shadow.
This can lead to false negatives, but cannot lead to false positives.

Depends on D110159.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110190
2021-09-22 07:23:21 +02:00
Dmitry Vyukov 41f8ef3e31 tsan: enable sse4.2 in tests
Pass -msse4.2 flag to the tests the same way we do for the runtime.
Layout of some structs in the runtime headers depends on the flag
(TSAN_VECTORIZE), so we need it to be consistent across the runtime
and tests.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110192
2021-09-22 07:13:47 +02:00
Dmitry Vyukov cf93f7677d tsan: move errno spoiling reporting into a separate function (NFC)
CallUserSignalHandler function is quite large and complex.
Move errno spoiling reporting into a separate function.
No logical changes.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110159
2021-09-22 07:12:53 +02:00
Dmitry Vyukov 20ee72d4cc tsan: don't call dlsym during exit
dlsym calls into dynamic linker which calls malloc and other things.
It's problematic to do it during the actual exit, because
it can happen from a singal handler or from within the runtime
after we reported the first bug, etc.
See https://github.com/google/sanitizers/issues/1440 for an example
(captured in the added test).
Initialize the callbacks during startup instead.

Depends on D110159.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D110166
2021-09-22 07:11:59 +02:00
Teresa Johnson 7acd1807dd [Sanitizer] Modify test to avoid bot timeouts
Change the mutex type to one that initializes on construction and
hopefully avoid what appear to be deadlock failures in the new test
on a couple bots, e.g.:

https://green.lab.llvm.org/green/job/clang-stage1-RA/24140/testReport/SanitizerCommon-Unit/__Sanitizer-x86_64-Test/SanitizerCommon_ReportFile/
2021-09-21 18:47:16 -07:00
Teresa Johnson 56dec4be9b [Sanitizer] Allow setting the report path to create directory
When setting the report path, recursively create the directory as
needed. This brings the profile path support for memprof on par with
normal PGO. The code was largely cloned from __llvm_profile_recursive_mkdir
in compiler-rt/lib/profile/InstrProfilingUtil.c.

Differential Revision: https://reviews.llvm.org/D109794
2021-09-21 16:42:42 -07:00
Toshihito Kikuchi 22ea0cea59 [compiler-rt] [windows] Add more assembly patterns for interception
To intercept the functions in Win11's ntdll.dll, we need to use the trampoline
technique because there are bytes other than 0x90 or 0xcc in the gaps between
exported functions.  This patch adds more patterns that appear in ntdll's
functions.

Bug: https://bugs.llvm.org/show_bug.cgi?id=51721

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D109941
2021-09-21 15:51:58 -07:00
Kostya Serebryany 11c533e1ea [sanitizer coverage] write the pc-table at the process exit
The current code writes the pc-table at the process startup,
which may happen before the common_flags() are initialized.
Move writing to the process end.
This is consistent with how we write the counters and avoids the problem with the uninitalized flags.
Add prints if verbosity>=1.

Reviewed By: kostik

Differential Revision: https://reviews.llvm.org/D110119
2021-09-21 09:09:25 -07:00
Dmitry Vyukov 9d7b7350c9 tsan: simplify thread context setting
Currently we set thr->tctx after OnStarted callback
taking thread registry mutex again and searching for the context.
But OnStarted already runs under the thread registry mutex
and has access to the context, so set it in the OnStarted.
This makes code simpler and faster.

Depends on D110132.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110133
2021-09-21 13:26:55 +02:00
Dmitry Vyukov 908256b0ea tsan: rearrange thread state callbacks (NFC)
Thread state functions are split into 2 parts:
tsan entry function (e.g. ThreadStart) and thread registry
state change callback (e.g. OnStart). Currently these
pairs of functions are located far from each other and
in reverse order. This makes it hard to read and follow the logic.
Reorder the code so that OnFoo directly follows ThreadFoo.
No other code changes.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110132
2021-09-21 13:26:36 +02:00
Dmitry Vyukov 6fe35ef419 tsan: fix debug format strings
Some of the DPrintf's currently produce -Wformat warnings if enabled.
Fix these format strings.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D110131
2021-09-21 13:23:10 +02:00
Andrew Browne c533b88a6d [DFSan] Add force_zero_label abilist option to DFSan. This can be used as a work-around for overtainting.
Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D109847
2021-09-17 12:57:40 -07:00
Jeremie Courreges-Anglas 3db9590605 [compiler-rt] Implement __clear_cache() on OpenBSD/riscv64 2021-09-17 15:42:05 -04:00
Matt Morehouse 750d5fc65c [HWASan] Intercept setjmp/longjmp on x86_64.
Reviewed By: xiangzhangllvm

Differential Revision: https://reviews.llvm.org/D109790
2021-09-17 07:10:57 -07:00
Leonard Chan f4092c7690 [compiler-rt][test] Ensure CMAKE_SYSROOT is added as a test cflag if provided
When running tests like SanitizerCommon-asan-x86_64-Linux :: Linux/crypt_r.cpp,
it may attempt to use the host header crypt.h rather than a sysroot header.
This is significant in the event where struct crypt_data defined on host is
different from the sysroot used to make the sanitizer runtime libraries. This
can result in logical differences between the expected size/layout of struct
crypt_data known by sanitizers and the strict crypt_data provided by the host crypt.h.

Since tests should still use the CMAKE_SYSROOT, this ensures that CMAKE_SYSROOT
is propagated to compiler-rt tests.

Differential Revision: https://reviews.llvm.org/D109796
2021-09-16 17:14:14 -07:00
Leonard Chan 47373f94a4 [compiler-rt][test] Add int128 requirement to TestCases/Misc/Linux/static-link.cpp
We hit some undefined symbol errors to 128-bit floating point functions when linking this test.

ld.lld: error: undefined symbol: __multf3
>>> referenced by strtof128_l.o:(round_and_return) in archive /usr/lib/x86_64-linux-gnu/libc.a
>>> referenced by strtof128_l.o:(round_and_return) in archive /usr/lib/x86_64-linux-gnu/libc.a
>>> referenced by strtof128_l.o:(round_and_return) in archive /usr/lib/x86_64-linux-gnu/libc.a
>>> referenced 4 more times
>>> did you mean: __muldf3
>>> defined in: /usr/local/google/home/leonardchan/llvm-monorepo/llvm-build-1-master-fuchsia-toolchain/lib/clang/14.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a

Host libc expects these to be defined, and compiler-rt will only define these
for certain platforms (see definition for CRT_LDBL_128BIT). Since we likely
can't do anything about the host libc, we can at least restrict the test to
check that these functions are supported.

Differential Revision: https://reviews.llvm.org/D109709
2021-09-16 17:04:55 -07:00
Mitch Phillips 65feb9f444 [symbolizer] Change libcxx paths in buildscript.
D107799 changed the paths from lib/libcxx(abi)?.a to
lib/<triple>/libcxx(abi)?.a. The build script needs to know to pick up
the files from the triple subfolder instead.

See https://lab.llvm.org/buildbot/#/builders/37/builds/6764 for buildbot log failure.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D109924
2021-09-16 15:11:09 -07:00
Aaron Green 5de8c7f138 [fuzzer][fuchsia] Close exception channel before exiting.
On Fuchsia, killing or exiting a process that has a thread listening to its own process's debugger exception channel can hang. Zircon may kill all the threads, send a synthetic exceptions to debugger, and wait for the debugger to have received them. This means the thread listening to the debug exception channel may be killed even as Zircon is waiting for that thread to drain the exception channel, and the process can become stuck in a half-dead state.

This situation is "weird" as it only arises when a process is trying to debug itself. Unfortunately, this is exactly the scenario for libFuzzer on Fuchsia: FuzzerUtilFuchsia spawns a crash-handling thread that acts like a debugger in order to be able to rewrite the crashed threads stack and resume them into libFuzzer's usual POSIX signal handlers. In practice, approximately 25% of fuzzers appear to hang on exit, after generating output and artifacts. These processes hang around until the platform is torn done, which is typically a ClusterFuzz VM. Thus, real-world impact has been somewhat mitigated. The issue should still be resolved for local users, though.

This change improves the behavior of exit() in libFuzzer by adding an atexit handler which closes an event shared with the crash handling thread. This signals to the crash handler that it should close the exception channel and be joined before the process actually exits.

Reviewed By: charco

Differential Revision: https://reviews.llvm.org/D109258
2021-09-16 11:57:12 -07:00
Alfonso Gregory a2c319fdc6 [LLVM][CMake][NFC] Resolve FIXME: Rename LLVM_CMAKE_PATH to LLVM_CMAKE_DIR throughout the project
This way, we do not need to set LLVM_CMAKE_PATH to LLVM_CMAKE_DIR when (NOT LLVM_CONFIG_FOUND)

Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D107717
2021-09-16 18:29:57 +02:00
Florian Mayer 95ba9f957a [hwasan] print globals in symbolizer-friendly format.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D109698
2021-09-16 13:29:17 +01:00
Teresa Johnson 242524741f [MemProf] Fix format warnings
Fix the warnings that show up with -Wformat in this file.
2021-09-15 16:27:21 -07:00
Teresa Johnson 37a5a3ae55 [MemProf] Avoid global lock when updating MIB cache
Previously we used a global Allocator-scope mutex to lock when adding a
deallocation to the MIB cache. This resulted in a lot of contention.
Instead add and use per-set mutexes.

Along with this, we now need to remove the global miss and access count
variables and instead utilize the per-set statistics to report the
overall miss rate.

Differential Revision: https://reviews.llvm.org/D109853
2021-09-15 16:06:11 -07:00
Dan Liew f4382d4b09 [Compiler-RT] For arm64e test suites use the SDK version as the minimum deployment target.
Previously we used the minimum deployment target used for the platform
(e.g. iOS is 9.0). Unfortunately this leads to ABI incompatibilities with
arm64e devices running newer OSs. In particular the following TSan test
cases that used libcxx would fail due to the ABI mismatch.

* Darwin/libcxx-shared-ptr-recursive.mm
* Darwin/libcxx-shared-ptr-stress.mm
* Darwin/libcxx-shared-ptr.mm
* libcxx/std_shared_ptr.cpp

Given that arm64e is not ABI stable we should ideally match the
deployment target for sanitizer runtimes and their tests cases to the
device when building for arm64e. Unfortunately having a mixed deployment
target (based on architecture) isn't currently supported by the build system
and is non-trivial to implement.

As a stop-gap measure this patch changes the sanitizer test suites (but not the
sanitizer runtimes themselves) to use a newer deployment target when
targetting arm64e.

The deployment target used for arm64e is the SDK version because this
"should" match the OS version running on the target device (it is a
configuration error to not match them).

rdar://83080611
2021-09-15 14:22:25 -07:00
Matt Morehouse acaca56471 [HWASan] Test longjmp(jmpbuf, 0).
Our interceptor needs to change retval to 1 when 0 is passed.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D109788
2021-09-15 12:11:59 -07:00
Michał Górny 210d72e9d6 [compiler-rt] Move -fno-omit-frame-pointer check to common config-ix
9ee64c3746 has started using
COMPILER_RT_HAS_OMIT_FRAME_POINTER_FLAG inside scudo.  However,
the relevant CMake check was performed in builtin-config-ix.cmake,
so the definition was missing when builtins were not built.  Move
the check to config-ix.cmake, so that it runs unconditionally of
the components being built.

Fixes PR#51847

Differential Revision: https://reviews.llvm.org/D109812
2021-09-15 18:32:33 +02:00
Fangrui Song 9111635cb7 [test] Fix asan/scudo -shared-libsan tests with -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=on
On x86_64-unknown-linux-gnu, `-m32` tests set LD_LIBRARY_PATH to
`config.compiler_rt_libdir` (`$build/lib/clang/14.0.0/lib/x86_64-unknown-linux-gnu`)
instead of i386-unknown-linux-gnu, so `-shared-libsan` executables
cannot find their runtime (e.g. `TestCases/replaceable_new_delete.cpp`).

Detect -m32 and -m64 in config.target_cflags, and adjust `config.compiler_rt_libdir`.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D108859
2021-09-15 09:07:47 -07:00
Matt Morehouse 0a07789fe9 [HWASan] Add missing newlines. 2021-09-15 09:06:01 -07:00
Matt Morehouse 1a3b3301d7 [HWASan] Catch cases where libc populated jmp_buf.
Some setjmp calls within libc cannot be intercepted while their matching
longjmp calls can be.  This causes problems if our setjmp/longjmp
interceptors don't use the exact same format as libc for populating and
reading the jmp_buf.

We add a magic field to our jmp_buf and populate it in setjmp.  This
allows our longjmp interceptor to notice when a libc jmp_buf is passed
to it.

See discussion on https://reviews.llvm.org/D109699 and
https://reviews.llvm.org/D69045.

Fixes https://github.com/google/sanitizers/issues/1244.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D109787
2021-09-15 07:53:54 -07:00
Jessica Clarke b8d83e83be [RISCV][compiler-rt] Fix an incorrect comment for RV64 __riscv_restore_12
This was presumably copied from the RV32 implementation and not updated
like the rest.
2021-09-15 15:25:59 +01:00
Jessica Clarke bbca392a7f [RISCV][compiler-rt] Move RV64 __riscv_restore_1/0 directives next to labels
This looks like it was copied from the RV32 version and not properly
updated. This has no functional effect but is not good style.
2021-09-15 14:42:22 +01:00
Jessica Clarke 3c885190af [RISCV][compiler-rt] Add missing __riscv_save_1/0 labels for RV64
These got missed in D91717.
2021-09-15 14:42:16 +01:00
Fangrui Song aaf62958f1 [CMake] Delete obsoleted COMPILER_RT_TEST_TARGET_TRIPLE
The last user has been removed from llvm-zorg for Android.
2021-09-14 17:46:50 -07:00
David Carlier fa7c01e578 [Sanitizers] intercept fparseln on freebsd
Reviewed By: vitalybuka, emaste

Differential Revision: https://reviews.llvm.org/D109753
2021-09-14 19:13:45 +01:00
Nico Weber 8cfab5de13 [Windows build] Use "DIA SDK" in sysroot
This updates llvm/utils/sysroot.py to include the "DIA SDK" folder in the
sysroot.

It also updates the build to look for the DIA SDK there if a sysroot is set.

This requires moving LLVM_WINSYSROOT to config-ix.cmake.

For the GN build, I chose to pass a qualified path to diaguids in libs instead
of pushing a config with a `/libpath:` flag. The former requires a GN with
https://gn-review.googlesource.com/c/gn/+/12200, the latter requires D109624.
The former is more like the cmake build, arguably a bit simpler, and it's
easier to check for the wrong GN revision and easier to update GN.

Differential Revision: https://reviews.llvm.org/D109708
2021-09-14 08:59:28 -04:00
David Carlier dc08cfae09 [Sanitizers] intercept accept4 on freebsd 2021-09-13 21:15:07 +01:00
David Carlier b8f6c1fd96 [Sanitizers] intercept getgrouplist on Linux and FreeBSD.
Reviewed By: vitalyb

Differential Revision: https://reviews.llvm.org/D109641
2021-09-13 18:42:43 +01:00
luxufan ff6069b891 [JITLink] Add initial native TLS support to ELFNix platform
This patch use the same way as the https://reviews.llvm.org/rGfe1fa43f16beac1506a2e73a9f7b3c81179744eb to handle the thread local variable.

It allocates 2 * pointerSize space in GOT to represent the thread key and data address. Instead of using the _tls_get_addr function, I customed a function __orc_rt_elfnix_tls_get_addr to get the address of thread local varible. Currently, this is a wip patch, only one TLS relocation R_X86_64_TLSGD is supported and I need to add the corresponding test cases.

To allocate the TLS  descriptor in GOT, I need to get the edge kind information in PerGraphGOTAndPLTStubBuilder, So I add a `Edge::Kind K` argument in some functions in PerGraphGOTAndPLTStubBuilder.h. If it is not suitable, I can think further to solve this problem.

Differential Revision: https://reviews.llvm.org/D109293
2021-09-13 14:35:49 +08:00
David Carlier 8fdd821aef [Sanitizers] intercept netent, protoent and mincore on FreeBSD.
netent on Linux in addition as well.

Reviewd By: vitalybuka

Differential Revision: https://reviews.llvm.org/D109287
2021-09-10 19:24:51 +01:00
Julian Lettner ee0ac45672 [TSan][Darwin] Add integration tests for dyld interposition
Add integration tests for dyld interposition: DYLD_LIBRARY_PATH and
DYLD_INSERT_LIBRARIES.

DYLD_INSERT_LIBRARIES is also relevant for TSan thread
finalization/destruction sequence in the presence of additional pthread
introspection hooks (libBacktraceRecording.dylib for Xcode 'Queue
Debugging' feature).

rdar://78739125

Differential Revision: https://reviews.llvm.org/D109332
2021-09-09 22:33:59 +02:00
Steven Wu 05eaa2b42f [CMake][Darwin] Ignore stderr during SDKSetting.plist parsing
Don't print stderr to commandline when configuring compiler-rt for
darwin platforms. NFC.

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D108156
2021-09-09 12:09:29 -07:00
Florian Mayer 039fd9af45 [NFC] [hwasan] move prints closer together.
this makes the code slightly more readable.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D109442
2021-09-09 13:39:11 +01:00
Tom Stellard 9ee64c3746 scudo: Only add no-omit-frame-pointer flags when the compiler supports them
Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D109196
2021-09-08 21:10:40 -07:00
Matt Morehouse ff77c4eac7 [libFuzzer] Add missing argument to CrashResistantMerge.
Fixes a build error caused by a bad merge conflict resolution for
https://reviews.llvm.org/D105084.
2021-09-08 11:48:33 -07:00
Leonard Chan e170cf5061 [compiler-rt][fuzzer] Do not link in libc++ in tests and disable exceptions
Differential Revision: https://reviews.llvm.org/D109208
2021-09-08 11:45:52 -07:00
Leonard Chan 056d024235 [compiler-rt] Use COMPILER_RT_TEST_CXX_COMPILER for linking compiler-rt tests
Before, COMPILER_RT_TEST_COMPILER was used which pointed to a C compiler. While
it is incorrect to assume either of these is the default compiler, using the
C++ one allows for linking cpp tests.

Differential Revision: https://reviews.llvm.org/D109207
2021-09-08 11:44:00 -07:00
Dan Liew 71841146b6 Fix `asan/TestCases/Darwin/scrible.cpp` to work on platforms where `long` is not 64-bits.
Previously the test was failing on platforms where `long` was less than
64-bits wide (e.g. older WatchOS simulators and arm64_32) because the
`padding` field was too small.

The test currently relies on the `my_object->isa` being scribbled or
left unmodified after `my_object` is freed. However, this was not the
case because the `isa` pointer intersected with
`ChunkHeader::free_context_id`.  `free_context_id` starts at the
beginning of user memory but it only initialized once the memory is
freed. This caused the `isa` pointer to change after it was freed
leading to the test crashing.

To fix this the `padding` field has been made explicitly 64-bits wide
(same size as `ChunkHeader::free_context_id`).

rdar://75806757

Differential Revision: https://reviews.llvm.org/D109409
2021-09-08 09:53:09 -07:00
gtt1995 a30dbbe924 Redistribute energy for Corpus
I found that the initial corpus allocation of fork mode has certain defects.
I designed a new initial corpus allocation strategy based on size grouping.
This method can give more energy to the small seeds in the corpus and
increase the throughput of the test.

Fuzzbench data (glibfuzzer is -fork_corpus_groups=1):
https://www.fuzzbench.com/reports/experimental/2021-08-05-parallel/index.html

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D105084
2021-09-08 09:22:45 -07:00
aristotelis e6597dbae8 Greedy set cover implementation of `Merger::Merge`
Extend the existing single-pass algorithm for `Merger::Merge` with an algorithm that gives better results. This new implementation can be used with a new **set_cover_merge=1** flag.

This greedy set cover implementation gives a substantially smaller final corpus (40%-80% less testcases) while preserving the same features/coverage. At the same time, the execution time penalty is not that significant (+50% for ~1M corpus files and far less for smaller corpora). These results were obtained by comparing several targets with varying size corpora.

Change `Merger::CrashResistantMergeInternalStep` to collect all features from each file and not just unique ones. This is needed for the set cover algorithm to work correctly. The implementation of the algorithm in `Merger::SetCoverMerge` uses a bitvector to store features that are covered by a file while performing the pass. Collisions while indexing the bitvector are ignored similarly to the fuzzer.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D105284
2021-09-07 09:42:38 -07:00
Frederic Cambus 4787ef3d0f [compiler-rt] Document that builtins is known to work on OpenBSD.
Differential Revision: https://reviews.llvm.org/D109346
2021-09-07 04:25:12 -04:00
David Carlier d669cc7321 [Sanitizer] Intercept clock_getcpuid/pthread_getcpuid on netbsd.
Reviewed By: mgorny

Differential Revision: https://reviews.llvm.org/D109278
2021-09-06 10:51:51 +01:00
Florian Mayer 2706b1a6e2 [hwasan] Test use-after-scope with -fno-exceptions.
Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D109224
2021-09-06 10:10:09 +01:00
David Spickett 915a8bb5ee Revert "[compiler-rt][Profile] Disable test on Arm/AArch64 Linux"
This reverts commit 8b86f8a325.

The inconsistent behaviour has been fixed with
5e50d3073a.
2021-09-06 08:45:06 +00:00
David Carlier 2833a2edac [Sanitizers] netbsd build fix due to wordexp interception. 2021-09-04 12:50:28 +01:00
David Carlier 08c3cdb8b8 [Sanitizers][PGO] missing return statement 2021-09-04 11:40:58 +01:00
Kazuaki Ishizaki a1e7e401d2 [compiler-rt] NFC: Fix trivial typo
Reviewed By: xgupta

Differential Revision: https://reviews.llvm.org/D77457
2021-09-04 14:12:58 +05:30
David Spickett 5e50d3073a [compiler-rt][Profile] Wait for child threads in set-file-object test
We've been seeing this test return 31 instead of 32 for the "functions"
line in this test on our AArch64 bots.

One possible cause is some of the children not finishing in time
before the llvm-profdata commands are run, if the machine is heavily loaded.

Wait for all the children to finish before exiting the parent.

Reviewed By: zequanwu

Differential Revision: https://reviews.llvm.org/D109222
2021-09-03 11:48:50 -07:00
David Spickett 8b86f8a325 [compiler-rt][Profile] Disable test on Arm/AArch64 Linux
While a fix for flaky results is being reviewed.
2021-09-03 12:58:09 +00:00
Florian Mayer abf8ed8a82 [hwasan] Support more complicated lifetimes.
This is important as with exceptions enabled, non-POD allocas often have
two lifetime ends: the exception handler, and the normal one.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D108365
2021-09-03 10:29:50 +01:00
Rainer Orth 79d58b4d30 [sanitizer_common] Define wordexp_wrde_dooffs for Solaris
The Solaris buildbots have been broken for some time:

  In file included from /opt/llvm-buildbot/home/solaris11-amd64/clang-solaris11-amd64/llvm/compiler-rt/lib/asan/asan_interceptors.cpp:174:
  /opt/llvm-buildbot/home/solaris11-amd64/clang-solaris11-amd64/llvm/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:4000:19: error: use of undeclared identifier 'wordexp_wrde_dooffs'
          ((flags & wordexp_wrde_dooffs) ? p->we_offs : 0) + p->we_wordc;
                    ^

This was caused by D108646 <https://reviews.llvm.org/D108646>; the fix is
equivalent to D108838 <https://reviews.llvm.org/D108838>.

Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D109193
2021-09-03 10:02:31 +02:00
David Carlier 25765d860d [Sanitizers] remove empty test case. 2021-09-02 23:53:38 +01:00
David Carlier 6f9a96e9cd [Sanitizers] intercept clock_getcpuclockid on FreeBSD, and pthread_getcpuclockid.
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D108884
2021-09-02 22:35:04 +01:00
Fangrui Song 68745a557e [InstrProfiling] Use llvm.compiler.used if applicable for Mach-O
Similar to D97585.

D25456 used `S_ATTR_LIVE_SUPPORT` to ensure the data variable will be retained
or discarded as a unit with the counter variable, so llvm.compiler.used is
sufficient. It allows ld to dead strip unneeded profc and profd variables.

Reviewed By: vsk

Differential Revision: https://reviews.llvm.org/D105445
2021-09-01 14:46:51 -07:00
Kostya Serebryany b0fdbadf9f [ubsan] warn inside the sigaction interceptor if static linking is suspected, and continue instead of crashing on null deref
[ubsan] warn inside the sigaction interceptor if static linking is suspected, and continue instead of crashing on null deref

Reviewed By: kostik

Differential Revision: https://reviews.llvm.org/D109081
2021-09-01 12:36:48 -07:00
Andrew Browne befb384484 [DFSan][NFC] Fix comment formatting. 2021-08-31 15:35:08 -07:00
Vedant Kumar 6c439a3817 [profile] Specify "-V" to otool to get expected test output
Newer Xcode toolchains ship a new otool implementation that prints out
section contents in a slightly different way than otool-classic. Specify
"-V" to otool to get the expected test output.

Differential Revision: https://reviews.llvm.org/D108929
2021-08-31 10:49:51 -07:00
Kai Luo a594362436 [AIX] Rename shared_libraries_to_archive -> objects_to_archive. NFC. 2021-08-31 06:47:06 +00:00
Xu Mingjie f10d003b0c [tsan] Add environment variable TSAN_SYMBOLIZER_PATH as we do in other sanitizers
ASan, LSan, MSan and UBSan all allow to use environment variable `*SAN_SYMBOLIZER_PATH` to pass the symbolizer path, this patch add `TSAN_SYMBOLIZER_PATH` to TSan.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D108911
2021-08-31 10:18:52 +08:00
Dmitry Vyukov 37b78291c6 tsan: add a comment to CallUserSignalHandler
Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D108907
2021-08-30 11:33:19 +02:00
David Carlier b35e636e40 [Sanitizers] PGO implements enable/disable SIGKILL on callers on FreeBSD.
Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D108867
2021-08-28 22:53:16 +01:00
David Carlier 84013d7093 [Sanitizers] wordexp interceptor build fix on FreeBSD.
Reviewers: vitalybuka

Reviewed-By: vitalybuka

Differential Revision: https://reviews.llvm.org/D108838
2021-08-27 23:58:06 +01:00
Michael Jones 4dde7064e7 [compiler-rt][obvious] fix typo, delete paren
I should've deleted a parenthesis on line 4222 in
https://reviews.llvm.org/D108843 and this patch fixes it.

Differential Revision: https://reviews.llvm.org/D108852
2021-08-27 22:55:39 +00:00
Michael Jones 012b664b2e [compiler-rt] fix real strlens that were missed
In reviews.llvm.org/D108316 I missed a few instances of REAL(strlen) and
this change fixes that, as well as restoring one that seems like it
shouldn't have been changed.

Reviewed By: hctim, vitalybuka

Differential Revision: https://reviews.llvm.org/D108843
2021-08-27 22:33:28 +00:00
Evgenii Stepanov f89ebe108e Support LLVM_ENABLE_PER_TARGET_RUNTIME_DIR in the sanitizer symbolizer build.
In this mode libc++ headers end up in two directories:
* include/<triple>/c++/v1 for the site config header
* include/c++/v1 for everything else

Also switch from -I to -isystem.

Differential Revision: https://reviews.llvm.org/D108841
2021-08-27 13:50:19 -07:00
Zequan Wu 1b05245119 [Profile] Support __llvm_profile_set_file_object in continuous mode.
Replace D107203, because __llvm_profile_set_file_object is usually used when the
process doesn't have permission to open/create file. That patch trying to copy
from old profile to new profile contradicts with the usage.

Differential Revision: https://reviews.llvm.org/D108242
2021-08-27 13:06:46 -07:00
Lang Hames b749ef9e22 [ORC][ORC-RT] Reapply "Introduce ELF/*nix Platform and runtime..." with fixes.
This reapplies e256445bff, which was reverted in 45ac5f5441 due to bot errors
(e.g. https://lab.llvm.org/buildbot/#/builders/112/builds/8599). The issue that
caused the bot failure was fixed in 2e6a4fce35.
2021-08-27 14:41:58 +10:00
Vitaly Buka f1bb30a495 [sanitizer] No THREADLOCAL in qsort and bsearch
qsort can reuse qsort_r if available.
bsearch always passes key as the first comparator argument, so we
can use it to wrap the original comparator.

Differential Revision: https://reviews.llvm.org/D108751
2021-08-26 16:55:06 -07:00
Vitaly Buka 96fa1eaae4 [sanitizer] Add basic qsort test 2021-08-26 12:03:26 -07:00
Vitaly Buka 39100c82d3 [NFC][sanitizer] Swap qsort_r and qsort code
To simplify future review.
2021-08-26 10:24:59 -07:00
Alex Richardson b475ce39e8 [sanitizer] Fix build on FreeBSD RISC-V
We have to avoid calling renameat2 and clone on FreeBSD.
Additionally, the mcontext structure has different members.

Reviewed By: jrtc27, luismarques

Differential Revision: https://reviews.llvm.org/D103886
2021-08-26 12:05:37 +01:00
Vitaly Buka 23a1e9f70b [sanitizer] Add new line to the test 2021-08-25 14:33:06 -07:00
Vitaly Buka c92631a59a [sanitizer] Fix VReport of symbol version
Version is already a string and does not need stringizing.
2021-08-25 14:32:15 -07:00
Vitaly Buka ea575598f5 [sanitizers] Basic realpath test 2021-08-25 14:32:15 -07:00
Vitaly Buka 5213f307ab Revert "Problem with realpath interceptor"
Breaks realpath(, nullptr) for all sanitizers.

Somehow INTERCEPT_FUNCTION and INTERCEPT_FUNCTION_VER return
false even if everything seemingly right.

And this is the issue for COMMON_INTERCEPT_FUNCTION_GLIBC_VER_MIN.
There is a check in every sanitlizer:
if (!INTERCEPT_FUNCTION_VER(name, ver) && !INTERCEPT_FUNCTION(name))

For non-versioned interceptors when INTERCEPT_FUNCTION returns false
it's not considered fatal, and it just prints a warning.

However INTERCEPT_FUNCTION_VER in this case will fallback to
INTERCEPT_FUNCTION replacing realpath with wrong version.

We need to investigate that before relanding the patch.

This reverts commit faef0d042f.
2021-08-25 13:55:23 -07:00
Florian Mayer 023f18bbaf [hwasan] do not check if freed pointer belonged to allocator.
In that case it is very likely that there will be a tag mismatch anyway.

We handle the case that the pointer belongs to neither of the allocators
by getting a nullptr from allocator.GetBlockBegin.

Reviewed By: hctim, eugenis

Differential Revision: https://reviews.llvm.org/D108383
2021-08-25 09:31:01 +01:00
Vitaly Buka 2d743af4e9 [msan] Unpoison trailing nullptr in wordexp interceptor
Differential Revision: https://reviews.llvm.org/D108665
2021-08-24 17:19:11 -07:00
Vitaly Buka 4c699b1cd0 [msan] Don't EXPECT_POISONED beyond the we_wordv
Partially reverts commit 629411d799.

EXPECT_POISONED argument is outside of the allocation so we can't
assume the state of shadow there.
2021-08-24 17:19:10 -07:00
Vitaly Buka 629411d799 [msan] Fix wordexp after D108646
I introduced this bug reformating the patch before commit.
2021-08-24 16:36:28 -07:00
Andrew Browne 76777b216b [DFSan] Add wrapper for getentropy().
Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D108604
2021-08-24 15:10:13 -07:00
Kostya Serebryany 8103b0700d [sanitizer coverage] add a basic default implementation of callbacks for -fsanitize-coverage=inline-8bit-counters,pc-table
[sanitizer coverage] add a basic default implementation of callbacks for -fsanitize-coverage=inline-8bit-counters,pc-table

Reviewed By: kostik

Differential Revision: https://reviews.llvm.org/D108405
2021-08-24 14:56:15 -07:00
Justin Cady d568e5325c [MSAN] Fix wordexp interception when WRDE_DOOFFS is used
Handle the case of wordexp being invoked with WRDE_DOOFFS and
we.we_offs set to a positive value, which will result in NULL
entries prepended to the result. With this change the entire
result, containing both NULL and actual entries, is unpoisoned.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D108646
2021-08-24 14:30:09 -07:00
Ben Langmuir 1c53cadf08 [orc] Fix unit tests that use ORC C API
* c_api_tests was failing to build after the API change to
  __orc_rt_CWrapperFunctionResultAllocate

* wrapper_function_utils_test was causing an assertion failure, because
  it was creating a result for `void(void)` with Size = 0, but seeing an
  uninitialized pointer, which it considered to be an out-of-bound
  error.

I noticed locally that making modifications to c_api.h is not causing
these unit tests to be rebuilt, which may be how the bug slipped in in
the first place.

Differential Revision: https://reviews.llvm.org/D108649
2021-08-24 14:19:46 -07:00
Mitch Phillips 433b2eaf91 [hwasan] Always untag short granule in shadow.
Fixes a regression when the allocator is disabled, and a dirty
allocation is re-used. This only occurs when the allocator is disabled,
so a test-only fix, but still necessary.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D108650
2021-08-24 14:10:04 -07:00
Vitaly Buka 266a8d5cfe [tsan] Fix sigaction interceptor after D107186
Set SA_SIGINFO only if we set sighandler, or we can set the flag, and
return it as 'old' without actual sigaction set.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D108616
2021-08-24 10:19:02 -07:00
Ben Langmuir 2f53fcc831 Fix compilation errors in C API when using C compiler
* Cannot use sizeof() on another union member
* nullptr vs NULL
* () vs (void)

Incidentally, fix an incorrect comment about memory ownership on the
argument to __orc_rt_CreateCWrapperFunctionResultFromOutOfBandError,
which is copied, not moved.
2021-08-24 10:12:32 -07:00
Michael Jones 40067b88c0 [compiler-rt] change internal internal libc invariants
llvm-libc is expected to be built with sanitizers and not use interceptors in
the long run. For now though, we have a hybrid process, where functions
implemented in llvm-libc are instrumented, and glibc fills and sanitizer
interceptors fill in the rest.

Current sanitizers have an invariant that the REAL(...) function called from
inside of an interceptor is uninstrumented. A lot of interceptors call strlen()
in order to figure out the size of the region to check/poison. Switch these
callsites over to the internal, unsanitized implementation.

Reviewed By: hctim, vitalybuka

Differential Revision: https://reviews.llvm.org/D108316
2021-08-24 16:41:15 +00:00
Lang Hames 8614cb9f99 [ORC-RT] Add non-const WrapperFunctionResult data access, simplify allocate.
WrapperFunctionResult no longer supports wrapping constant data, so this patch
provides direct non-const access to the wrapped data. Since wrapped data can now
be written, the WrapperFunctionResult::allocate method can be simplified to
return a WrapperFunctionResult.

This is essentially the same change (and with the same motivation) as LLVM
commit 8b117830b1, but applied to the ORC runtime's WrapperFunctionResult code.
2021-08-24 17:34:59 +10:00
Petr Hosek 8111f2f7ee [profile] Update counter offset to account for binary ids
In the raw profile, binary ids immediately follow the header so when
computing counters offset we need to account for the new section.

Differential Revision: https://reviews.llvm.org/D108608
2021-08-23 21:43:30 -07:00
Amy Kwan 4cd8dd3fe0 [scudo][standalone] Link tests against libatomic if libatomic exists
It is possible that libatomic does not exist on some systems. This patch updates
the scudo standalone tests to link against libatomic if the library exists.

This is an update to the original patch: https://reviews.llvm.org/D64134 and
aims to resolve https://bugs.llvm.org/show_bug.cgi?id=51431.

Differential Revision: https://reviews.llvm.org/D108503
2021-08-22 13:47:04 -05:00
Florian Mayer de916a7b12 Revert "[hwasan] do not check if freed pointer belonged to allocator."
This reverts commit 119146f8ae.
2021-08-20 12:21:00 +01:00
Florian Mayer 119146f8ae [hwasan] do not check if freed pointer belonged to allocator.
In that case it is very likely that there will be a tag mismatch anyway.

We handle the case that the pointer belongs to neither of the allocators
by getting a nullptr from allocator.GetBlockBegin.

Reviewed By: hctim, eugenis

Differential Revision: https://reviews.llvm.org/D108383
2021-08-20 10:12:47 +01:00
Martin Storsjö e6407356ba [builtins] Move Windows/ARM frontends for fix/float functions into the individual source files
This avoids pulling in all of them if only one of them is needed
(if builtins are built without -ffunction-sections), and matches how
the similar aliases for AEABI are set up.

Differential Revision: https://reviews.llvm.org/D107815
2021-08-20 09:13:32 +03:00
Jinsong Ji 337bd67d83 [CRT][LIT] split target_cflags using shlex
Update the commit to use shlex.split, as the win32 is not supported.

See https://reviews.llvm.org/D108329 for discussions.
2021-08-19 21:44:20 -04:00
Brian Cain 68ab571e22 [sanitizer] Fix for CMAKE_CXX_FLAGS update
With unquoted ${CMAKE_CXX_FLAGS}, the REGEX fails when it's empty:

```CMake Error at lib/scudo/standalone/CMakeLists.txt:14 (string):
string sub-command REGEX, mode REPLACE needs at least 6 arguments total to
command.```
2021-08-19 12:05:55 -07:00
Jinsong Ji 0541ce4ef9 [CRT][LIT] build the target_cflags for Popen properly
We recently enabled crt for powerpc in
https://reviews.llvm.org/rGb7611ad0b16769d3bf172e84fa9296158f8f1910.

And we started to see some unexpected error message when running
check-runtimes.

eg:
https://lab.llvm.org/buildbot/#/builders/57/builds/9488/steps/6/logs/stdio
line 100 - 103:

"
clang-14: error: unknown argument: '-m64 -fno-function-sections'
clang-14: error: unknown argument: '-m64 -fno-function-sections'
clang-14: error: unknown argument: '-m64 -fno-function-sections'
clang-14: error: unknown argument: '-m64 -fno-function-sections'
"

Looks like we shouldn't strip the space at the beginning,
or else the command line passed to subprocess won't work well.

Reviewed By: phosek, MaskRay

Differential Revision: https://reviews.llvm.org/D108329
2021-08-19 15:39:53 +00:00
Marco Elver 303d278ad2 [tsan] Fix pthread_once() on Mac OS X
Change 636428c727 enabled BlockingRegion hooks for pthread_once().
Unfortunately this seems to cause crashes on Mac OS X which uses
pthread_once() from locations that seem to result in crashes:

| ThreadSanitizer:DEADLYSIGNAL
| ==31465==ERROR: ThreadSanitizer: stack-overflow on address 0x7ffee73fffd8 (pc 0x00010807fd2a bp 0x7ffee7400050 sp 0x7ffee73fffb0 T93815)
|     #0 __tsan::MetaMap::GetSync(__tsan::ThreadState*, unsigned long, unsigned long, bool, bool) tsan_sync.cpp:195 (libclang_rt.tsan_osx_dynamic.dylib:x86_64+0x78d2a)
|     #1 __tsan::MutexPreLock(__tsan::ThreadState*, unsigned long, unsigned long, unsigned int) tsan_rtl_mutex.cpp:143 (libclang_rt.tsan_osx_dynamic.dylib:x86_64+0x6cefc)
|     #2 wrap_pthread_mutex_lock sanitizer_common_interceptors.inc:4240 (libclang_rt.tsan_osx_dynamic.dylib:x86_64+0x3dae0)
|     #3 flockfile <null>:2 (libsystem_c.dylib:x86_64+0x38a69)
|     #4 puts <null>:2 (libsystem_c.dylib:x86_64+0x3f69b)
|     #5 wrap_puts sanitizer_common_interceptors.inc (libclang_rt.tsan_osx_dynamic.dylib:x86_64+0x34d83)
|     #6 __tsan::OnPotentiallyBlockingRegionBegin() cxa_guard_acquire.cpp:8 (foo:x86_64+0x100000e48)
|     #7 wrap_pthread_once tsan_interceptors_posix.cpp:1512 (libclang_rt.tsan_osx_dynamic.dylib:x86_64+0x2f6e6)

From the stack trace it can be seen that the caller is unknown, and the
resulting stack-overflow seems to indicate that whoever the caller is
does not have enough stack space or otherwise is running in a limited
environment not yet ready for full instrumentation.

Fix it by reverting behaviour on Mac OS X to not call BlockingRegion
hooks from pthread_once().

Reported-by: azharudd

Reviewed By: glider

Differential Revision: https://reviews.llvm.org/D108305
2021-08-19 13:17:45 +02:00
Vitaly Buka 03bd05f0e8 [sanitizer] Use TMPDIR in Android test
TMPDIR was added long time ago, so no need to use EXTERNAL_STORAGE.
2021-08-18 19:05:21 -07:00
Vitaly Buka 3d4d1b9b29 [scudo] Don't build SCUDO for Android
Android 11 uses scudo_standalone as default
allocator making difficult to test legacy scudo.
2021-08-18 18:32:54 -07:00
Azharuddin Mohammed b4b8e1446a [tsan] Disable all Trace unit tests on Mac
In an earlier commit (7338be0e6e), only the MemoryAccessSize unit test
was disabled whereas the other tests which are also failing were not.
2021-08-18 11:47:51 -07:00
Mitch Phillips fd51ab6341 [hwasan] Don't report short-granule shadow as overwritten.
The shadow for a short granule is stored in the last byte of the
granule. Currently, if there's a tail-overwrite report (a
buffer-overflow-write in uninstrumented code), we report the shadow byte
as a mismatch against the magic.

Fix this bug by slapping the shadow into the expected value. This also
makes sure that if the uninstrumented WRITE does clobber the shadow
byte, it reports the shadow was actually clobbered as well.

Reviewed By: eugenis, fmayer

Differential Revision: https://reviews.llvm.org/D107938
2021-08-18 11:25:57 -07:00
Jinsong Ji 66e2772e42 [InstrProfiling] Support relative CountersPtr for PlatformOther
D104556 change the CountersPtr to be relative, however, it did not
update the pointer initialization in  __llvm_profile_register_function,
so the platform (eg:AIX) that use __llvm_profile_register_function is now totaly
broken, any PGO code will SEGV.

This patch update the code to reflect that the Data->CountersPtr is now
relative.

Reviewed By: MaskRay, davidxl

Differential Revision: https://reviews.llvm.org/D108304
2021-08-18 17:45:39 +00:00
Alexander Potapenko 7338be0e6e [tsan] Disable Trace.MemoryAccessSize on Mac
According to comments at https://reviews.llvm.org/D107911,
Trace.MemoryAccessSize fails on Mac buildbots.
Because this test is newly introduced, and is the only user of the code
added in that patch, disable the test on Mac till the problem is
resolved.

Differential Revision: https://reviews.llvm.org/D108294
2021-08-18 15:09:36 +02:00
Lang Hames 45ac5f5441 Revert "[ORC-RT][ORC] Introduce ELF/*nix Platform and runtime support."
This reverts commit e256445bff.

This commit broke some of the bots (see e.g.
https://lab.llvm.org/buildbot/#/builders/112/builds/8599). Reverting while I
investigate.
2021-08-18 20:42:23 +10:00
David Spickett 6c7956ea49 [compiler-rt][fuzzer] Explain why fuzzer-flags test is unsupported on AArch64
See https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=4d1c6335455aeeda9de8a5eb58998f919ea35a1e
for the specific change that fixes the issue.
2021-08-18 10:10:19 +00:00
Azharuddin Mohammed 593275c93c Mark tsan cxa_guard_acquire test as unsupported on Darwin
This test started failing after recent commit `636428c727cd`. Marking it
as unsupported to get the bot back to green until we get it sorted out.
2021-08-18 01:44:20 -07:00
Lang Hames e256445bff [ORC-RT][ORC] Introduce ELF/*nix Platform and runtime support.
This change adds support to ORCv2 and the Orc runtime library for static
initializers, C++ static destructors, and exception handler registration for
ELF-based platforms, at present Linux and FreeBSD on x86_64. It is based on the
MachO platform and runtime support introduced in bb5f97e3ad.

Patch by Peter Housel. Thanks very much Peter!

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D108081
2021-08-18 15:00:22 +10:00
Brian Cain f2880329eb [compiler-rt] Add hexagon support to profile
Adds build support for hexagon linux to profile
2021-08-17 19:59:24 -07:00
Brian Cain 065274b9cf [compiler-rt] Add hexagon support to safestack
Adds build support for hexagon linux to safestack.
2021-08-17 19:59:24 -07:00
Brian Cain c172726f6f [compiler-rt] Add hexagon support to CFI
Adds build support for hexagon linux to cfi.
2021-08-17 19:59:24 -07:00
Brian Cain 5b650b1467 [sanitizer] Add hexagon support to scudo
Adds build support for hexagon linux to scudo, scudo-standalone
2021-08-17 19:59:24 -07:00
Brian Cain 217191b70a [sanitizer] Add hexagon support to ubsan
Adds build support for hexagon linux to ubsan.
2021-08-17 19:59:24 -07:00
Brian Cain 043926a3a0 [sanitizer] Add hexagon support to asan
Adds build support for hexagon linux to asan.
2021-08-17 19:59:24 -07:00
Brian Cain 5ac9d41067 [sanitizer] Add hexagon support to lsan
Adds build support for hexagon linux to lsan.
2021-08-17 19:59:24 -07:00
Brian Cain 9ea59e1cd0 [sanitizer] Add hexagon support to sanitizer-common
Adds build support for hexagon linux to sanitizer common.
2021-08-17 19:59:24 -07:00
Evgenii Stepanov 8a570a873b [hwasan] Support malloc in atfork.
Before this change we were locking the StackDepot in the fork()
interceptor. This results in a deadlock when allocator functions are
used in a pthread_atfork() callback.

Instead, set up a pthread_atfork() callback at init that locks/unlocks
both StackDepot and the allocator. Since our callback is set up very
early, the pre-fork callback is executed late, and both post-fork ones
are executed early, which works perfect for us.

Differential Revision: https://reviews.llvm.org/D108063
2021-08-17 15:29:49 -07:00
Evgenii Stepanov c9ce76febb (NFC) clang-format hwasan/hwasan_linux.cpp
Differential Revision: https://reviews.llvm.org/D108224
2021-08-17 15:03:46 -07:00