Commit Graph

15344 Commits

Author SHA1 Message Date
Vitaly Buka 81f9dc8eee [sanitizer] Lock/Unlock stack store on fork
Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D115210
2021-12-07 11:17:16 -08:00
Clemens Wasser d9aa5f6158 Format lsan.h
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115262
2021-12-07 10:52:35 -08:00
Clemens Wasser 33e3554ea3 Change __lsan::kMaxAllowedMallocSize literal types to unsigned long long
It is required for the [Leak Sanitizer port to Windows](https://reviews.llvm.org/D115103).
The currently used `unsigned long` type is 64 bits wide on UNIX like systems but only 32 bits wide on Windows.
Because of that, the literal `8UL << 30` causes an integer overflow on Windows.
By changing the type of the literals to `unsigned long long`, we have consistent behavior and no overflows on all Platforms.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115186
2021-12-07 10:38:13 -08:00
Kirill Stoimenov 591eb2bdb6 [ASan] Refactored SHADOW_OFFSET implementation to use SHADOW_OFFSET_CONST.
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115213
2021-12-07 17:53:38 +00:00
David Spickett 6bfbb89e96 [compiler-rt][libFuzzer] Disable counters test on arm
This test is either very slow or loops forever on 32 bit Arm.

One of a few tests causing timeouts on our buildbots:
https://lab.llvm.org/buildbot/#/builders/190/builds/513
2021-12-07 11:55:11 +00:00
Vitaly Buka fc3a260a0f [sanitizer] Don't lock for StackStore::Allocated() 2021-12-07 01:00:01 -08:00
Vitaly Buka 7151c71481 [sanitizer] Fix CompressStackStore VPrint message 2021-12-07 01:00:01 -08:00
Vitaly Buka faf694b662 [NFC][sanitizer] Non-copyable ScopedBlockSignals 2021-12-06 22:37:28 -08:00
Vitaly Buka 061f22dda9 [NFC][lsan] Refactor LockThreadRegistry/LockAllocator calls 2021-12-06 22:27:11 -08:00
Vitaly Buka aa407c102d [NFC][sanitizer] Fix typo in comment 2021-12-06 22:27:11 -08:00
Nico Weber 63d518f31a [tsan] Move tsan/rtl build rules into tsan/rtl/CMakeLists.txt
That way, the build rules are closer to the source files they describe.

No intended behavior change.

Differential Revision: https://reviews.llvm.org/D115155
2021-12-06 19:58:30 -05:00
Kirill Stoimenov f4606d2b70 Revert "[ASan] Added asan_shadow_defines.h, which contatins shadow offset for various platforms."
This reverts commit 2f3bb59f51.

Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D115193
2021-12-06 23:13:31 +00:00
Kirill Stoimenov 2f3bb59f51 [ASan] Added asan_shadow_defines.h, which contatins shadow offset for various platforms.
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115075
2021-12-06 21:45:46 +00:00
Dmitry Vyukov 954582cdfc tsan: disable dlopen_static_tls.cpp test on powerpc64
Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D115142
2021-12-06 13:13:43 +01:00
Dimitry Andric bbba9d8c1b [XRay] fix more -Wformat warnings
Building xray with recent clang on a 64-bit system results in a number
of -Wformat warnings:

    compiler-rt/lib/xray/xray_allocator.h:70:11: warning: format specifies type 'int' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
              RoundedSize, B);
              ^~~~~~~~~~~
    compiler-rt/lib/xray/xray_allocator.h:119:11: warning: format specifies type 'int' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
              RoundedSize, B);
              ^~~~~~~~~~~

Since `__sanitizer::uptr` has the same size as `size_t`, these can be
fixed by using the printf specifier `%zu`.

    compiler-rt/lib/xray/xray_basic_logging.cpp:348:46: warning: format specifies type 'int' but the argument has type '__sanitizer::tid_t' (aka 'unsigned long long') [-Wformat]
          Report("Cleaned up log for TID: %d\n", GetTid());
                                          ~~     ^~~~~~~~
                                          %llu
    compiler-rt/lib/xray/xray_basic_logging.cpp:353:62: warning: format specifies type 'int' but the argument has type '__sanitizer::tid_t' (aka 'unsigned long long') [-Wformat]
          Report("Skipping buffer for TID: %d; Offset = %llu\n", GetTid(),
                                           ~~                    ^~~~~~~~
                                           %llu

Since `__sanitizer::tid_t` is effectively declared as `unsigned long
long`, these can be fixed by using the printf specifier `%llu`.

    compiler-rt/lib/xray/xray_basic_logging.cpp:354:14: warning: format specifies type 'unsigned long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
                 TLD.BufferOffset);
                 ^~~~~~~~~~~~~~~~

Since `BufferOffset` is declared as `size_t`, this one can be fixed by
using `%zu` as a printf specifier.

    compiler-rt/lib/xray/xray_interface.cpp:172:50: warning: format specifies type 'int' but the argument has type 'uint64_t' (aka 'unsigned long') [-Wformat]
        Report("Unsupported sled kind '%d' @%04x\n", Sled.Address, int(Sled.Kind));
                                       ~~            ^~~~~~~~~~~~
                                       %lu

Since ``xray::SledEntry::Address` is declared as `uint64_t`, this one
can be fixed by using `PRIu64`, and adding `<cinttypes>`.

    compiler-rt/lib/xray/xray_interface.cpp:308:62: warning: format specifies type 'long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
        Report("System page size is not a power of two: %lld\n", PageSize);
                                                        ~~~~     ^~~~~~~~
                                                        %zu
    compiler-rt/lib/xray/xray_interface.cpp:359:64: warning: format specifies type 'long long' but the argument has type 'size_t' (aka 'unsigned long') [-Wformat]
        Report("Provided page size is not a power of two: %lld\n", PageSize);
                                                          ~~~~     ^~~~~~~~
                                                          %zu

Since `PageSize` is declared as `size_t`, these can be fixed by using
`%zu` as a printf specifier.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114469
2021-12-04 20:01:20 +01:00
Dmitry Vyukov fd26417a74 tsan: disable dlopen_static_tls.cpp test on aarch64
Fails on bots: https://lab.llvm.org/buildbot#builders/184/builds/1580

Differential Revision: https://reviews.llvm.org/D115095
2021-12-04 13:01:47 +01:00
Vitaly Buka 9c491c873c [sanitizer] Hook up LZW into stack store
Depends on D114503.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114924
2021-12-03 23:38:41 -08:00
Kevin Athey 6c2be3015e [NFC][sanitizer] Add test for command line flag for enable-noundef-analysis.
A simple unit test to demonstrate the flags working correctly.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114485
2021-12-03 15:44:36 -08:00
Vitaly Buka 98bb198693 [sanitizer] Add Lempel–Ziv–Welch encoder/decoder
It's very simple, fast and efficient for the stack depot compression if used on entire pointers.

Reviewed By: morehouse, kstoimenov

Differential Revision: https://reviews.llvm.org/D114918
2021-12-03 13:11:40 -08:00
Vitaly Buka 5f1d1854eb [NFC][sanitizer] Iterator adaptors for Leb128 encoding
It's similar to back_insert_iterator

Needed for D114924

Reviewed By: morehouse, kstoimenov

Differential Revision: https://reviews.llvm.org/D114980
2021-12-03 12:51:55 -08:00
Vitaly Buka 6318001209 [sanitizer] Support IsRssLimitExceeded in all sanitizers
Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D115000
2021-12-03 12:45:44 -08:00
Choongwoo Han 46282fad06 [Sanitizer] Use CreateDirectoryA for report dirs
Using `_mkdir` of CRT in Asan Init leads to launch failure and hanging in Windows.

You can trigger it by calling:
> set ASAN_OPTIONS=log_path=a/a/a
> .\asan_program.exe

And their crash dump shows the following stack trace:
```
_guard_dispatch_icall_nop()
__acrt_get_utf8_acp_compatibility_codepage()
_mkdir(const char * path)
```

I guess there could be a cfg guard in CRT, which may lead to calling uninitialized cfg guard function address. Also, `_mkdir` supports UTF-8 encoding of the path and calls _wmkdir, but that's not necessary for this case since other file apis in sanitizer_win.cpp assumes only ANSI code case, so it makes sense to use CreateDirectoryA matching other file api calls in the same file.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D114760
2021-12-03 12:34:05 -08:00
Hans Wennborg c361ab0612 [msan] Don't block SIGSYS in ScopedBlockSignals
Seccomp-BPF-sandboxed processes rely on being able to process SIGSYS
signals.

Differential revision: https://reviews.llvm.org/D115057
2021-12-03 20:41:08 +01:00
Mitch Phillips 572a0721a0 [HWASan] Try 'google' prefixed apex directories in symbolizer.
Google-signed apexes appear on Android build servers' symbol files as
being under /apex/com.google.android.<foo>/. In reality, the apexes are
always installed as /apex/com.android.<foo>/ (note the lack of
'google'). In order for local symbolization under hwasan_symbolize to
work correctly, we also try the 'google' directory.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D114919
2021-12-03 10:35:03 -08:00
Dmitry Vyukov 4a5086dce3 tsan: disable munmap_invalid.cpp test on darwin
It failed on bots:
https://green.lab.llvm.org/green//job/clang-stage1-RA/25954/consoleFull#-1417328700a1ca8a51-895e-46c6-af87-ce24fa4cd561
and it  doesn't provide the test output.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114972
2021-12-03 09:03:45 +01:00
Vitaly Buka 550fd071ed [lsan] Deflake fork_and_leak test 2021-12-02 18:06:04 -08:00
Vitaly Buka 36e6a259c8 [NFC][sanitizer] Remove SetSoftRssLimitExceededCallback
According comments on D44404, something like that was the goal.

Reviewed By: morehouse, kstoimenov

Differential Revision: https://reviews.llvm.org/D114991
2021-12-02 14:37:02 -08:00
Vitaly Buka 3195610b2d [NFC][sanitizer] Reorder functions in cpp file 2021-12-02 14:35:59 -08:00
Vitaly Buka d48d8670b5 [NFC][sanitizer] Rename RssLimitExceeded -> IsRssLimitExceeded 2021-12-02 12:52:00 -08:00
Kirill Stoimenov 351ef92cd9 [ASan] Fixed asan_mapping.h format.
Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D114987
2021-12-02 20:24:44 +00:00
Vitaly Buka 655c0bfe2c [NFC][sanitizer] Use more bytes of sanitizer_stack_store_test pointers 2021-12-02 11:49:29 -08:00
Leonard Chan 534d105e03 [compiler-rt] Fix incorrect variable names used 2021-12-02 11:46:50 -08:00
Vitaly Buka 25feff4b2e [sanitizer] Start background thread once
Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114933
2021-12-02 11:45:37 -08:00
Vitaly Buka 67207797e9 [asan] Remove confusing workaround
The goal is to identify the bot and try to fix it.

SetSoftRssLimitExceededCallback is AsanInitInternal as I assume
that only MaybeStartBackgroudThread needs to be delayed to constructors.
Later I want to move MaybeStartBackgroudThread call into sanitizer_common.

If it needs to be reverted please provide to more info, like bot, or details about setup.

Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D114934
2021-12-02 11:44:40 -08:00
Vitaly Buka 78cc133c63 [sanitizer] Add delta compression stack depot
Compress by factor 4x, takes about 10ms per 8 MiB block.

Depends on D114498.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114503
2021-12-02 10:46:41 -08:00
Vitaly Buka 7185b1f406 [sanitizer] Add compress_stack_depot flag
Depends on D114494.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114495
2021-12-02 10:42:53 -08:00
Vitaly Buka 323bfad72d [sanitizer] DEFINE_REAL_PTHREAD_FUNCTIONS for hwasan, lsan, msan
It should be NFC, as they already intercept pthread_create.

This will let us to fix BackgroundThread for these sanitizerts.
In in followup patches I will fix MaybeStartBackgroudThread for them
and corresponding tests.

Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D114935
2021-12-02 10:24:04 -08:00
Dmitry Vyukov 1b576585eb tsan: tolerate munmap with invalid arguments
We call UnmapShadow before the actual munmap, at that point we don't yet
know if the provided address/size are sane. We can't call UnmapShadow
after the actual munmap becuase at that point the memory range can
already be reused for something else, so we can't rely on the munmap
return value to understand is the values are sane.
While calling munmap with insane values (non-canonical address, negative
size, etc) is an error, the kernel won't crash. We must also try to not
crash as the failure mode is very confusing (paging fault inside of the
runtime on some derived shadow address).

Such invalid arguments are observed on Chromium tests:
https://bugs.chromium.org/p/chromium/issues/detail?id=1275581

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114944
2021-12-02 17:50:51 +01:00
Dmitry Vyukov 97b4e63117 tsan: fix false positives in dynamic libs with static tls
The added test demonstrates  loading a dynamic library with static TLS.
Such static TLS is a hack that allows a dynamic library to have faster TLS,
but it can be loaded only iff all threads happened to allocate some excess
of static TLS space for whatever reason. If it's not the case loading fails with:

dlopen: cannot load any more object with static TLS

We used to produce a false positive because dlopen will write into TLS
of all existing threads to initialize/zero TLS region for the loaded library.
And this appears to be racing with initialization of TLS in the thread
since we model a write into the whole static TLS region (we don't what part
of it is currently unused):

WARNING: ThreadSanitizer: data race (pid=2317365)
  Write of size 1 at 0x7f1fa9bfcdd7 by main thread:
    0 memset
    1 init_one_static_tls
    2 __pthread_init_static_tls
    [[ this is where main calls dlopen ]]
    3 main
  Previous write of size 8 at 0x7f1fa9bfcdd0 by thread T1:
    0 __tsan_tls_initialization

Fix this by ignoring accesses during dlopen.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114953
2021-12-02 17:47:05 +01:00
Vitaly Buka 8aabde5a4b [NFC][sanitizer] Check &real_pthread_join
It's a weak function which may be undefined.
2021-12-01 23:59:33 -08:00
Vitaly Buka ae234a7545 [NFC][sanitizer] Fix "not used" warning in test 2021-12-01 20:16:25 -08:00
Steven Wan f9d585d0dd Revert "[sanitizer] Add compress_stack_depot flag"
This is failing on clang-s390x-linux,
https://lab.llvm.org/buildbot/#/builders/94/builds/6748.
This reverts commit bf18253b0e.
2021-12-01 20:21:52 -05:00
Julian Lettner 863b117411 [TSan][Darwin] Prevent inlining of functions in tests
Prevent inlining of functions so we can FileCheck the generated stack
traces.
2021-12-01 17:00:52 -08:00
Julian Lettner 6703fe25b7 [TSan][Darwin] Mark test unsupported 2021-12-01 15:50:10 -08:00
Vitaly Buka e599aa80c0 [sanitizer] Implement MprotectReadOnly and MprotectNoAccess
MprotectReadOnly for Win and Fuchsia
MprotectNoAccess for Fuchsia
2021-12-01 14:50:50 -08:00
Vitaly Buka 86f48fbb1c [NFC][sanitizer] constexpr in sanitizer_dense_map_info 2021-12-01 13:45:42 -08:00
Hans Wennborg b1bc627e7e Revert "[sanitizer] Add delta compression stack depot"
Broke the build on Windows, where MprotectReadOnly() isn't defined, see comment
on the code review.

> Compress by factor 4x, takes about 10ms per 8 MiB block.
>
> Depends on D114498.
>
> Reviewed By: morehouse
>
> Differential Revision: https://reviews.llvm.org/D114503

This reverts commit 1d8f295759.
2021-12-01 20:02:51 +01:00
Kirill Stoimenov b6f3c8de6a [ASan] Fixed include order.
Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114835
2021-12-01 17:59:23 +00:00
Dmitry Vyukov 09859113ed Revert "tsan: new runtime (v3)"
This reverts commit 66d4ce7e26.

Chromium tests started failing:
https://bugs.chromium.org/p/chromium/issues/detail?id=1275581
2021-12-01 18:00:46 +01:00
Vitaly Buka 1d8f295759 [sanitizer] Add delta compression stack depot
Compress by factor 4x, takes about 10ms per 8 MiB block.

Depends on D114498.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114503
2021-11-30 20:19:53 -08:00
Snehasish Kumar 3a4d373ec2 [memprof] Align each rawprofile section to 8b.
The first 8b of each raw profile section need to be aligned to 8b since
the first item in each section is a u64 count of the number of items in
the section.
Summary of changes:
* Assert alignment when reading counts.
* Update test to check alignment, relax some size checks to allow padding.
* Update raw binary inputs for llvm-profdata tests.

Differential Revision: https://reviews.llvm.org/D114826
2021-11-30 20:12:43 -08:00
Vitaly Buka bf18253b0e [sanitizer] Add compress_stack_depot flag
Depends on D114494.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114495
2021-11-30 19:08:58 -08:00
Vitaly Buka 7c1d777983 [NFC][sanitizer] Add entry point for compression
Add Compression::Test type which just pretends packing,
but does nothing useful. It's only called from test for now.

Depends on D114493.

Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D114494
2021-11-30 16:55:29 -08:00
Julian Lettner 858eb8fc11 [TSan][Darwin] Avoid crashes due to interpreting non-zero shadow content as a pointer
We would like to use TLS to store the ThreadState object (or at least a
reference ot it), but on Darwin accessing TLS via __thread or manually
by using pthread_key_* is problematic, because there are several places
where interceptors are called when TLS is not accessible (early process
startup, thread cleanup, ...).

Previously, we used a "poor man's TLS" implementation, where we use the
shadow memory of the pointer returned by pthread_self() to store a
pointer to the ThreadState object.

The problem with that was that certain operations can populate shadow
bytes unbeknownst to TSan, and we later interpret these non-zero bytes
as the pointer to our ThreadState object and crash on when dereferencing
the pointer.

This patch changes the storage location of our reference to the
ThreadState object to "real" TLS.  We make this work by artificially
keeping this reference alive in the pthread_key destructor by resetting
the key value with pthread_setspecific().

This change also fixes the issue were the ThreadState object is
re-allocated after DestroyThreadState() because intercepted functions
can still get called on the terminating thread after the
THREAD_TERMINATE event.

Radar-Id: rdar://problem/72010355

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D110236
2021-11-30 14:49:23 -08:00
Vitaly Buka aeeacbd989 [NFC][sanitizer] Fail test quickly 2021-11-30 14:00:07 -08:00
Snehasish Kumar 1cf11df591 [memprof] Disable pedantic warnings, suppress variadic macro warning.
The memprof unit tests use an older version of gmock (included in the
repo) which does not build cleanly with -pedantic:
https://github.com/google/googletest/issues/2650
For now just silence the warning by disabling pedantic and add the
appropriate flags for gcc and clang.
2021-11-30 12:03:19 -08:00
Snehasish Kumar a2ce97cc3f [memprof] Fix unit test build after refactoring shared header.
The memprof unittest also needs to include the MemProfData.inc header
directly to have access to MEMPROF_RAW_MAGIC and MEMPROF_RAW_VERSION
globals.
2021-11-30 11:35:42 -08:00
Snehasish Kumar 7cca33b40f [memprof] Extend llvm-profdata to display MemProf profile summaries.
This commit adds initial support to llvm-profdata to read and print
summaries of raw memprof profiles.
Summary of changes:
* Refactor shared defs to MemProfData.inc
* Extend show_main to display memprof profile summaries.
* Add a simple raw memprof profile reader.
* Add a couple of tests to tools/llvm-profdata.

Differential Revision: https://reviews.llvm.org/D114286
2021-11-30 10:45:26 -08:00
Ryan Mansfield 455a2b824d Fix file extension of alignment-assumption-ignorelist.cppp test
During the renaming of blacklist to ignorelist this test got renamed
incorrectly.

Differential revision: https://reviews.llvm.org/D114710
2021-11-30 17:21:46 +01:00
Vitaly Buka a06d352756 [NFC][sanitizer] Track progress of populating the block
In multi-threaded application concurrent StackStore::Store may
finish in order different from assigned Id. So we can't assume
that after we switch writing the next block the previous is done.

The workaround is to count exact number of uptr stored into the block,
including skipped tail/head which were not able to fit entire trace.

Depends on D114490.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114493
2021-11-29 18:44:39 -08:00
Vitaly Buka 25a7e4b9f7 [sanitizer] Add Leb128 encoding/decoding
Reviewed By: dvyukov, kstoimenov

Differential Revision: https://reviews.llvm.org/D114464
2021-11-29 17:12:34 -08:00
Matt Morehouse 2022e2fcd0 [HWASan] Disable LTO test on aarch64.
It fails for non-Android aarch64 bots as well.
2021-11-29 08:08:17 -08:00
Vitaly Buka 168bc7ce7e [sanitizer] Remove storeIds and use padding of StackDepotNode
Depends on D114489.

Reviewed By: morehouse, dvyukov

Differential Revision: https://reviews.llvm.org/D114490
2021-11-28 01:58:49 -08:00
Vitaly Buka cc2794abea [sanitizer] Switch StackStore from pointers to 32bit IDs
Depends on D114488.

Reviewed By: morehouse, dvyukov, kstoimenov

Differential Revision: https://reviews.llvm.org/D114489
2021-11-28 01:44:28 -08:00
Vitaly Buka 8ae815cb19 [sanitizer] Switch StackStore to 8 MiB blocks
Larger blocks are more convenient for compressions.
Blocks are allocated with MmapNoReserveOrDie to save some memory.

Also it's 15% faster on StackDepotBenchmarkSuite

Depends on D114464.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114488
2021-11-26 14:00:10 -08:00
Patrick Oppenlander 06f1d63cb1 [compiler-rt][CMake] Support powerpcspe build
Reviewed By: jhibbits, MaskRay

Differential Revision: https://reviews.llvm.org/D98836
2021-11-26 13:29:06 -08:00
Patrick Oppenlander 30029e7407 [compiler-rt][CMake] Support powerpc32 on Linux
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D98831
2021-11-26 13:28:20 -08:00
Benjamin Kramer 0e099a64be [tsan] Relax atexit5.cpp a bit more so it's not as dependent on the standard library implementation 2021-11-26 14:02:34 +01:00
Dmitry Vyukov a1dc97e472 tsan: remember and print function that installed at_exit callbacks
Sometimes stacks for at_exit callbacks don't include any of the user functions/files.
For example, a race with a global std container destructor will only contain
the container type name and our at_exit_wrapper function. No signs what global variable
this is.
Remember and include in reports the function that installed the at_exit callback.
This should give glues as to what variable is being destroyed.

Depends on D114606.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114607
2021-11-26 08:00:55 +01:00
Dmitry Vyukov 3f87788de1 tsan: add a test for on_exit
Depends on D114605.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114606
2021-11-26 08:00:43 +01:00
Dmitry Vyukov 9ea3bd5a1c tsan: add test for __cxa_atexit
Add a test for a common C++ bug when a global object is destroyed
while background threads still use it.

Depends on D114604.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114605
2021-11-26 08:00:29 +01:00
Dmitry Vyukov c2f0de06c9 tsan: check stack in atexit4.cpp test
Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114604
2021-11-26 08:00:19 +01:00
Quinn Pham 5c162ec545 [NFC][compiler-rt] Inclusive language: replace master/slave with primary/secondary
[NFC] As part of using inclusive language within the llvm project, this patch
replaces master and slave with primary and secondary respectively in
`sanitizer_mac.cpp`.

Reviewed By: ZarkoCA

Differential Revision: https://reviews.llvm.org/D114255
2021-11-25 13:30:56 -06:00
Dmitry Vyukov 66d4ce7e26 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Depends on D112602.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112603
2021-11-25 18:32:04 +01:00
Dmitry Vyukov 976bb4724c tsan: fix another potential deadlock in fork
Linux/fork_deadlock.cpp currently hangs in debug mode in the following stack.
Disable memory access handling in OnUserAlloc/Free around fork.

1  0x000000000042c54b in __sanitizer::internal_sched_yield () at sanitizer_linux.cpp:452
2  0x000000000042da15 in __sanitizer::StaticSpinMutex::LockSlow (this=0x57ef02 <__sanitizer::internal_allocator_cache_mu>) at sanitizer_mutex.cpp:24
3  0x0000000000423927 in __sanitizer::StaticSpinMutex::Lock (this=0x57ef02 <__sanitizer::internal_allocator_cache_mu>) at sanitizer_mutex.h:32
4  0x000000000042354c in __sanitizer::GenericScopedLock<__sanitizer::StaticSpinMutex>::GenericScopedLock (this=this@entry=0x7ffcabfca0b8, mu=0x1) at sanitizer_mutex.h:367
5  0x0000000000423653 in __sanitizer::RawInternalAlloc (size=size@entry=72, cache=cache@entry=0x0, alignment=8, alignment@entry=0) at sanitizer_allocator.cpp:52
6  0x00000000004235e9 in __sanitizer::InternalAlloc (size=size@entry=72, cache=0x1, cache@entry=0x0, alignment=4, alignment@entry=0) at sanitizer_allocator.cpp:86
7  0x000000000043aa15 in __sanitizer::SymbolizedStack::New (addr=4802655) at sanitizer_symbolizer.cpp:45
8  0x000000000043b353 in __sanitizer::Symbolizer::SymbolizePC (this=0x7f578b77a028, addr=4802655) at sanitizer_symbolizer_libcdep.cpp:90
9  0x0000000000439dbe in __sanitizer::(anonymous namespace)::StackTraceTextPrinter::ProcessAddressFrames (this=this@entry=0x7ffcabfca208, pc=4802655) at sanitizer_stacktrace_libcdep.cpp:36
10 0x0000000000439c89 in __sanitizer::StackTrace::PrintTo (this=this@entry=0x7ffcabfca2a0, output=output@entry=0x7ffcabfca260) at sanitizer_stacktrace_libcdep.cpp:109
11 0x0000000000439fe0 in __sanitizer::StackTrace::Print (this=0x18) at sanitizer_stacktrace_libcdep.cpp:132
12 0x0000000000495359 in __sanitizer::PrintMutexPC (pc=4802656) at tsan_rtl.cpp:774
13 0x000000000042e0e4 in __sanitizer::InternalDeadlockDetector::Lock (this=0x7f578b1ca740, type=type@entry=2, pc=pc@entry=4371612) at sanitizer_mutex.cpp:177
14 0x000000000042df65 in __sanitizer::CheckedMutex::LockImpl (this=<optimized out>, pc=4) at sanitizer_mutex.cpp:218
15 0x000000000042bc95 in __sanitizer::CheckedMutex::Lock (this=0x600001000000) at sanitizer_mutex.h:127
16 __sanitizer::Mutex::Lock (this=0x600001000000) at sanitizer_mutex.h:165
17 0x000000000042b49c in __sanitizer::GenericScopedLock<__sanitizer::Mutex>::GenericScopedLock (this=this@entry=0x7ffcabfca370, mu=0x1) at sanitizer_mutex.h:367
18 0x000000000049504f in __tsan::TraceSwitch (thr=0x7f578b1ca980) at tsan_rtl.cpp:656
19 0x000000000049523e in __tsan_trace_switch () at tsan_rtl.cpp:683
20 0x0000000000499862 in __tsan::TraceAddEvent (thr=0x7f578b1ca980, fs=..., typ=__tsan::EventTypeMop, addr=4499472) at tsan_rtl.h:624
21 __tsan::MemoryAccessRange (thr=0x7f578b1ca980, pc=4499472, addr=135257110102784, size=size@entry=16, is_write=true) at tsan_rtl_access.cpp:563
22 0x000000000049853a in __tsan::MemoryRangeFreed (thr=thr@entry=0x7f578b1ca980, pc=pc@entry=4499472, addr=addr@entry=135257110102784, size=16) at tsan_rtl_access.cpp:487
23 0x000000000048f6bf in __tsan::OnUserFree (thr=thr@entry=0x7f578b1ca980, pc=pc@entry=4499472, p=p@entry=135257110102784, write=true) at tsan_mman.cpp:260
24 0x000000000048f61f in __tsan::user_free (thr=thr@entry=0x7f578b1ca980, pc=4499472, p=p@entry=0x7b0400004300, signal=true) at tsan_mman.cpp:213
25 0x000000000044a820 in __interceptor_free (p=0x7b0400004300) at tsan_interceptors_posix.cpp:708
26 0x00000000004ad599 in alloc_free_blocks () at fork_deadlock.cpp:25
27 __tsan_test_only_on_fork () at fork_deadlock.cpp:32
28 0x0000000000494870 in __tsan::ForkBefore (thr=0x7f578b1ca980, pc=pc@entry=4904437) at tsan_rtl.cpp:510
29 0x000000000046fcb4 in syscall_pre_fork (pc=1) at tsan_interceptors_posix.cpp:2577
30 0x000000000046fc9b in __sanitizer_syscall_pre_impl_fork () at sanitizer_common_syscalls.inc:3094
31 0x00000000004ad5f5 in myfork () at syscall.h:9
32 main () at fork_deadlock.cpp:46

Depends on D114595.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114597
2021-11-25 17:08:00 +01:00
Dmitry Vyukov b584741d06 tsan: fix Java heap block begin in reports
We currently use a wrong value for heap block
(only works for C++, but not for Java).
Use the correct value (we already computed it before, just forgot to use).

Depends on D114593.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114595
2021-11-25 17:07:53 +01:00
Dmitry Vyukov debac0ef37 tsan: add a benchmark for vector memory accesses
Depends on D114592.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114593
2021-11-25 17:07:46 +01:00
Dmitry Vyukov 5cac2b956b tsan: add a test for vector memory accesses
Add a basic test that checks races between vector/non-vector
read/write accesses of different sizes/offsets in different orders.
This gives coverage of __tsan_read/write16 callbacks.

Depends on D114591.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114592
2021-11-25 17:07:18 +01:00
Dmitry Vyukov d841086ae6 tsan: enable -msse4 when compiling tests
Vector SSE accesses make compiler emit __tsan_[unaligned_]read/write16 callbacks.
Make it possible to test these.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114591
2021-11-25 17:07:02 +01:00
Dimitry Andric 187d9979f2 cfi: fix more -Wformat warnings
Building cfi with recent clang on a 64-bit system results in the
following warnings:

    compiler-rt/lib/cfi/cfi.cpp:233:64: warning: format specifies type 'void *' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
        VReport(1, "Can not handle: symtab > strtab (%p > %zx)\n", symtab, strtab);
                                                     ~~            ^~~~~~
                                                     %lu
    compiler-rt/lib/sanitizer_common/sanitizer_common.h:231:46: note: expanded from macro 'VReport'
        if ((uptr)Verbosity() >= (level)) Report(__VA_ARGS__); \
                                                 ^~~~~~~~~~~
    compiler-rt/lib/cfi/cfi.cpp:253:59: warning: format specifies type 'void *' but the argument has type '__sanitizer::uptr' (aka 'unsigned long') [-Wformat]
        VReport(1, "Can not handle: symtab %p, strtab %zx\n", symtab, strtab);
                                           ~~                 ^~~~~~
                                           %lu
    compiler-rt/lib/sanitizer_common/sanitizer_common.h:231:46: note: expanded from macro 'VReport'
        if ((uptr)Verbosity() >= (level)) Report(__VA_ARGS__); \
                                                 ^~~~~~~~~~~

Since `__sanitizer::uptr` has the same size as `size_t`, consistently
use `%z` as a printf specifier.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114466
2021-11-24 21:48:17 +01:00
Kirill Stoimenov a489d6c5e0 [ASan] Moved __asan_test_only_reported_buggy_pointer to ReportGenericError, which is needed for shared optimzied callback tests.
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114486
2021-11-24 09:54:13 -08:00
Dmitry Vyukov 6f85d68e6a tsan: include internal allocator into deadlock detection
Now that we lock the internal allocator around fork,
it's possible it will create additional deadlocks.
Add a fake mutex that substitutes the internal allocator
for the purposes of deadlock detection.

Depends on D114531.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114532
2021-11-24 17:01:57 +01:00
Dmitry Vyukov 348389263c tsan: lock internal allocator around fork
There is a small chance that the internal allocator is locked
during fork and then the new process is created with locked
internal allocator and any attempts to use it will deadlock.
For example, if detected a suppressed race in the parent during fork
and then another suppressed race after the fork.
This becomes much more likely with the new tsan runtime
as it uses the internal allocator for more things.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114531
2021-11-24 17:01:28 +01:00
Dmitry Vyukov a68b52e0a3 tsan: add another fork deadlock test
The test tries to provoke internal allocator to be locked during fork
and then force the child process to use the internal allocator.
This test sometimes deadlocks with the new tsan runtime.

Depends on D114514.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114515
2021-11-24 13:25:53 +01:00
Dmitry Vyukov 06677d6a9f sanitizer_common: remove SANITIZER_USE_MALLOC
It was introduced in:
9cffc9550b tsan: allow to force use of __libc_malloc in sanitizer_common
and used in:
512a18e518 tsan: add standalone deadlock detector
and later used for Go support.
But now both uses are gone. Nothing defines SANITIZER_USE_MALLOC.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114514
2021-11-24 13:24:19 +01:00
Dmitry Vyukov 764b35d89f tsan: extend mmap test
Test size larger than clear_shadow_mmap_threshold,
which is handled differently.

Depends on D114348.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114366
2021-11-24 10:57:21 +01:00
Vitaly Buka b9fd7247a7 [sanitizer] Add Abs<T> 2021-11-23 22:25:36 -08:00
Vitaly Buka 55792b5ac4 [sanitizer] Fail instead of crash without real_pthread_create 2021-11-23 20:32:09 -08:00
Weverything 1150f02c77 Revert "tsan: new runtime (v3)"
This reverts commit ebd47b0fb7.
This was causing unexpected behavior in programs.
2021-11-23 18:32:32 -08:00
Vitaly Buka 6889592ebc [NFC][sanitizer] Limit StackStore stack size/tag to 1 byte
Nothing uses more than 8bit now. So the rest of the headers can store other data.
kStackTraceMax is 256 now, but all sanitizers by default store just 20-30 frames here.
2021-11-23 16:56:34 -08:00
Vitaly Buka 402a406323 [NFC][sanitizer] Test for b80affb8a1 2021-11-23 16:56:24 -08:00
Florian Mayer 26d1edfb10 [hwasan] support python3 in hwasan_sanitize
Verified no diff exist between previous version, new version python 2, and python 3 for an example stack.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D114404
2021-11-23 15:30:30 -08:00
Vitaly Buka 4058637f7a [NFC][sanitizer] Reuse forEach for operator== 2021-11-23 15:23:25 -08:00
Vitaly Buka 09256fe980 [sanitizer] Add DenseMap::forEach 2021-11-23 15:23:25 -08:00
Vitaly Buka b1a68b170c [NFC][sanitizer] Make method const 2021-11-23 13:50:07 -08:00
Vitaly Buka abd86619cf [NFC][sanitizer] Extract StackTraceHeader struct 2021-11-23 13:50:06 -08:00
Vitaly Buka 67a1c45def [NFC][sanitizer] Add StackStoreTest
Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114463
2021-11-23 13:02:11 -08:00
Vitaly Buka b80affb8a1 [NFC][sanitizer] Early return for empty StackTraces
Current callers should filter them out anyway,
but with this patch we don't need rely on that assumption.
2021-11-23 12:53:54 -08:00
Vitaly Buka cdc80a6017 [NFC][sanitizer] Move StackStore::Allocated into cpp file 2021-11-23 12:46:19 -08:00
Pirama Arumuga Nainar 1df7289af6 [compiler-rt/profile] Include __llvm_profile_get_magic in module signature
The INSTR_PROF_RAW_MAGIC_* number in profraw files should match during
profile merging.  This causes an error with 32-bit and 64-bit variants
of the same code.  The module signatures for the two binaries are
identical but they use different INSTR_PROF_RAW_MAGIC_* causing a
failure when profile-merging is used.  Including it when computing the
module signature yields different signatures for the 32-bit and 64-bit
profiles.

Differential Revision: https://reviews.llvm.org/D114054
2021-11-23 10:03:43 -08:00
Dmitry Vyukov d75ed9864a tsan: disable signal_sync2.cpp test on powerpc64
Fails 1 out of 10 runs on powerpc bots:
https://lab.llvm.org/buildbot/#/builders/121/builds/13391

Reviewed By: nemanjai

Differential Revision: https://reviews.llvm.org/D114426
2021-11-23 17:58:26 +01:00
Dmitry Vyukov ebd47b0fb7 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Differential Revision: https://reviews.llvm.org/D112603
2021-11-23 11:44:59 +01:00
Snehasish Kumar a4b92d6158 [memprof] Remove the "Live on exit:" print for text format.
We dropped the printing of live on exit blocks in rG1243cef245f6 -
the commit changed the insertOrMerge logic. Remove the message since it
is no longer needed (all live blocks are inserted into the hashmap)
before serializing/printing the profile. Furthermore, the original
intent was to capture evicted blocks so it wasn't entirely correct.

Also update the binary format test invocation to remove the redundant
print_text directive now that it is the default.

Differential Revision: https://reviews.llvm.org/D114285
2021-11-22 13:30:48 -08:00
Matt Morehouse 615ecd8afc [HWASan] Move LTO test to separate file.
The test fails on Android for an unknown reason but is still worth
having for x86.
2021-11-22 13:19:11 -08:00
Haowei Wu 692131f41a [compiler-rt] Explicitly set dependency on libcxx for MemProfUnitTest
MemProfUnitTest now depends on libcxx but the dependency is not
explicitly expressed in build system, causing build races. This patch
addresses this issue.

Differential Revision: https://reviews.llvm.org/D114267
2021-11-22 12:47:12 -08:00
Matt Morehouse 5684a614dd [HWASan] Remove -Wa,-mrelax-relocations=no flag.
We no longer need it after https://reviews.llvm.org/D113220.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113221
2021-11-22 11:21:27 -08:00
Manoj Gupta 2e67276d98 libfuzzer: Disable broken tests for arm
libfuzzer was recently enabled for Arm32 in D112091.
A few tests apparently do not work with arm32 so disable them.
The list of tests was obtained from
https://lab.llvm.org/buildbot/#/builders/190/builds/513

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114312
2021-11-22 10:33:14 -08:00
Dmitry Vyukov 5f18ae3988 Revert "tsan: new runtime (v3)"
Summary:
This reverts commit 1784fe0532.

Broke some bots:
https://lab.llvm.org/buildbot#builders/57/builds/12365
http://green.lab.llvm.org/green/job/clang-stage1-RA/25658/

Reviewers: vitalybuka, melver

Subscribers:
2021-11-22 19:08:48 +01:00
Florian Mayer ed8b5b37ab [hwasan] fix arguments to symbolizer.
new versions do not accept -inlining of -functions (tested with 11 and
13).

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D114303
2021-11-22 09:59:06 -08:00
Dmitry Vyukov 1784fe0532 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Depends on D112602.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112603
2021-11-22 15:55:39 +01:00
Dmitry Vyukov e69d50d9ff tsan: disable instrumentation in runtime callbacks in tests
All runtime callbacks must be non-instrumented with the new tsan runtime
(it's now more picky with respect to recursion into runtime).
Disable instrumentation in Darwin tests as we do in all other tests now.

Differential Revision: https://reviews.llvm.org/D114348
2021-11-22 15:48:29 +01:00
Dmitry Vyukov 6a3958247a tsan: add another fork test
Add a fork test that models what happens on Mac
where fork calls malloc/free inside of our atfork
callbacks.

Reviewed By: vitalybuka, yln

Differential Revision: https://reviews.llvm.org/D114250
2021-11-22 08:36:51 +01:00
Dimitry Andric 7dbbb5d3a4 compiler-rt: Use FreeBSD's elf_aux_info to detect AArch64 HW features
Using the out-of-line LSE atomics helpers for AArch64 on FreeBSD also
requires adding support for initializing __aarch64_have_lse_atomics
correctly. On Linux this is done with getauxval(3), on FreeBSD with
elf_aux_info(3), which has a slightly different interface.

Differential Revision: https://reviews.llvm.org/D109330
2021-11-20 12:12:03 +01:00
Vitaly Buka d591a46d17 [NFC][sanitizer] Fix naming in StackStore 2021-11-18 19:39:20 -08:00
Vitaly Buka 8210948a46 [NFC][sanitizer] constexpr StackStore::StackStore() 2021-11-18 19:24:29 -08:00
Vitaly Buka bb50e97103 [NFC][sanitizer] Change StackStore API to use StackTrace 2021-11-18 18:53:51 -08:00
Vitaly Buka 6bf71be9f9 [NFC][sanitizer] Move inline implementation of StackStore into cpp 2021-11-18 18:38:06 -08:00
Vitaly Buka 368a3b5221 [NFC][sanitizer] Rename PersistentAllocator to StackStore 2021-11-18 18:37:51 -08:00
Vitaly Buka b6169e231e [nfc][dfsan] Remove obsolete comment 2021-11-18 18:37:13 -08:00
Vitaly Buka 930ccf0191 [NFC][sanitizer] Rename persistent_allocator to stack_store 2021-11-18 18:36:58 -08:00
Vitaly Buka b0ea7ef0d6 [msan] Clang-format the test 2021-11-18 18:30:49 -08:00
Manoj Gupta 2782cb8da0 libfuzzer: All building libfuzzer for ARM32
We need libfuzzer libraries on Arm32 so that we can fuzz
Arm32 binaries on Linux (Chrome OS). Android already
allows Arm32 for libfuzzer.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D112091
2021-11-18 17:30:24 -08:00
Dmitry Vyukov 111d8f785b tsan: remove quadratic behavior in pthread_join
pthread_join needs to map pthread_t of the joined thread to our Tid.
Currently we do this with linear search over all threads.
This has quadratic complexity and becomes much worse with the new
tsan runtime, which memorizes all threads that ever existed.

To resolve this add a hash map of live threads only (that are still
associated with pthread_t) and use it for the mapping.

With the new tsan runtime some programs spent 1/3 of time in this mapping.
After this change the mapping disappears from profiles.

Depends on D113996.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D113997
2021-11-17 19:41:49 -08:00
Vitaly Buka c26dbc4ab4 [sanitizer] Fix DenseMap for compiler-rt
Depends on D114047.

Differential Revision: https://reviews.llvm.org/D114048
2021-11-17 19:10:20 -08:00
Vitaly Buka 7612cfd31d [NFC][sanitizer] Fix headers of DenseMap
Depends on D114046.

Differential Revision: https://reviews.llvm.org/D114047
2021-11-17 19:10:20 -08:00
Vitaly Buka 234a8301cd [NFC][sanitizer] Clang format copied code
Depends on D114045.

Differential Revision: https://reviews.llvm.org/D114046
2021-11-17 19:10:20 -08:00
Vitaly Buka 54adc1675e [NFC][sanitizer] Add unchanged DenseMap
It's just a copy even without reformatting.

Reviewed By: dvyukov, melver

Differential Revision: https://reviews.llvm.org/D114045
2021-11-17 19:10:19 -08:00
Vitaly Buka 254f9311fe [NFC][sanitizer] Fix veradic-macro warning in RAW_CHECK 2021-11-17 18:10:54 -08:00
Vitaly Buka 02eca53a50 [sanitizer] Add a few of type_traits tools
For D114047
2021-11-17 17:52:40 -08:00
Leonard Chan 1a84d1c81e [compiler-rt][asan] Re-add `self`
We ran into errors where this wasn't defined in Fuchsia's asan implementation.
2021-11-17 15:48:41 -08:00
Vitaly Buka 22e66a97cb Revert "[sanitizer] Add a few of type_traits tools"
Does not work with GCC

This reverts commit a82ee2be9c.
2021-11-17 15:46:16 -08:00
Vitaly Buka a82ee2be9c [sanitizer] Add a few of type_traits tools
For D114047
2021-11-17 15:06:38 -08:00
Jonas Paulsson 4c32e3d967 [SystemZ] [Sanitizer] Bugfixes in internal_clone().
The __flags variable needs to be of type 'long' in order to get sign extended
properly.

internal_clone() uses an svc (Supervisor Call) directly (as opposed to
internal_syscall), and therefore needs to take care to set errno and return
-1 as needed.

Review: Ulrich Weigand
2021-11-17 14:42:08 -05:00
Dmitry Vyukov bdabf3c38a asan: don't use thread user_id
asan does not use user_id for anything,
so don't pass it to ThreadCreate.
Passing a random uninitialized field of AsanThread
as user_id does not make much sense anyway.

Depends on D113921.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113922
2021-11-17 14:49:33 +01:00
Dmitry Vyukov d1f72f02d0 memprof: don't use thread user_id
memprof does not use user_id for anything,
so don't pass it to ThreadCreate.
Passing a random field of MemprofThread as user_id
does not make much sense anyway.

Depends on D113920.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113921
2021-11-17 14:49:25 +01:00
Dmitry Vyukov 033b94ab52 lsan: remove pthread_detach/join interceptors
They don't seem to do anything useful in lsan.
They are needed only if a tools needs to execute
some custom logic during detach/join, or if it uses
thread registry quarantine. Lsan does none of this.
And if a tool cares then it would also need to intercept
pthread_tryjoin_np and pthread_timedjoin_np, otherwise
it will mess thread states.
Fwiw, asan does not intercept these functions either.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113920
2021-11-17 14:49:15 +01:00
Vitaly Buka 82fc4cc60b [scudo] Handle mallinfo2
mallinfo is deprecated by GLIBC

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D113951
2021-11-16 19:29:37 -08:00
Evgenii Stepanov 913d78c40c [scudo] Regression test for the MTE crash in storeEndMarker.
The original problem was fixed in D105261.

Differential Revision: https://reviews.llvm.org/D114022
2021-11-16 13:43:15 -08:00
Pirama Arumuga Nainar 078279ff01 [compiler-rt/profile] Reland mark __llvm_profile_raw_version as hidden
Since libclang_rt.profile is added later in the command line, a
definition of __llvm_profile_raw_version is not included if it is
provided from an earlier object, e.g.  from a shared dependency.

This causes an extra dependence edge where if libA.so depends on libB.so
and both are coverage-instrumented, libA.so uses libB.so's definition of
__llvm_profile_raw_version.  This leads to a runtime link failure if the
libB.so available at runtime does not provide this symbol (but provides
the other dependent symbols).  Such a scenario can occur in Android's
mainline modules.
E.g.:
  ld -o libB.so libclang_rt.profile-x86_64.a
  ld -o libA.so -l B libclang_rt.profile-x86_64.a

libB.so has a global definition of __llvm_profile_raw_version.  libA.so
uses libB.so's definition of __llvm_profile_raw_version.  At runtime,
libB.so may not be coverage-instrumented (i.e. not export
__llvm_profile_raw_version) so runtime linking of libA.so will fail.

Marking this symbol as hidden forces each binary to use the definition
of __llvm_profile_raw_version from libclang_rt.profile.  The visiblity
is unchanged for Apple platforms where its presence is checked by the
TAPI tool.

Reviewed By: MaskRay, phosek, davidxl

Differential Revision: https://reviews.llvm.org/D111759
2021-11-16 11:37:00 -08:00
Dmitry Vyukov d0c138ec8a tsan: disable bench_threads.cpp on aarch64
The new test started failing on bots with:

CHECK failed: tsan_rtl.cpp:327 "((addr + size)) <= ((TraceMemEnd()))"
   (0xf06200e03010, 0xf06200000000) (tid=4073872)

https://lab.llvm.org/buildbot#builders/179/builds/1761

This is a latent bug in aarch64 virtual address space layout,
there is not enough address space to fit traces for all threads.
But since the trace space is going away with the new tsan runtime
(D112603), disable the test.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D113990
2021-11-16 16:53:04 +01:00
Dmitry Vyukov c7081b5b4c tsan: fix crash during thread exit
Use of gethostent provokes caching of some resources inside of libc.
They are freed in __libc_thread_freeres very late in thread lifetime,
after our ThreadFinish. __libc_thread_freeres calls free which
previously crashed in malloc hooks.
Fix it by setting ignore_interceptors for finished threads,
which in turn prevents malloc hooks.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D113989
2021-11-16 16:43:09 +01:00
Dmitry Vyukov c971f989ee tsan: de-hardcode number of unused bits in trace events
Precisely specifying the unused parts of the bitfield is critical for
performance. If we don't specify them, compiler will generate code to load
the old value and shuffle it to extract the unused bits to apply to the new
value. If we specify the unused part and store 0 in there, all that
unnecessary code goes away (store of the 0 const is combined with other
constant parts).

I don't see a good way to ensure we cover all of u64 bits with fields.
So at least introduce named kUnusedBits consts and check that bits
sum up to 64.

Depends on D113978.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D113979
2021-11-16 16:00:14 +01:00
Dmitry Vyukov fa24d58279 tsan: use smaller trace parts for Go
In the old runtime we used to use different number of trace parts
for C++ and Go to reduce trace memory consumption for Go.
But now it's easier and better to use smaller parts because
we already use minimal possible number of parts for C++ (3).

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D113978
2021-11-16 15:59:33 +01:00
Dmitry Vyukov 87261caa55 tsan: use pthread_equal instead of direct pthread_t comparison
man pthread_equal:
  The pthread_equal() function is necessary because thread IDs
  should be considered opaque: there is no portable way for
  applications to directly compare two pthread_t values.

Depends on D113916.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113919
2021-11-16 07:51:24 +01:00
Dmitry Vyukov 64b45399e5 tsan: speed up pthread_setname_np
pthread_setname_np does linear search over all thread descriptors
to map pthread_t to the thread descriptor. This has O(N^2) complexity
and becomes much worse in the new tsan runtime that keeps all ever
existed threads in the thread registry.
Replace linear search with direct access if pthread_setname_np
is called for the current thread (a very common case).

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113916
2021-11-16 07:51:08 +01:00
ZijunZhao d2b43605c9 add tsan shared lib
Change-Id: Ic83ff1ec86d6a7d61b07fa3df7e0cb2790b5ebc7
2021-11-16 00:42:30 +00:00
Vitaly Buka 118757af11 [msan] Disabled test failing on new GLIBC 2021-11-15 11:18:52 -08:00
Matt Kulukundis 2d9bdd9dba Fix a deadlock in __cxa_guard_abort in tsan
hat tip: @The_Whole_Daisy for helping to isolate

Reviewed By: dvyukov, fowles

Differential Revision: https://reviews.llvm.org/D113713
2021-11-15 10:39:08 +01:00
Vitaly Buka cda72e9297
[msan] Fix test with GLIBC 2.34
PTHREAD_STACK_MIN is not a constexpr
2021-11-14 20:15:45 -08:00
Vitaly Buka 846f33572e [sanitizer] Disable test incompartible with recert GLIBC 2021-11-14 16:25:29 -08:00
Vitaly Buka dd63c6e6b9 [sanitizer] Fix test with GLIBC 2.34 2021-11-14 11:30:50 -08:00
Vitaly Buka 7e980c46ed [sanitizer] Another try to fix the test with GLIBC 2.34 2021-11-14 00:32:23 -08:00
hyeongyu kim 7f7cab6bb1 [sanitizer][aarch64] fix clone system call's inline assembly
Return value of the system call was not returned normally.
It was discussed at https://reviews.llvm.org/D105169.
2021-11-14 09:45:40 +09:00
Vitaly Buka e56d680fe8 [sanitizer] Fix test for GLIBC 2.31
Newer GLIBC uses sysconf to get SIGSTKSZ.
2021-11-13 14:28:38 -08:00
Dmitry Vyukov b5ff187b7b tsan: mmap shadow stack
We used to mmap C++ shadow stack as part of the trace region
before ed7f3f5bc9 ("tsan: move shadow stack into ThreadState"),
which moved the shadow stack into TLS. This started causing
timeouts and OOMs on some of our internal tests that repeatedly
create and destroy thousands of threads.
Allocate C++ shadow stack with mmap and small pages again.
This prevents the observed timeouts and OOMs.
But we now need to be more careful with interceptors that
run after thread finalization because FuncEntry/Exit and
TraceAddEvent all need the shadow stack.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113786
2021-11-13 09:12:55 +01:00
Vitaly Buka dd87c5b322 Revert "[sanitizer] Fix test linking"
This reverts commit afafa883a4.

-pthread was not the fix. Symbols removed from GLIBC 2.34
Fixed with e60b3fcefa.
2021-11-12 23:42:35 -08:00
Vitaly Buka e60b3fcefa [sanitizer] Don't test __pthread_mutex_lock with GLIBC 2.34 2021-11-12 23:42:04 -08:00
Vitaly Buka 89fb2c71a2 [asan] More leaks in test
It fails to detect a single leak with GLIBC 2.34.
2021-11-12 17:59:14 -08:00
Vitaly Buka 4b768eeb97 [asan] Fix "no matching function" on GCC 2021-11-12 17:42:25 -08:00
Vitaly Buka afafa883a4 [sanitizer] Fix test linking 2021-11-12 16:52:58 -08:00
Ben Langmuir 2a739f2789 [ORC][ORC-RT] Register type metadata from __swift5_types MachO section
Similar to how the other swift sections are registered by the ORC
runtime's macho platform, add the __swift5_types section, which contains
type metadata. Add a simple test that demonstrates that the swift
runtime recognized the registered types.

rdar://85358530

Differential Revision: https://reviews.llvm.org/D113811
2021-11-12 16:39:59 -08:00
Vitaly Buka cb0e14ce6d [sanitizer] Switch dlsym hack to internal_allocator
Since glibc 2.34, dlsym does
  1. malloc 1
  2. malloc 2
  3. free pointer from malloc 1
  4. free pointer from malloc 2
These sequence was not handled by trivial dlsym hack.

This fixes https://bugs.llvm.org/show_bug.cgi?id=52278

Reviewed By: eugenis, morehouse

Differential Revision: https://reviews.llvm.org/D112588
2021-11-12 16:11:10 -08:00
Vitaly Buka 07092ea6bd [asan] Fix GCC warning "left shift count >= width"
Fixes PR52385
2021-11-12 13:04:00 -08:00
Lang Hames 3fb641618f [ORC-RT][llvm-jitlink] Fix a buggy check in ORC-RT MachO TLV deregistration.
The check was failing because it was matching against the end of the range, not
the start.

This bug wasn't causing the ORC-RT MachO TLV regression test to fail because
we were only logging deallocation errors (including TLV deregistration errors)
and not actually returning a failure code. This commit updates llvm-jitlink to
report the errors properly.
2021-11-12 10:36:17 -08:00
Dmitry Vyukov 79fbba9b79 Revert "tsan: new runtime (v3)"
Summary:
This reverts commit ac95b8d954.
There is a number of bot failures:
http://45.33.8.238/mac/38755/step_4.txt
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/38135/consoleFull#-148886289949ba4694-19c4-4d7e-bec5-911270d8a58c

Reviewers: vitalybuka, melver

Subscribers:
2021-11-12 17:49:47 +01:00
Dmitry Vyukov ac95b8d954 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Depends on D112602.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112603
2021-11-12 14:31:49 +01:00
Dmitry Vyukov 19c1d03f97 tsan: ignore some errors in the clone_setns test
Some bots failed with:
unshare failed: 1
https://lab.llvm.org/buildbot/#/builders/70/builds/14101

Look only for the target EINVAL error.

Differential Revision: https://reviews.llvm.org/D113759
2021-11-12 14:12:36 +01:00
Dmitry Vyukov e91595bf94 tsan: don't start background thread after clone
Start the background thread only after fork, but not after clone.
For fork we did this always and it's known to work (or user code has adopted).
But if we do this for the new clone interceptor some code (sandbox2) fails.
So model we used to do for years and don't start the background thread after clone.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D113744
2021-11-12 12:58:49 +01:00
Dmitry Vyukov a6728382c6 tsan: fix XMM register corruption in hacky call
The compiler does not recognize HACKY_CALL as a call
(we intentionally hide it from the compiler so that it can
compile non-leaf functions as leaf functions).
To compensate for that hacky call thunk saves and restores
all caller-saved registers. However, it saves only
general-purposes registers and does not save XMM registers.
This is a latent bug that was masked up until a recent "NFC" commit
d736002e90 ("tsan: move memory access functions to a separate file"),
which allowed more inlining and exposed the 10-year bug.
Save and restore caller-saved XMM registers (all) as well.

Currently the bug manifests as e.g. frexp interceptor messes the
return value and the added test fails with:
  i=8177 y=0.000000 exp=4

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D113742
2021-11-12 12:53:47 +01:00
Vitaly Buka 64d4420824 [NFC][lsan] Simplify root_regions initialization 2021-11-11 23:42:46 -08:00
Simon Moll 496e7f330c [VE] Disable incompatible compiler-rt tests
Some compiler-rt tests are inherently incompatible with VE because..

* No consistent denormal support on VE. We skip denormal fp inputs in builtin tests.
* `madvise` unsupported on VE.
* Instruction alignment requirements.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D113093
2021-11-12 08:40:03 +01:00
Snehasish Kumar aacaebc6c2 [memprof] Make the raw binary format the default.
Set the default memprof serialization format as binary. 9 tests are
updated to use print_text=true. Also fixed an issue with concatenation
of default and test specified options (missing separator).

Differential Revision: https://reviews.llvm.org/D113617
2021-11-11 11:29:36 -08:00
Snehasish Kumar 545866cb05 [memprof] Add a raw binary format to serialize memprof profiles.
This change implements the raw binary format discussed in
https://lists.llvm.org/pipermail/llvm-dev/2021-September/153007.html

Summary of changes
* Add a new memprof option to choose binary or text (default) format.
* Add a rawprofile library which serializes the MIB map to profile.
* Add a unit test for rawprofile.
* Mark sanitizer procmaps methods as virtual to be able to mock them.
* Extend memprof_profile_dump regression test.

Differential Revision: https://reviews.llvm.org/D113317
2021-11-11 11:29:36 -08:00
Snehasish Kumar 1243cef245 [memprof] Replace the block cache with a hashmap.
The existing implementation uses a cache + eviction based scheme to
record heap profile information. This design was adopted to ensure a
constant memory overhead (due to fixed number of cache entries) along
with incremental write-to-disk for evictions. We find that since the
number to entries to track is O(unique-allocation-contexts) the overhead
of keeping all contexts in memory is not very high. On a clang workload,
the max number of unique allocation contexts was ~35K, median ~11K.
For each context, we (currently) store 64 bytes of data - this amounts
to 5.5MB (max). Given the low overheads for a complex workload, we can
simplify the implementation by using a hashmap without eviction.

Other changes:
* Memory map is dumped at the end rather than startup. The relative
order in the profile dump is unchanged since we no longer have evicted
entries at runtime.
* Added a test to check meminfoblocks are merged.

Differential Revision: https://reviews.llvm.org/D111676
2021-11-11 11:29:36 -08:00
Snehasish Kumar fc7162414e [memprof] Move the MemInfoBlock definition to a separate header.
Move the memprof MemInfoBlock struct to it's own header as requested
during the review of D111676.

Differential Revision: https://reviews.llvm.org/D113315
2021-11-11 11:29:36 -08:00
Snehasish Kumar d19470540a [sanitizer] Add a ForEach callback interface for AddrHashMap.
This change adds a ForEach method to the AddrHashMap class which can
then be used to iterate over all the key value pairs in the hash map.
I intend to use this in an upcoming change to the memprof runtime.

Added a unit test to cover basic insertion and the ForEach callback.

Differential Revision: https://reviews.llvm.org/D111368
2021-11-11 11:29:36 -08:00
Dmitry Vyukov bc84b2857f tsan: enable clone interceptor only on Linux
Clone does not exist on Mac.
There are chances it will break on other OSes.
Enable it incrementally starting with Linux only,
other OSes can enable it later as needed.

Reviewed By: melver, thakis

Differential Revision: https://reviews.llvm.org/D113693
2021-11-11 19:27:47 +01:00
Dmitry Vyukov 82de586d4b tsan: intercept clone
gtest uses clone for death tests and it needs the same
handling as fork to prevent deadlock (take runtime mutexes
before and release them after).

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D113677
2021-11-11 18:55:54 +01:00
Vitaly Buka 3330a6670e [sanitizer] Fix solaris build 2021-11-10 20:41:46 -08:00
Emily Shi 919ca9fc04 [compiler-rt] separate lsan, ubsan, stats configuration from SANITIZER_COMMON_SUPPORTED_OS
Currently, SANITIZER_COMMON_SUPPORTED_OS is being used to enable many libraries.
Unfortunately this makes it impossible to selectively disable a library based on the OS.
This patch removes this limitation by adding a separate list of supported OSs for the lsan, ubsan, ubsan_minimal, and stats libraries.

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D113444
2021-11-10 19:16:20 -08:00
Vitaly Buka aa15e426ea [NFC][msan] Clang format includes 2021-11-10 18:17:09 -08:00
Vitaly Buka 1da33a51f1 [NFC][asan][memprov] Remove dlsym hack from posix_memalign
It was added for RTEMS which was removed at D104279.
2021-11-10 16:35:14 -08:00
Vitaly Buka 651797f488 [NFC][sanitizer] Move GET_MALLOC_STACK_TRACE closer to the use 2021-11-10 15:42:07 -08:00
Vitaly Buka be60b6aac5 [NFC][sanitizer] Make const PointerIsMine and FromPrimary 2021-11-10 15:22:29 -08:00
MarcoFalke fa1729067c
[compiler-rt] Fix typo in DeadlockDetector (chanding->changing) 2021-11-10 21:51:47 +01:00
Dokyung Song 614b8bac01 [libFuzzer] Deflake entropic exec-time test.
Entropic scheduling with exec-time option can be misled, if inputs
on the right path to become crashing inputs accidentally take more
time to execute before it's added to the corpus. This patch, by letting
more of such inputs added to the corpus (four inputs of size 7 to 10,
instead of a single input of size 2), reduces possibilities of being
influenced by timing flakiness.

A longer-term fix could be to reduce timing flakiness in the fuzzer;
one way could be to execute inputs multiple times and take average of
their execution time before they are added to the corpus.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D113544
2021-11-10 07:43:21 -08:00
Vitaly Buka ffd9c123e7 [dfsan] Dfsan version of D113328
Depends on D113328.

Differential Revision: https://reviews.llvm.org/D113454
2021-11-09 18:23:55 -08:00
Vitaly Buka f2c2292fa8 [msan] Block signals in MsanThread::Init
If async signal handler called when we MsanThread::Init
signal handler may trigger false reports.
I failed to reproduce this locally for a test.

Differential Revision: https://reviews.llvm.org/D113328
2021-11-09 18:23:55 -08:00
Vitaly Buka 84b8dc710c [NFC][sanitizer] Extract ScopedBlockSignals
Differential Revision: https://reviews.llvm.org/D113452
2021-11-09 18:23:55 -08:00
Kostya Serebryany b7f3a4f4fa [sancov] add tracing for loads and store
add tracing for loads and stores.

The primary goal is to have more options for data-flow-guided fuzzing,
i.e. use data flow insights to perform better mutations or more agressive corpus expansion.
But the feature is general puspose, could be used for other things too.

Pipe the flag though clang and clang driver, same as for the other SanitizerCoverage flags.
While at it, change some plain arrays into std::array.

Tests: clang flags test, LLVM IR test, compiler-rt executable test.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D113447
2021-11-09 14:35:13 -08:00
zijunzhao 45d28e3a30 Revert "add tsan shared lib"
This reverts commit 92c9b340be.
2021-11-09 16:36:06 +00:00
Matt Morehouse 846ec2c3cc [HWASan] Ensure RNG is initialized in GenerateRandomTag
Fixes a CHECK-failure caused by glibc's pthread_getattr_np
implementation calling realloc.  Essentially, Thread::GenerateRandomTag
gets called during Thread::Init and before Thread::InitRandomState:

  HWAddressSanitizer: CHECK failed: hwasan_thread.cpp:134 "((random_buffer_)) != (0)" (0x0, 0x0) (tid=314)
    #0 0x55845475a662 in __hwasan::CheckUnwind()
    #1 0x558454778797 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long)
    #2 0x558454766461 in __hwasan::Thread::GenerateRandomTag(unsigned long)
    #3 0x55845475c58b in __hwasan::HwasanAllocate(__sanitizer::StackTrace*, unsigned long, unsigned long, bool)
    #4 0x55845475c80a in __hwasan::hwasan_realloc(void*, unsigned long, __sanitizer::StackTrace*)
    #5 0x5584547608aa in realloc
    #6 0x7f6f3a3d8c2c in pthread_getattr_np
    #7 0x5584547790dc in __sanitizer::GetThreadStackTopAndBottom(bool, unsigned long*, unsigned long*)
    #8 0x558454779651 in __sanitizer::GetThreadStackAndTls(bool, unsigned long*, unsigned long*, unsigned long*, unsigned long*)
    #9 0x558454761bca in __hwasan::Thread::InitStackAndTls(__hwasan::Thread::InitState const*)
    #10 0x558454761e5c in __hwasan::HwasanThreadList::CreateCurrentThread(__hwasan::Thread::InitState const*)
    #11 0x55845476184f in __hwasan_thread_enter
    #12 0x558454760def in HwasanThreadStartFunc(void*)
    #13 0x7f6f3a3d6fa2 in start_thread
    #14 0x7f6f3a15b4ce in __clone

Also reverts 7a3fb71c3c, as it's now
unneeded.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D113045
2021-11-09 07:01:55 -08:00
Djordje Todorovic 2ca1cd7e37 [cmake] By default do not instrument compiler-rt if LLVM_BUILD_INSTRUMENTED_COVERAGE is ON
Applying the same rules as for LLVM_BUILD_INSTRUMENTED build in the cmake files.
By having this patch, we are able to disable/enable instrument+coverage build
of the compiler-rt project when building instrumented LLVM.

Differential Revision: https://reviews.llvm.org/D108127
2021-11-09 10:59:25 +01:00
Vitaly Buka 63886c21ec [NFC][dfsan] Split Init and ThreadStart 2021-11-08 19:16:55 -08:00
Vitaly Buka a5b06ad39a Partially revert "[NFC][msan] Split ThreadStart and Init"
I don't know if removing "if (!start_routine_)" from ThreadStart
is NFC.

This reverts commit b3267bb3af.
2021-11-08 19:16:55 -08:00
Vitaly Buka b3267bb3af [NFC][msan] Split ThreadStart and Init 2021-11-08 18:58:33 -08:00
ZijunZhao 92c9b340be add tsan shared lib
Change-Id: Ic83ff1ec86d6a7d61b07fa3df7e0cb2790b5ebc7
2021-11-09 02:34:37 +00:00
Vitaly Buka 33af9a31c0 [NFC] Move setuid.c test into sanitizer_common 2021-11-08 18:24:01 -08:00
David CARLIER 443820179a [compiler-rt] TlsBaseAddr value for darwin arm64
getting the tls base address. unlike linux arm64, the tpidr_el0 returns always 0 (aka unused)
thus using tpidrro_el0 instead clearing up the cpu id encoded in the lower bits.

Reviewed-By: yln

Differential Revision: https://reviews.llvm.org/D112866
2021-11-08 22:26:32 +00:00
Snehasish Kumar 9305e3b6d7 [NFC] Update the test compiler to use clangxx.
This ensures that the c++ test gets the right CXXFLAGS if required.
2021-11-08 13:57:32 -08:00
hyeongyu kim fd9b099906 Revert "[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default"
This reverts commit aacfbb953e.

Revert "Fix lit test failures in CodeGenCoroutines"

This reverts commit 63fff0f5bf.
2021-11-09 02:15:55 +09:00
Manoj Gupta db27867dfc [compiler-rt] Produce the right arch suffix for arm baremetal
D98452 introduced a mismatch between clang expectations for
builtin name for baremetal targets on arm. Fix it by
adding a case for baremetal. This now matches the output of
"clang -target armv7m-none-eabi -print-libgcc-file-name \
-rtlib=compiler-rt"

Reviewed By: mstorsjo

Differential Revision: https://reviews.llvm.org/D113357
2021-11-07 08:51:35 -08:00
Fangrui Song 70986ea3d6 [sanitizer][aarch64] Add cast to drop reliance on the type of uc_mcontext.__reserved
https://sourceware.org/bugzilla/show_bug.cgi?id=22742
uc_mcontext.__reserved probably should not be considered user visible API but
unfortunate it is: it is the only way to access cpu states of some Linux
asm/sigcontext.h extensions. That said, the declaration may be
long double __reserved[256]; (used by musl)
instead of
unsigned char __reserved[4096] __attribute__((__aligned__(16))); (glibc)
to avoid dependency on a GNU variable attribute.
2021-11-06 23:26:05 -07:00
Fangrui Song 815b9f53d8 [hwasan] Replace _Unwind_Word with uintptr_t
GCC introduced `__attribute__((mode(unwind_word)))` to work around
Cell Broadband Engine SPU (which was removed from GCC in 2019-09),
which is irrelevant to hwasan.
_Unwind_GetGR/_Unwind_GetCFA from llvm-project/libunwind don't use unwind_word.

Using _Unwind_Word can lead to build failures if libunwind's unwind.h is
preferred over unwind.h in the Clang resource directory (e.g. built with GCC).
2021-11-06 22:34:50 -07:00
hyeongyukim aacfbb953e [Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default
Turning on `enable_noundef_analysis` flag allows better codegen by removing freeze instructions.
I modified clang by renaming `enable_noundef_analysis` flag to `disable-noundef-analysis` and turning it off by default.

Test updates are made as a separate patch: D108453

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105169

[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default (2)

This patch updates test files after D105169.
Autogenerated test codes are changed by `utils/update_cc_test_checks.py,` and non-autogenerated test codes are changed as follows:

(1) I wrote a python script that (partially) updates the tests using regex: {F18594904} The script is not perfect, but I believe it gives hints about which patterns are updated to have `noundef` attached.

(2) The remaining tests are updated manually.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D108453

Resolve lit failures in clang after 8ca4b3e's land

Fix lit test failures in clang-ppc* and clang-x64-windows-msvc

Fix missing failures in clang-ppc64be* and retry fixing clang-x64-windows-msvc

Fix internal_clone(aarch64) inline assembly
2021-11-06 19:19:22 +09:00
Vitaly Buka 39ead64e3f [sanitizer] Intercept lstat on Linux
It's availible from GLIBC 2.33
Fixes use-of-uninitialized-value llvm/lib/Support/Unix/Path.inc:467:29 in llvm::sys::fs::remove(llvm::Twine const&, bool)
2021-11-06 00:52:54 -07:00
Juneyoung Lee 89ad2822af Revert "[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default"
This reverts commit 7584ef766a.
2021-11-06 15:39:19 +09:00
Juneyoung Lee 7584ef766a [Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default
Turning on `enable_noundef_analysis` flag allows better codegen by removing freeze instructions.
I modified clang by renaming `enable_noundef_analysis` flag to `disable-noundef-analysis` and turning it off by default.

Test updates are made as a separate patch: D108453

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105169
2021-11-06 15:36:42 +09:00
Vitaly Buka 3c7960cba1 [libFuzzer] Disable Msan on InternalStrnlen
It's called from ATTRIBUTE_NO_SANITIZE_MEMORY code.
It worked as expected if inlined and complained otherwise.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D113323
2021-11-05 15:24:09 -07:00
Martin Liska 78d3e0a4f1 sanitizer: Fix -Wpedantic GCC warning
Fixes:
sanitizer_stacktrace.h:212:5: warning: ISO C++ forbids braced-groups within expressions [-Wpedantic]

Differential Revision: https://reviews.llvm.org/D113292
2021-11-05 18:05:23 +01:00
Martin Liska 13a442ca49 Enable -Wformat-pedantic and fix fallout.
Differential Revision: https://reviews.llvm.org/D113172
2021-11-05 13:12:35 +01:00
Vitaly Buka bbc213afd1 [asan] compiler-rt version of D113143 2021-11-04 23:51:16 -07:00
Martin Liska 629b40dafa Fix -Wformat warnings reported by GCC.
Differential Revision: https://reviews.llvm.org/D113099
2021-11-04 08:51:43 +01:00
Julian Lettner f643afa25f Revert "Mark tsan cxa_guard_acquire test as unsupported on Darwin"
This reverts commit 593275c93c.

This test now passes again.
2021-11-03 19:07:56 -07:00
Lang Hames 870fc844d1 [ORC-RT] Add SPS serialization for span<const char> / SPSSequence<char>. 2021-11-03 13:43:49 -07:00
Vitaly Buka 91f0a6ad4e [asan] Disable test on Android Arm 32bit
Caused by D111703.
2021-11-03 13:12:56 -07:00
Vitaly Buka e28c64ecb1 [sanitizer] Disable new test on Android
Test added with D113055
2021-11-03 11:57:04 -07:00
Tamir Duberstein f639882be8 [sanitizer] Allow getsockname with NULL addrlen
This is already permitted in getpeername, and returns EFAULT
on Linux (does not crash the program).

Fixes https://github.com/google/sanitizers/issues/1451.

Differential Revision: https://reviews.llvm.org/D113055
2021-11-03 10:23:01 -07:00
Tamir Duberstein 33d9b7b4b2 [sanitizer] Mark before deref in PosixSpawnImpl
Read each pointer in the argv and envp arrays before dereferencing
it; this correctly marks an error when these pointers point into
memory that has been freed.

Differential Revision: https://reviews.llvm.org/D113046
2021-11-03 10:18:06 -07:00
Rich Lowe de6f7252da [sanitizer_common] Fix readlink error handling in sanitizer_procmaps_solaris.cpp
As pointed out in Bug 52371, the Solaris version of
`MemoryMappingLayout::Next` completely failed to handle `readlink` errors
or properly NUL-terminate the result.

This patch fixes this.  Originally provided in the PR with slight
formatting changes.

Tested on `amd64-pc-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D112998
2021-11-02 22:06:17 +01:00
Matt Morehouse 498a4c2fd7 [HWASan] Print short tags in tag mismatch description.
I recently spent some extra time debugging a false positive because I
didn't realize the "real" tag was in the short granule.  Adding the
short tag here makes it more obvious that we could be dealing with a
short granule.

Reviewed By: hctim, eugenis

Differential Revision: https://reviews.llvm.org/D112949
2021-11-02 08:00:33 -07:00
Matt Morehouse 8c60e0b632 [HWASan] Print short tags in __hwasan_print_shadow.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D112959
2021-11-02 07:59:25 -07:00
David Carlier 56513e3e42 [sanitizers] remove extra comma for sha2 interceptors 2021-11-02 10:35:44 +00:00
Vitaly Buka 08e0b9235b Revert "[NFC][sanitizer] Disable a test with large stderr output"
This reverts commit 6eb38e5171.

The test was not the reason of sanitizer-x86_64-linux-android failures.
2021-11-02 01:25:46 -07:00
Vitaly Buka 869989dd71 [sanitizer] Restore trivial posix_spawn init in test
It's still needed on Darwin.
2021-11-01 23:58:18 -07:00
Vitaly Buka 57d34cb223 Revert "[sanitizer] Fix compilation of the test"
Revert "[sanitizer] Fix PosixSpawnImpl which fails exitcode test"

This reverts commit 8cabd8f0d2 and part of 9213202abd.

It is not enough for Android.
2021-11-01 22:25:26 -07:00
Vitaly Buka 8cabd8f0d2 [sanitizer] Fix compilation of the test 2021-11-01 21:50:50 -07:00
Vitaly Buka 15361a98aa [sanitizer] Add test for 9213202abd 2021-11-01 21:14:11 -07:00
Vitaly Buka 9213202abd [sanitizer] Fix PosixSpawnImpl which fails exitcode test 2021-11-01 21:03:02 -07:00
Vitaly Buka 0d59efbba4 [sanitizer] Fix test include on Darwin 2021-11-01 16:49:50 -07:00
Matt Morehouse 4d8b0aa5c0 [HWASan] Apply TagMaskByte to every global tag.
Previously we only applied it to the first one, which could allow
subsequent global tags to exceed the valid number of bits.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D112853
2021-11-01 06:31:44 -07:00
Matt Morehouse b298d6d162 [sanitizer] Disable posix_spawn.c test on Android. 2021-11-01 06:22:48 -07:00
David Carlier b4fb42300e [sanitizer] msan build fix on non linux platforms 2021-11-01 07:18:43 +00:00
Vitaly Buka 948b91a08e [NFC][sanitizer] Atomix relaxed in TwoLevelMap
This is NOOP in x86_64.
On arch64 it avoids Data Memory Barrier with visible improvements on micro benchmarks.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D112391
2021-10-31 12:18:03 -07:00
Vitaly Buka e1240745ef [sanitizer] Add posix_spawn interceptor
To make pid initialized for Msan.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D112784
2021-10-31 12:10:54 -07:00
Lang Hames 81a6eabf71 [ORC-RT] Add missing file that was left out of ff846fcb64. 2021-10-31 10:53:17 -07:00
Lang Hames ff846fcb64 [ORC][ORC-RT] Switch MachO EH/TLV registration from EPC-calls to alloc actions.
MachOPlatform used to make an EPC-call (registerObjectSections) to register the
eh-frame and thread-data sections for each linked object with the ORC runtime.

Now that JITLinkMemoryManager supports allocation actions we can use these
instead of an EPC call. This saves us one EPC-call per object linked, and
manages registration/deregistration in the executor, rather than the controller
process. In the future we may use this to allow JIT'd code in the executor to
outlive the controller object while still being able to be cleanly destroyed.

Since the code for allocation actions must be available when the actions are
run, and since the eh-frame registration code lives in the ORC runtime itself,
this change required that MachO eh-frame support be split out of
macho_platform.cpp and into its own macho_ehframe_registration.cpp file that has
no other dependencies. During bootstrap we start by forcing emission of
macho_ehframe_registration.cpp so that eh-frame registration is guaranteed to be
available for the rest of the bootstrap process. Then we load the rest of the
MachO-platform runtime support, erroring out if there is any attempt to use
TLVs. Once the bootstrap process is complete all subsequent code can use all
features.
2021-10-31 10:27:40 -07:00
Dmitry Vyukov d31b2dc235 tsan: add debugging code for ptrace test failures
Debugging of crashes on powerpc after commit:
c80604f7a3 ("tsan: remove real func check from interceptors")
Somehow replacing if with DCHECK leads to strange failures in:
SanitizerCommon-tsan-powerpc64le-Linux :: Linux/ptrace.cpp
https://lab.llvm.org/buildbot/#/builders/105
https://lab.llvm.org/buildbot/#/builders/121
https://lab.llvm.org/buildbot/#/builders/57

The hypothesis is that something writes out-of-bounds
into pt_regs on stack and that corrupts internal tsan state.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112703
2021-10-29 17:37:02 +02:00
Dmitry Vyukov 014c6b0736 sanitizer_common: fix ParseUnixMemoryProfile test linking on Mac
The ParseUnixMemoryProfile function is defined only for a subset
of platforms. Define the test for the same set of platforms.

Also disable the test for 32-bit platforms b/c the pointer
values used in the test are 64-bit and don't fit into 32-bit uptr.

Reported-by: Jan Svoboda (jansvoboda11)

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112815
2021-10-29 16:26:24 +02:00
Rainer Orth 2f353c040f [sanitizer_common] Fix Posix/mmap_write_exec.cpp on 32-bit Solaris
The new `Posix/mmap_write_exec.cpp` test FAILs on 32-bit Solaris/x86.  This
happens because only `mmap` is intercepted, but not `mmap64` which is used
for largefile support.

Fixed by also intercepting `mmap64`.

Tested on `amd64-pc-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D112810
2021-10-29 14:29:41 +02:00
Dmitry Vyukov af0a8b8317 sanitizer_common: bump default file max size to 256MB
I am hitting some cases where /proc/self/maps does not fit into 64MB.
256MB is lots of memory, but it's not radically more than the current 64MB.
Ideally we should read/parse these huge files incrementally,
but that's lots of work for a debugging/introspection interface.
So for now just bump the limit.

Depends on D112793.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112794
2021-10-29 14:03:26 +02:00
Dmitry Vyukov e8861fa6c3 sanitizer_common: fix crashes in parsing of memory profiles
ParseUnixMemoryProfile assumes well-formed input with \n at the end, etc.
It can over-read the input and crash on basically every line
in the case of malformed input.
ReadFileToBuffer has cap the max file size (64MB) and returns
truncated contents if the file is larger. Thus even if kernel behaves,
ParseUnixMemoryProfile crashes on too large /proc/self/smaps.
Fix input over-reading in ParseUnixMemoryProfile.

Depends on D112792.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112793
2021-10-29 14:03:12 +02:00
Dmitry Vyukov 4acad5df33 sanitizer_common: add memory profile parsing test
Depends on D112790.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112792
2021-10-29 14:02:50 +02:00
Dmitry Vyukov ae833dbb81 sanitizer_common: remove unnecessary GetMemoryProfile arg
stats_size argument is unnecessary in GetMemoryProfile and in the callback.
It just clutters code. The callback knowns how many stats to expect.

Depends on D112789.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112790
2021-10-29 13:44:34 +02:00
Dmitry Vyukov d4602f759a sanitizer_common: make parsing of smaps testable (NFC)
Move parsing of /proc/self/smaps into a separate function
so that it can be tested.

Depends on D112788.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112789
2021-10-29 13:44:28 +02:00
Dmitry Vyukov a871c34e8d sanitizer_common: fix up onprint.cpp test more
D112630 ("sanitizer_common: fix up onprint.cpp test")
added O_CREAT, but we also need O_TRUNC b/c the file
may not exist, or may exist as well.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112788
2021-10-29 13:44:04 +02:00
Lang Hames dc8e5e1dc0 [ORC-RT] Add a WrapperFunctionCall utility.
WrapperFunctionCall represents a call to a wrapper function as a pair of a
target function (as an ExecutorAddr), and an argument buffer range (as an
ExecutorAddrRange). WrapperFunctionCall instances can be serialized via
SPS to send to remote machines (only the argument buffer address range is
copied, not any buffer content).

This utility will simplify the implementation of JITLinkMemoryManager
allocation actions in the ORC runtime.
2021-10-28 17:07:28 -07:00
Vitaly Buka 6eb38e5171 [NFC][sanitizer] Disable a test with large stderr output 2021-10-28 15:15:20 -07:00
David CARLIER c9174f63b6 [compiler-rt] fix asan buildbot failure on unit test for darwin 2021-10-28 18:48:54 +01:00
David CARLIER bb168f3207 [compiler-rt] update detect_write_exec option for apple devices.
Reviewed By: yln, vitalybuka

Differential Revision: https://reviews.llvm.org/D111390
2021-10-28 17:08:23 +01:00