Commit Graph

4899 Commits

Author SHA1 Message Date
Lang Hames 5255ca9726 [llvm-jitlink] Update handling of library options.
Adds -L<search-path> and -l<library> options that are analogous to ld's
versions.

Each instance of -L<search-path> or -l<library> will apply to the most recent
-jd option on the command line (-jd <name> creates a JITDylib with the given
name). Library names will match against JITDylibs first, then llvm-jitlink will
look through the search paths for files named <search-path>/lib<library>.dylib
or <search-path>/lib<library>.a.

The default "main" JITDylib will link against all JITDylibs created by -jd
options, and all JITDylibs will link against the process symbols (unless
-no-process-symbols is specified).

The -dlopen option is renamed -preload, and will load dylibs into the JITDylib
for the ORC runtime only.

The effect of these changes is to make it easier to describe a non-trivial
program layout to llvm-jitlink for testing purposes. E.g. the following
invocation describes a program consisting of three JITDylibs: "main" (created
implicitly) containing main.o, "Foo" containing foo1.o and foo2.o, and linking
against library "bar" (not a JITDylib, so it must be a .dylib or .a on disk)
and "Baz" (which is a JITDylib), and "Baz" containing baz.o.

llvm-jitlink \
  main.o \
  -jd Foo foo1.o foo2.o -L${HOME}/lib -lbar -lBaz
  -jd Baz baz.o
2021-12-15 13:49:30 +11:00
Kirill Stoimenov c13524856b [ASan] Shared optimized callbacks implementation.
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114558
2021-12-14 15:55:14 +00:00
Arthur Eubanks fff876c20e Revert "[ASan] Shared optimized callbacks implementation."
This reverts commit ebc31d2e72.

Causes test failures on Windows, see comments on D114558.
2021-12-13 15:26:55 -08:00
Vitaly Buka 75504789bd [NFC][asan] XFAIL a test on THUMB
Followup to D114934
2021-12-13 12:07:13 -08:00
Dmitry Vyukov b332134921 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Depends on D112602.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112603
2021-12-13 12:48:34 +01:00
Dmitry Vyukov b088833375 tsan: deflake dlopen_static_tls.cpp
Currently the test calls dlclose in the thread
concurrently with the main thread calling a function
from the dynamic library. This is not good.
Wait for the main thread to call the function
before calling dlclose.

Depends on D115612.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D115613
2021-12-13 12:01:40 +01:00
Dmitry Vyukov 7de546e9e8 tsan: deflake flush_memory.cpp
The test contains a race and checks that it's detected.
But the race may not be detected since we are doing aggressive flushes
and if the state flush happens between racing accesses, tsan won't
detect the race). So return 1 to make the test deterministic
regardless of the race.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D115612
2021-12-13 12:01:30 +01:00
Florian Mayer edd2b99a57 [sanitizers] include build ids in stacks on linux.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D114294
2021-12-10 14:24:03 -08:00
Florian Mayer 06430acc00 Revert "[sanitizers] include build ids in stacks on linux."
This reverts commit 2a31b240df.
2021-12-10 11:52:26 -08:00
Florian Mayer 2a31b240df [sanitizers] include build ids in stacks on linux.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D114294
2021-12-10 11:30:34 -08:00
Kirill Stoimenov ebc31d2e72 [ASan] Shared optimized callbacks implementation.
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114558
2021-12-10 15:34:05 +00:00
Kirill Stoimenov ab396165fe Revert "[ASan] Shared optimized callbacks implementation."
This reverts commit 428ed61a92.

Build bot failure:
https://lab.llvm.org/buildbot/#/builders/37
https://lab.llvm.org/buildbot/#/builders/37/builds/9041

Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D115489
2021-12-10 02:13:34 +00:00
Kirill Stoimenov 76d3d07845 [ASan] Fixed Windows test by excluding macro instantiated INTERFACE_FUNCTION.
Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D115478
2021-12-10 00:04:37 +00:00
Kirill Stoimenov 428ed61a92 [ASan] Shared optimized callbacks implementation.
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114558
2021-12-09 23:04:56 +00:00
Matt Morehouse 2d6dfce4aa [libFuzzer] Remove entropic-scale-per-exec-time.test.
The test has been flaky for years, and I think we should remove it to
eliminate noise on the buildbot.

Neither me nor dokyungs have been able to fully deflake the test, and it
tests a non-default Entropic flag.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115453
2021-12-09 12:20:06 -08:00
Vitaly Buka d71775cd2a [asan] Run background thread for asan only on THUMB
As in D114934, or lsan crashes on the same bot.
2021-12-09 10:43:06 -08:00
Vitaly Buka fa24c4a1c0 [sanitizer] Run Stack compression in background thread
Depends on D114495.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D114498
2021-12-09 10:29:50 -08:00
Petr Hosek a1e7f62801 Revert "[sanitizer] Run Stack compression in background thread"
This reverts commit e5c2a46c5e as this
change introduced a linker error when building sanitizer runtimes:

  ld.lld: error: undefined symbol: __sanitizer::internal_start_thread(void* (*)(void*), void*)
  >>> referenced by sanitizer_stackdepot.cpp:133 (compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp:133)
  >>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stackdepot.cpp.obj:(__sanitizer::(anonymous namespace)::CompressThread::NewWorkNotify())
2021-12-09 09:56:48 -08:00
Jonas Devlieghere 396113c19f Revert "tsan: new runtime (v3)"
This reverts commit 5a33e41281 becuase it
breaks LLDB.

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/39208/
2021-12-09 09:18:10 -08:00
Dmitry Vyukov 5a33e41281 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Depends on D112602.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112603
2021-12-09 09:09:52 +01:00
Kirill Stoimenov f75cce0be8 Revert "[ASan] Shared optimized callbacks implementation."
This reverts commit f71c553a30.

Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D115407
2021-12-09 00:18:29 +00:00
Kirill Stoimenov f71c553a30 [ASan] Shared optimized callbacks implementation.
This change moves optimized callbacks from each .o file to compiler-rt. Instead of using code generation it uses direct assembly implementation. Please note that the 'or' version is not implemented and it will produce unresolved external if somehow 'or' version is requested.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114558
2021-12-09 00:00:08 +00:00
Vitaly Buka e5c2a46c5e [sanitizer] Run Stack compression in background thread
Depends on D114495.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D114498
2021-12-08 13:00:09 -08:00
Vitaly Buka 9601085bc5 [NFC][asan] Remove -std=c++20 from test
Fails after D110215
with errors like
/usr/include/x86_64-linux-gnu/sys/types.h:33:9: error: unknown type name '__u_char'
typedef __u_char u_char;
2021-12-07 22:36:54 -08:00
David Spickett 6bfbb89e96 [compiler-rt][libFuzzer] Disable counters test on arm
This test is either very slow or loops forever on 32 bit Arm.

One of a few tests causing timeouts on our buildbots:
https://lab.llvm.org/buildbot/#/builders/190/builds/513
2021-12-07 11:55:11 +00:00
Dmitry Vyukov 954582cdfc tsan: disable dlopen_static_tls.cpp test on powerpc64
Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D115142
2021-12-06 13:13:43 +01:00
Dmitry Vyukov fd26417a74 tsan: disable dlopen_static_tls.cpp test on aarch64
Fails on bots: https://lab.llvm.org/buildbot#builders/184/builds/1580

Differential Revision: https://reviews.llvm.org/D115095
2021-12-04 13:01:47 +01:00
Vitaly Buka 9c491c873c [sanitizer] Hook up LZW into stack store
Depends on D114503.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114924
2021-12-03 23:38:41 -08:00
Kevin Athey 6c2be3015e [NFC][sanitizer] Add test for command line flag for enable-noundef-analysis.
A simple unit test to demonstrate the flags working correctly.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D114485
2021-12-03 15:44:36 -08:00
Vitaly Buka 6318001209 [sanitizer] Support IsRssLimitExceeded in all sanitizers
Reviewed By: kstoimenov

Differential Revision: https://reviews.llvm.org/D115000
2021-12-03 12:45:44 -08:00
Dmitry Vyukov 4a5086dce3 tsan: disable munmap_invalid.cpp test on darwin
It failed on bots:
https://green.lab.llvm.org/green//job/clang-stage1-RA/25954/consoleFull#-1417328700a1ca8a51-895e-46c6-af87-ce24fa4cd561
and it  doesn't provide the test output.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114972
2021-12-03 09:03:45 +01:00
Vitaly Buka 550fd071ed [lsan] Deflake fork_and_leak test 2021-12-02 18:06:04 -08:00
Vitaly Buka 7185b1f406 [sanitizer] Add compress_stack_depot flag
Depends on D114494.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114495
2021-12-02 10:42:53 -08:00
Dmitry Vyukov 1b576585eb tsan: tolerate munmap with invalid arguments
We call UnmapShadow before the actual munmap, at that point we don't yet
know if the provided address/size are sane. We can't call UnmapShadow
after the actual munmap becuase at that point the memory range can
already be reused for something else, so we can't rely on the munmap
return value to understand is the values are sane.
While calling munmap with insane values (non-canonical address, negative
size, etc) is an error, the kernel won't crash. We must also try to not
crash as the failure mode is very confusing (paging fault inside of the
runtime on some derived shadow address).

Such invalid arguments are observed on Chromium tests:
https://bugs.chromium.org/p/chromium/issues/detail?id=1275581

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114944
2021-12-02 17:50:51 +01:00
Dmitry Vyukov 97b4e63117 tsan: fix false positives in dynamic libs with static tls
The added test demonstrates  loading a dynamic library with static TLS.
Such static TLS is a hack that allows a dynamic library to have faster TLS,
but it can be loaded only iff all threads happened to allocate some excess
of static TLS space for whatever reason. If it's not the case loading fails with:

dlopen: cannot load any more object with static TLS

We used to produce a false positive because dlopen will write into TLS
of all existing threads to initialize/zero TLS region for the loaded library.
And this appears to be racing with initialization of TLS in the thread
since we model a write into the whole static TLS region (we don't what part
of it is currently unused):

WARNING: ThreadSanitizer: data race (pid=2317365)
  Write of size 1 at 0x7f1fa9bfcdd7 by main thread:
    0 memset
    1 init_one_static_tls
    2 __pthread_init_static_tls
    [[ this is where main calls dlopen ]]
    3 main
  Previous write of size 8 at 0x7f1fa9bfcdd0 by thread T1:
    0 __tsan_tls_initialization

Fix this by ignoring accesses during dlopen.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114953
2021-12-02 17:47:05 +01:00
Steven Wan f9d585d0dd Revert "[sanitizer] Add compress_stack_depot flag"
This is failing on clang-s390x-linux,
https://lab.llvm.org/buildbot/#/builders/94/builds/6748.
This reverts commit bf18253b0e.
2021-12-01 20:21:52 -05:00
Julian Lettner 863b117411 [TSan][Darwin] Prevent inlining of functions in tests
Prevent inlining of functions so we can FileCheck the generated stack
traces.
2021-12-01 17:00:52 -08:00
Julian Lettner 6703fe25b7 [TSan][Darwin] Mark test unsupported 2021-12-01 15:50:10 -08:00
Dmitry Vyukov 09859113ed Revert "tsan: new runtime (v3)"
This reverts commit 66d4ce7e26.

Chromium tests started failing:
https://bugs.chromium.org/p/chromium/issues/detail?id=1275581
2021-12-01 18:00:46 +01:00
Vitaly Buka bf18253b0e [sanitizer] Add compress_stack_depot flag
Depends on D114494.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114495
2021-11-30 19:08:58 -08:00
Ryan Mansfield 455a2b824d Fix file extension of alignment-assumption-ignorelist.cppp test
During the renaming of blacklist to ignorelist this test got renamed
incorrectly.

Differential revision: https://reviews.llvm.org/D114710
2021-11-30 17:21:46 +01:00
Matt Morehouse 2022e2fcd0 [HWASan] Disable LTO test on aarch64.
It fails for non-Android aarch64 bots as well.
2021-11-29 08:08:17 -08:00
Vitaly Buka 8ae815cb19 [sanitizer] Switch StackStore to 8 MiB blocks
Larger blocks are more convenient for compressions.
Blocks are allocated with MmapNoReserveOrDie to save some memory.

Also it's 15% faster on StackDepotBenchmarkSuite

Depends on D114464.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D114488
2021-11-26 14:00:10 -08:00
Benjamin Kramer 0e099a64be [tsan] Relax atexit5.cpp a bit more so it's not as dependent on the standard library implementation 2021-11-26 14:02:34 +01:00
Dmitry Vyukov a1dc97e472 tsan: remember and print function that installed at_exit callbacks
Sometimes stacks for at_exit callbacks don't include any of the user functions/files.
For example, a race with a global std container destructor will only contain
the container type name and our at_exit_wrapper function. No signs what global variable
this is.
Remember and include in reports the function that installed the at_exit callback.
This should give glues as to what variable is being destroyed.

Depends on D114606.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114607
2021-11-26 08:00:55 +01:00
Dmitry Vyukov 3f87788de1 tsan: add a test for on_exit
Depends on D114605.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114606
2021-11-26 08:00:43 +01:00
Dmitry Vyukov 9ea3bd5a1c tsan: add test for __cxa_atexit
Add a test for a common C++ bug when a global object is destroyed
while background threads still use it.

Depends on D114604.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114605
2021-11-26 08:00:29 +01:00
Dmitry Vyukov c2f0de06c9 tsan: check stack in atexit4.cpp test
Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D114604
2021-11-26 08:00:19 +01:00
Dmitry Vyukov 66d4ce7e26 tsan: new runtime (v3)
This change switches tsan to the new runtime which features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutimes

Depends on D112602.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D112603
2021-11-25 18:32:04 +01:00
Dmitry Vyukov b584741d06 tsan: fix Java heap block begin in reports
We currently use a wrong value for heap block
(only works for C++, but not for Java).
Use the correct value (we already computed it before, just forgot to use).

Depends on D114593.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D114595
2021-11-25 17:07:53 +01:00