Commit Graph

387 Commits

Author SHA1 Message Date
Petr Hosek 22acda48ff [CMake] Cache the compiler-rt library search results
There's a lot of duplicated calls to find various compiler-rt libraries
from build of runtime libraries like libunwind, libc++, libc++abi and
compiler-rt. The compiler-rt helper module already implemented caching
for results avoid repeated Clang invocations.

This change moves the compiler-rt implementation into a shared location
and reuses it from other runtimes to reduce duplication and speed up
the build.

Differential Revision: https://reviews.llvm.org/D88458
2021-10-27 17:53:03 -07:00
Petr Hosek ba4920e98e Revert "[CMake] Cache the compiler-rt library search results"
This reverts commit 0eed292fba, there
are compiler-rt build failures that appear to have been introduced
by this change.
2021-10-21 10:32:01 -07:00
Petr Hosek 0eed292fba [CMake] Cache the compiler-rt library search results
There's a lot of duplicated calls to find various compiler-rt libraries
from build of runtime libraries like libunwind, libc++, libc++abi and
compiler-rt. The compiler-rt helper module already implemented caching
for results avoid repeated Clang invocations.

This change moves the compiler-rt implementation into a shared location
and reuses it from other runtimes to reduce duplication and speed up
the build.

Differential Revision: https://reviews.llvm.org/D88458
2021-10-18 14:44:07 -07:00
Leonard Chan 77d5ccdc6f [compiler-rt][test] Add shared_unwind requirement
When using a static libunwind, the check_memcpy.c can fail because it checks
that tsan intercepted all memcpy/memmoves in the final binary. Though if the
static libunwind is not instrumented, then this will fail because it may contain
regular memcpy/memmoves.

This adds a new REQUIRES check for ensuring that this test won't run unless a
dynamic libunwind.so is provided.

Differential Revision: https://reviews.llvm.org/D111194
2021-10-06 11:10:36 -07:00
Leonard Chan ac191bcc99 [compiler-rt][test] Add REQUIRES for checking static libc++abi
intercept-rethrow-exception.cc fails when running runtimes tests if linking in
a hermetic libc++abi. This is because if libc++abi is used, then asan expects
to intercept __cxa_rethrow_primary_exception on linux, which should unpoison the
stack. If we statically link in libc++abi though, it will contain a strong
definition for __cxa_rethrow_primary_exception which wins over the weakly
defined interceptor provided by asan, causing the test to fail by not unpoisoning
the stack on the exception being thrown.

It's likely no one has encountered this before and possible that upstream tests
opt for dynamically linking where the interceptor can work properly. An ideal
long term solution would be to update the interceptor and libc++[abi] APIs to
work for this case, but that will likely take a long time to work out. In the
meantime, since the test isn't necessarily broken, we can just add another
REQUIRES check to make sure that it's only run if we aren't statically linking
in libc++abi.

Differential Revision: https://reviews.llvm.org/D109938
2021-09-22 15:25:05 -07:00
Dmitry Vyukov 7142eb17fb sanitizers: compile with -O1 under debug
Tsan's check_memcpy.c test was disabled under debug because it failed.
But it points to real issues and does not help to just disable it.
I tried to enable it and see what fail and the first hit was default ctor for:

  struct ChainedOriginDepotDesc {
    u32 here_id;
    u32 prev_id;
  };

initializing these fields to 0's help partially,
but compiler still emits memset before calling ctor.
I did not try to see what's the next failure, because if it fails
on such small structs, it won't be realistic to fix everything
and keep working.

Compile runtimes with -O1 under debug instead.
It seems to fix all current failures. At least I run check-tsan
under clang/gcc x debug/non-debug and all combinations passed.
-O1 does not usually use too aggressive optimizations
and sometimes even makes debugging easier because machine code
is not exceedingly verbose.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D107962
2021-08-16 10:01:50 +02:00
Dmitry Vyukov 123c58ea26 sanitizer_common: enable format string checking
Enable -Wformat in sanitizer_common now that it's
cleaned up from existing warnings.
But disable it in all sanitizers for now since
they are not cleaned up yet, but inherit sanitizer_common CFLAGS.

Depends on D107980.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D107981
2021-08-13 13:44:52 +02:00
Dmitry Vyukov 9c8f888f5f sanitizer_common: prepare for enabling format string checking
The __attribute__((format)) was added somewhere in 2012,
the lost during refactoring, then re-added in 2014 but
to te source files, which is a no-op.
Move it back to header files so that it actually takes effect.
But over the past 7 years we've accumulated whole lot of
format string bugs of different types, so disable the warning
with -Wno-format for now for incremental clean up.

Among the bugs that it warns about are all kinds of bad things:
 - wrong sizes of arguments
 - missing/excessive arguments
 - printing wrong things (e.g. *ptr instead of ptr)
 - completely messed up format strings
 - security issues where external string is used as format

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D107977
2021-08-13 13:43:57 +02:00
Dmitry Vyukov 2f6ac22b08 sanitizers: turn thread-safety errors into warnings
There were 2 issues reported on https://reviews.llvm.org/D105716:
1. FreeBSD phtread.h is annotated with thread-safety attributes
and this causes errors in gtest headers.
2. If sanitizers are compiled with an older versions of clang
(which supports the annotations, but has some false positives
in analysis not present in later versions of clang), compilation
fails with errors.

Switch the errors to warnings by default.
Some CI bots enable COMPILER_RT_ENABLE_WERROR, which should
turn these warnings back into errors.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D107826
2021-08-10 20:03:30 +02:00
Dimitry Andric b260f3fdda sanitizer_common: disable thread safety annotations for googletest
Recently in 0da172b176 thread safety warnings-as-errors were enabled.
However, googletest is currently not compatible with thread safety
annotations. On FreeBSD, which has the pthread functions marked with
such annotations, this results in errors when building the compiler-rt
tests:

    In file included from compiler-rt/lib/interception/tests/interception_test_main.cpp:15:
    In file included from llvm/utils/unittest/googletest/include/gtest/gtest.h:62:
    In file included from llvm/utils/unittest/googletest/include/gtest/internal/gtest-internal.h:40:
    llvm/utils/unittest/googletest/include/gtest/internal/gtest-port.h:1636:3: error: mutex 'mutex_' is still held at the end of function [-Werror,-Wthread-safety-analysis]
      }
      ^
    llvm/utils/unittest/googletest/include/gtest/internal/gtest-port.h:1633:32: note: mutex acquired here
        GTEST_CHECK_POSIX_SUCCESS_(pthread_mutex_lock(&mutex_));
                                   ^
    llvm/utils/unittest/googletest/include/gtest/internal/gtest-port.h:1645:32: error: releasing mutex 'mutex_' that was not held [-Werror,-Wthread-safety-analysis]
        GTEST_CHECK_POSIX_SUCCESS_(pthread_mutex_unlock(&mutex_));
                                   ^
    2 errors generated.

At some point googletest will hopefully be made compatible with thread
safety annotations, but for now add corresponding `-Wno-thread-*` flags
to `COMPILER_RT_GTEST_CFLAGS` to silence these warnings-as-errors.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D107491
2021-08-05 20:07:24 +02:00
Dan Liew fb0a929512 [Compiler-RT] On Apple Platforms switch to always emitting full debug info
Previously the build used `-gline-tables-only` when `COMPILER_RT_DEBUG`
was off (default) and `-g` when `COMPILER_RT_DEBUG` was on. The end
result of this meant that the release build of the Sanitizer runtimes
were difficult to debug (e.g. information about variables and function
arguments were missing).

Presumably the reason for preferring `-gline-tables-only` for release
builds was to save space. However, for Apple platforms this doesn't
matter because debug info lives in separate `.dSYM` files (which aren't
shipped) rather than in the shipped `.dylib` files.

Now on Apple platforms we always emit full debug info if the compiler
supports it and we emit a fatal error if `-g` isn't supported.

rdar://79223184

Differential Revision: https://reviews.llvm.org/D107501
2021-08-04 19:45:33 -07:00
Michael Jones 6ed60fb8a2 [libc] add integration tests for scudo in libc
This change adds tests to make sure that SCUDO is being properly
included with llvm libc. This change also adds the toggles to properly
use SCUDO, as GWP-ASan is enabled by default and must be included for
SCUDO to function.

Reviewed By: sivachandra, hctim

Differential Revision: https://reviews.llvm.org/D106919
2021-08-04 20:06:09 +00:00
Dmitry Vyukov 3ea3b6b2d4 sanitizers: build tests with -g
We currently build tests without -g, which is quite inconvenient.
Crash stacks don't have line numbers, gdb don't how line numbers either.
Always build tests with -g.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D107168
2021-07-31 08:55:12 +02:00
Marco Elver 98033fdc50 sanitizer_common: Fix build for tests
It turns out that COMPILER_RT_TEST_COMPILER_CFLAGS is actually a string
that is being appended to and not a list.

Therefore, append the thread-safety flags to the string. Because CMake
separates list elements by ';' when turning into a string, also
substitute ';' with ' '.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D105829
2021-07-12 15:49:40 -07:00
Dmitry Vyukov 10158b52dc sanitizer_common: fix 32-bit build
https://reviews.llvm.org/D105716 enabled thread safety annotations,
and that broke 32-bit build:
https://green.lab.llvm.org/green/job/lldb-cmake/33604/consoleFull#-77815080549ba4694-19c4-4d7e-bec5-911270d8a58c

1. Enable thread-safety analysis in unit tests
(this catches the breakage even in 64-bit mode).
2. Add NO_THREAD_SAFETY_ANALYSIS to sanitizer_allocator_primary32.h
to unbreak the build.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105808
2021-07-12 14:03:13 +02:00
Dmitry Vyukov 0da172b176 sanitizer_common: add thread safety annotations
Enable clang Thread Safety Analysis for sanitizers:
https://clang.llvm.org/docs/ThreadSafetyAnalysis.html

Thread Safety Analysis can detect inconsistent locking,
deadlocks and data races. Without GUARDED_BY annotations
it has limited value. But this does all the heavy lifting
to enable analysis and allows to add GUARDED_BY incrementally.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105716
2021-07-12 11:46:49 +02:00
Leonard Chan 398bfa2ead [compiler-rt][Fuchsia] Disable interceptors while enabling new/delete replacements
This disables use of hwasan interceptors which we do not use on Fuchsia. This
explicitly sets the macro for defining the hwasan versions of new/delete.

Differential Revision: https://reviews.llvm.org/D103544
2021-07-07 15:14:34 -07:00
Petr Hosek cadfaf2df4 [compiler-rt] Make use of undefined symbols configurable
We want to disable the use of undefined symbols on Fuchsia, but there
are cases where it might be desirable so may it configurable.

Differential Revision: https://reviews.llvm.org/D104728
2021-06-22 14:25:36 -07:00
Nikita Popov ae1093921f Revert "[compiler-rt] Make use of undefined symbols configurable"
This reverts commit ed7086ad46.
This reverts commit b9792638b0.

This breaks cmake with message:

    CMake Error at llvm-project/compiler-rt/CMakeLists.txt:449:
      Parse error.  Expected "(", got newline with text "
2021-06-22 21:20:20 +02:00
Petr Hosek ed7086ad46 [CMake] Fix the option declaration
This addresses build issue introduced in
b9792638b0.
2021-06-22 11:58:26 -07:00
Petr Hosek b9792638b0 [compiler-rt] Make use of undefined symbols configurable
We want to disable the use of undefined symbols on Fuchsia, but there
are cases where it might be desirable so may it configurable.

Differential Revision: https://reviews.llvm.org/D104728
2021-06-22 11:49:31 -07:00
Petr Hosek fa5f425209 [compiler-rt][CMake] Drop flags that are set by default for Fuchsia
-Wl,-z,now is set by the Fuchsia driver, -Wl,-z,relro is the default
in LLD.
2021-06-22 11:49:30 -07:00
Ryan Prichard b834d63094 [sanitizer] Android ELF TLS is supported from Q (API 29)
Reviewed By: oontvoo, MaskRay

Differential Revision: https://reviews.llvm.org/D103214
2021-05-27 14:53:49 -07:00
Lang Hames 5e537ea1d7 [ORC-RT] Re-apply "Initial ORC Runtime directories and build..." with fixes.
This reapplies 1e1d75b190, which was reverted in ce1a4d5323 due to build
failures.

The unconditional dependencies on clang and llvm-jitlink in
compiler-rt/test/orc/CMakeLists.txt have been removed -- they don't appear to
be necessary, and I suspect they're the cause of the build failures seen
earlier.
2021-04-24 16:00:20 -07:00
Lang Hames ce1a4d5323 Revert "[ORC-RT] Initial ORC Runtime directories and build system files."
Some builders failed with a missing clang dependency. E.g.

CMake Error at /Users/buildslave/jenkins/workspace/clang-stage1-RA/clang-build \
  /lib/cmake/llvm/AddLLVM.cmake:1786 (add_dependencies):
The dependency target "clang" of target "check-compiler-rt" does not exist.

Reverting while I investigate.

This reverts commit 1e1d75b190.
2021-04-23 20:36:59 -07:00
Lang Hames 1e1d75b190 [ORC-RT] Initial ORC Runtime directories and build system files.
This patch contains initial directories and build files for the ORC runtime.

Differential Revision: https://reviews.llvm.org/D100711
2021-04-23 20:21:22 -07:00
Jim Lin dd4c999c23 fix typo in a CMake SANITIZER_CAN_USE_CXXABI variable initial definition
The current variable name isn't used anywhere else, which indicates it's
a typo.  Let's fix it before someone copy+pastes it somewhere else.

Reviewed By: Jim

Differential Revision: https://reviews.llvm.org/D39157
2021-04-12 14:05:37 +08:00
Petr Hosek 61a792b39b [CMake] Rename RUNTIMES_BUILD to LLVM_RUNTIMES_BUILD
This avoid potential conflict with other internal variables.

Differential Revision: https://reviews.llvm.org/D97838
2021-03-03 10:58:51 -08:00
Raul Tambre 480643a95c [CMake] Remove dead code setting policies to NEW
cmake_minimum_required(VERSION) calls cmake_policy(VERSION),
which sets all policies up to VERSION to NEW.
LLVM started requiring CMake 3.13 last year, so we can remove
a bunch of code setting policies prior to 3.13 to NEW as it
no longer has any effect.

Reviewed By: phosek, #libunwind, #libc, #libc_abi, ldionne

Differential Revision: https://reviews.llvm.org/D94374
2021-01-19 17:19:36 +02:00
Stella Stamenova 1410b72be3 [compiler-rt] Fix a bug in the cmakelists file when CMAKE_CXX_FLAGS are empty
Right now, the regex expression will fail if the flags were not set. Instead, we should follow the pattern of other llvm projects and quote the expression, so that it can work even when the flags are not set.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D92586
2020-12-03 10:25:52 -08:00
Vitaly Buka e51631ca4c [sanitizer] Fix Android API level parsing on arm 2020-11-14 01:54:45 -08:00
Vitaly Buka 234857f730 [sanitizer] Fix -fno-emulated-tls setup
COMPILER_RT_TEST_COMPILER_CFLAGS is a string
2020-11-04 19:23:28 -08:00
Vitaly Buka 09ec07827b [sanitizer] Get Android API from --target
Depends on D90792.

Differential Revision: https://reviews.llvm.org/D90793
2020-11-04 14:05:44 -08:00
Vy Nguyen 6b67e22ea3 Fix breakage in D89615 (due to cmake version 3.16.5)
Differential Revision: https://reviews.llvm.org/D90764
2020-11-04 11:19:01 -05:00
Vy Nguyen aa662f61de Disable emulated-tls for compiler-rt+tests on Android if ELF_TLS is presence.
This is necessary for enabling LSAN on Android (D89251) because:
 - LSAN will have false negatives if run with emulated-tls.
 - Bionic ELF-TLS is not compatible with Gold (hence the need for LLD)

Differential Revision: https://reviews.llvm.org/D89615
2020-11-04 09:49:45 -05:00
Vitaly Buka e86205680e [sanitizer] Remove ANDROID_NDK_VERSION 2020-11-04 01:15:25 -08:00
Vitaly Buka 9c31e12609 [sanitizer] Remove -Wno-non-virtual-dtor
Warning should be fixed with d48f2d7c02
2020-11-04 00:51:33 -08:00
Vy Nguyen 707d69ff32 Use LLD for Android compiler-rt
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D90720
2020-11-04 00:51:18 -08:00
Petr Hosek 11efd002b1 [CMake] Avoid accidental C++ standard library dependency in sanitizers
While sanitizers don't use C++ standard library, we could still end
up accidentally including or linking it just by the virtue of using
the C++ compiler. Pass -nostdinc++ and -nostdlib++ to avoid these
accidental dependencies.

Reviewed By: smeenai, vitalybuka

Differential Revision: https://reviews.llvm.org/D88922
2020-10-31 02:37:38 -07:00
Petr Hosek d11710dae6 [NFC][CMake] Move some COMPILER_RT variables setup
Part of D88922
2020-10-30 20:09:50 -07:00
Teresa Johnson 3d4bba302d [MemProf] Memory profiling runtime support
See RFC for background:
http://lists.llvm.org/pipermail/llvm-dev/2020-June/142744.html

Follow on companion to the clang/llvm instrumentation support in D85948
and committed earlier.

This patch adds the compiler-rt runtime support for the memory
profiling.

Note that much of this support was cloned from asan (and then greatly
simplified and renamed). For example the interactions with the
sanitizer_common allocators, error handling, interception, etc.

The bulk of the memory profiling specific code can be found in the
MemInfoBlock, MemInfoBlockCache, and related classes defined and used
in memprof_allocator.cpp.

For now, the memory profile is dumped to text (stderr by default, but
honors the sanitizer_common log_path flag). It is dumped in either a
default verbose format, or an optional terse format.

This patch also adds a set of tests for the core functionality.

Differential Revision: https://reviews.llvm.org/D87120
2020-10-16 09:47:02 -07:00
Petr Hosek 220de1f32a Revert "[CMake] Avoid accidental C++ standard library dependency in sanitizers"
This reverts commit 287c318690 which broke
sanitizer tests that use C++ standard library.
2020-10-14 18:44:09 -07:00
Petr Hosek 287c318690 [CMake] Avoid accidental C++ standard library dependency in sanitizers
While sanitizers don't use C++ standard library, we could still end
up accidentally including or linking it just by the virtue of using
the C++ compiler. Pass -nostdinc++ and -nostdlib++ to avoid these
accidental dependencies.

Differential Revision: https://reviews.llvm.org/D88922
2020-10-14 18:26:56 -07:00
Adhemerval Zanella 039126c97d [sanitizer] Disable fast_unwind_on_malloc as default for arm-linux-gnu
ARM thumb/thumb2 frame pointer is inconsistent on GCC and Clang [1]
and fast-unwider is also unreliable when mixing arm and thumb code [2].

The fast unwinder on ARM tries to probe and compare the frame-pointer
at different stack layout positions and it works reliable only on
systems where all the libraries were built in arm mode (either with
gcc or clang) or with clang in thmb mode (which uses the same stack
frame pointer layout in arm and thumb).

However when mixing objects built with different abi modes the
fast unwinder is still problematic as shown by the failures on the
AddressSanitizer.ThreadStackReuseTest. For these failures, the
malloc is called by the loader itself and since it has been built
with a thum enabled gcc, the stack frame is not correctly obtained
and the suppression rule is not applied (resulting in a leak warning).

The check for fast-unwinder-works is also changed: instead of checking
f it is explicit enabled in the compiler flags, it now checks if
compiler defined thumb pre-processor.

This should fix BZ#44158.

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92172
[2] https://bugs.llvm.org/show_bug.cgi?id=44158

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D88958
2020-10-12 14:36:08 -03:00
David Tenty 89074bdc81 [AIX][compiler-rt] Use the AR/ranlib mode flag for 32-bit and 64-bit mode
since we will be building both 32-bit and 64-bit compiler-rt builtins
from a single configuration.

Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D87113
2020-09-22 11:10:47 -04:00
Raul Tambre 86bd8f82cc [CMake] Remove dead FindPythonInterp code
LLVM has bumped the minimum required CMake version to 3.13.4, so this has become dead code.

Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D87189
2020-09-08 15:23:23 +03:00
Jinsong Ji 0cefaec2e8 [compiler-rt] Don't build llvm-lit in RUNTIMES-BUILD
With runtimes-build, we lost map config for llvm-lit, so we can NOT run
lit from source dir anymore.
All the subdir target like: ninja check-llvm-codegen-powerpc will fail
too.

We will get some cfg error like:
  File "../lvm-project/llvm/test/lit.cfg.py", line 21, in <module>
    config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
AttributeError: 'NoneType' object has no attribute 'use_lit_shell'

This is reset of map config in llvm-lit rebuild.
We already have llvm-lit in runtimes-build, so don't build it.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D87002
2020-09-03 01:08:16 +00:00
Shoaib Meenai 2c80e2fe51 [runtimes] Use llvm-libtool-darwin for runtimes build
It's full featured now and we can use it for the runtimes build instead
of relying on an external libtool, which means the CMAKE_HOST_APPLE
restriction serves no purpose either now. Restrict llvm-lipo to Darwin
targets while I'm here, since it's only needed there.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D86367
2020-08-24 13:48:30 -07:00
Louis Dionne afa1afd410 [CMake] Bump CMake minimum version to 3.13.4
This upgrade should be friction-less because we've already been ensuring
that CMake >= 3.13.4 is used.

This is part of the effort discussed on llvm-dev here:

  http://lists.llvm.org/pipermail/llvm-dev/2020-April/140578.html

Differential Revision: https://reviews.llvm.org/D78648
2020-07-22 14:25:07 -04:00
Petr Hosek cf1d044843 [CMake][compiler-rt] Only set lld as available when it was built
The `-fuse-ld=lld` check might succeed because there's a system lld,
but that lld may be out-of-date which would cause any tests that
attempt to use for LTO fail. This was observed on some of the bots.

Differential Revision: https://reviews.llvm.org/D81629
2020-06-29 15:09:05 -07:00