Commit Graph

418 Commits

Author SHA1 Message Date
Kai Luo a594362436 [AIX] Rename shared_libraries_to_archive -> objects_to_archive. NFC. 2021-08-31 06:47:06 +00:00
Brian Cain f2880329eb [compiler-rt] Add hexagon support to profile
Adds build support for hexagon linux to profile
2021-08-17 19:59:24 -07:00
Brian Cain 065274b9cf [compiler-rt] Add hexagon support to safestack
Adds build support for hexagon linux to safestack.
2021-08-17 19:59:24 -07:00
Brian Cain c172726f6f [compiler-rt] Add hexagon support to CFI
Adds build support for hexagon linux to cfi.
2021-08-17 19:59:24 -07:00
Brian Cain 5b650b1467 [sanitizer] Add hexagon support to scudo
Adds build support for hexagon linux to scudo, scudo-standalone
2021-08-17 19:59:24 -07:00
Brian Cain 217191b70a [sanitizer] Add hexagon support to ubsan
Adds build support for hexagon linux to ubsan.
2021-08-17 19:59:24 -07:00
Brian Cain 043926a3a0 [sanitizer] Add hexagon support to asan
Adds build support for hexagon linux to asan.
2021-08-17 19:59:24 -07:00
Brian Cain 5ac9d41067 [sanitizer] Add hexagon support to lsan
Adds build support for hexagon linux to lsan.
2021-08-17 19:59:24 -07:00
Brian Cain 9ea59e1cd0 [sanitizer] Add hexagon support to sanitizer-common
Adds build support for hexagon linux to sanitizer common.
2021-08-17 19:59:24 -07:00
Daniel Kolesa b7611ad0b1 [CMake] Enable clang_rt.crt{begin,end} on ppc32/ppc64 2021-08-11 18:25:41 -07:00
Michael Jones 6ed60fb8a2 [libc] add integration tests for scudo in libc
This change adds tests to make sure that SCUDO is being properly
included with llvm libc. This change also adds the toggles to properly
use SCUDO, as GWP-ASan is enabled by default and must be included for
SCUDO to function.

Reviewed By: sivachandra, hctim

Differential Revision: https://reviews.llvm.org/D106919
2021-08-04 20:06:09 +00:00
Vitaly Buka f08229f49e [sanitizer] Remove cpplint
As code diverge from Google style we need
to add more and more exceptions to suppress
conflicts with clang-format and clang-tidy.
As this point it does not provide a additional value.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D107197
2021-07-30 15:21:04 -07:00
Raul Tambre b31080c596 [compiler-rt][CMake][arm64] Use a custom target for symlinking LSE sources
On Apple platforms the builtins may be built for both arm64 and arm64e.
With Makefile generators separate targets are built using Make sub-invocations.
This causes a race when creating the symlink which may sometimes fail.

Work around this by using a custom target that the builtin targets depend on.
This causes any sub-invocations to depend on the symlinks having been created before.

Mailing list thread: https://lists.llvm.org/pipermail/llvm-dev/2021-July/151822.html

Reviewed By: thakis, steven_wu

Differential Revision: https://reviews.llvm.org/D106305
2021-07-26 21:42:27 +03:00
John Ericson 1e03c37b97 Prepare Compiler-RT for GnuInstallDirs, matching libcxx, document all
This is a second attempt at D101497, which landed as
9a9bc76c0e but had to be reverted in
8cf7ddbdd4.

This issue was that in the case that `COMPILER_RT_INSTALL_PATH` is
empty, expressions like "${COMPILER_RT_INSTALL_PATH}/bin" evaluated to
"/bin" not "bin" as intended and as was originally.

One solution is to make `COMPILER_RT_INSTALL_PATH` always non-empty,
defaulting it to `CMAKE_INSTALL_PREFIX`. D99636 adopted that approach.
But, I think it is more ergonomic to allow those project-specific paths
to be relative the global ones. Also, making install paths absolute by
default inhibits the proper behavior of functions like
`GNUInstallDirs_get_absolute_install_dir` which make relative install
paths absolute in a more complicated way.

Given all this, I will define a function like the one asked for in
https://gitlab.kitware.com/cmake/cmake/-/issues/19568 (and needed for a
similar use-case).

---

Original message:

Instead of using `COMPILER_RT_INSTALL_PATH` through the CMake for
complier-rt, just use it to define variables for the subdirs which
themselves are used.

This preserves compatibility, but later on we might consider getting rid
of `COMPILER_RT_INSTALL_PATH` and just changing the defaults for the
subdir variables directly.

---

There was a seaming bug where the (non-Apple) per-target libdir was
`${target}` not `lib/${target}`. I suspect that has to do with the docs
on `COMPILER_RT_INSTALL_PATH` saying was the library dir when that's no
longer true, so I just went ahead and fixed it, allowing me to define
fewer and more sensible variables.

That last part should be the only behavior changes; everything else
should be a pure refactoring.

---

I added some documentation of these variables too. In particular, I
wanted to highlight the gotcha where `-DSomeCachePath=...` without the
`:PATH` will lead CMake to make the path absolute. See [1] for
discussion of the problem, and [2] for the brief official documentation
they added as a result.

[1]: https://cmake.org/pipermail/cmake/2015-March/060204.html

[2]: https://cmake.org/cmake/help/latest/manual/cmake.1.html#options

In 38b2dec37e the problem was somewhat
misidentified and so `:STRING` was used, but `:PATH` is better as it
sets the correct type from the get-go.

---

D99484 is the main thrust of the `GnuInstallDirs` work. Once this lands,
it should be feasible to follow both of these up with a simple patch for
compiler-rt analogous to the one for libcxx.

Reviewed By: phosek, #libc_abi, #libunwind

Differential Revision: https://reviews.llvm.org/D105765
2021-07-13 15:21:41 +00:00
Martin Storsjö 8cf7ddbdd4 Revert "Prepare Compiler-RT for GnuInstallDirs, matching libcxx"
This reverts commit 9a9bc76c0e.

That commit broke "ninja install" when building compiler-rt for mingw
targets, building standalone (pointing cmake at the compiler-rt
directory) with cmake 3.16.3 (the one shipped in ubuntu 20.04), with
errors like this:

-- Install configuration: "Release"
CMake Error at cmake_install.cmake:44 (file):
  file cannot create directory: /include/sanitizer.  Maybe need
  administrative privileges.
Call Stack (most recent call first):
  /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/cmake_install.cmake:37 (include)

FAILED: include/CMakeFiles/install-compiler-rt-headers
cd /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/include && /usr/bin/cmake -DCMAKE_INSTALL_COMPONENT="compiler-rt-headers" -P /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/cmake_install.cmake
ninja: build stopped: subcommand failed.
2021-07-10 10:45:54 +03:00
John Ericson 9a9bc76c0e Prepare Compiler-RT for GnuInstallDirs, matching libcxx
Instead of using `COMPILER_RT_INSTALL_PATH` through the CMake for
complier-rt, just use it to define variables for the subdirs which
themselves are used.

This preserves compatibility, but later on we might consider getting rid
of `COMPILER_RT_INSTALL_PATH` and just changing the defaults for the
subdir variables directly.

---

There was a seaming bug where the (non-Apple) per-target libdir was
`${target}` not `lib/${target}`. I suspect that has to do with the docs
on `COMPILER_RT_INSTALL_PATH` saying was the library dir when that's no
longer true, so I just went ahead and fixed it, allowing me to define
fewer and more sensible variables.

That last part should be the only behavior changes; everything else
should be a pure refactoring.

---

D99484 is the main thrust of the `GnuInstallDirs` work. Once this lands,
it should be feasible to follow both of these up with a simple patch for
compiler-rt analogous to the one for libcxx.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D101497
2021-07-09 20:41:53 +00:00
Kai Luo ae79854e1c [AIX][compiler-rt] Deliver libatomic.a at top level library directory
Install libatomic.a in top level library directory so that compiler can find it in search directories.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D104908
2021-06-29 03:14:20 +00:00
Petr Hosek 959dbd1761 Revert "Revert "[cmake] [compiler-rt] Call llvm_setup_rpath() when adding shared libraries.""
This reverts commit 21c008d5a5 since
it broke the build on macOS and Windows with the following error:

  The install of the clang_rt.<na,e> target requires changing an
  RPATH from the build tree, but this is not supported with the Ninja
  generator unless on an ELF-based platform.  The
  CMAKE_BUILD_WITH_INSTALL_RPATH variable may be set to avoid this relinking
  step.
2021-06-22 16:11:29 -07:00
Petr Hosek 21c008d5a5 Revert "[cmake] [compiler-rt] Call llvm_setup_rpath() when adding shared libraries."
This reverts commit 78fd93e039 as
a follow up to D91099.
2021-06-22 12:42:39 -07:00
Kai Luo 6393164cf2 [AIX][compiler-rt] Fix cmake build of libatomic for cmake-3.16+
cmake-3.16+ for AIX changes the default behavior of building a `SHARED` library which breaks AIX's build of libatomic, i.e., cmake-3.16+ builds `SHARED` as an archive of dynamic libraries. To fix it, we have to build `libatomic.so.1` as `MODULE` which keeps `libatomic.so.1` as an normal dynamic library.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D103786
2021-06-12 00:12:29 +00:00
John Paul Adrian Glaubitz 27ef43d279 [compiler-rt] Add platform detection support for x32
Currently, the compiler-rt build system checks only whether __X86_64
is defined to determine whether the default compiler-rt target arch
is x86_64. Since x32 defines __X86_64 as well, we must also check that
the default pointer size is eight bytes and not four bytes to properly
detect a 64-bit x86_64 compiler-rt default target arch.

Reviewed By: hvdijk, vitalybuka

Differential Revision: https://reviews.llvm.org/D99988
2021-06-11 19:44:04 +01:00
Jon Roelofs cc9c895d88 [compiler-rt][profile] Explicitly specify PROFILE_SOURCES extensions. NFC 2021-05-21 14:46:08 -07:00
Kai Luo d56729b4a4 [AIX][compiler-rt] Build and install standalone libatomic
On AIX, we have to ship `libatomic.a` for compatibility. First, a new `clang_rt.atomic` is added. Second, use added cmake modules for AIX, we are able to build a compatible libatomic.a for AIX. The second step can't be perfectly implemented with cmake now since AIX's archive approach is kinda unique, i.e., archiving shared libraries into a static archive file.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D102155
2021-05-16 05:04:08 +00:00
Dan Liew 6f4f0afaa8 [Compiler-rt] Fix bug when considering CMake path returned by llvm-config.
The previous check was wrong because it only checks that the LLVM CMake
directory exists. However, it's possible that the directory exists but
the `LLVMConfig.cmake` file does not. When this happens we would
incorectly try to include the non-existant file.

To fix this we make the check stricter by checking that the file
we want to include actually exists.

This is a follow up to fd28517d87.

rdar://76870467
2021-04-20 11:57:20 -07:00
Dan Liew fd28517d87 [CMake][Compiler-rt] Make it possible to configure standalone compiler-rt without `LLVMConfig.cmake`.
Previously it wasn't possible to configure a standalone compiler-rt
build if the `LLVMConfig.cmake` file isn't present in a shipped
toolchain.

This patch adds a fallback behaviour for when `LLVMConfig.cmake` is not
available in the toolchain being used for configure. The fallback
behaviour mocks out the bare minimum required to make a configure
succeed when the host is Darwin. Support for other platforms could
be added in future patches.

The new code path is taken either in one of the following cases:

* `llvm-config` is not available.
* `llvm-config` is available but it provides an invalid path for the CMake files.

The motivation here is to be able to generate the compiler-rt lit test
suites for an arbitrary LLVM toolchain and then run the tests against
it.

The invocation to do this looks something like.

```
CC=/path/to/cc \
CXX=/path/to/c++ \
cmake \
  -G Ninja \
  -DLLVM_CONFIG_PATH=/path/to/llvm-config \
  -DCOMPILER_RT_INCLUDE_TESTS=ON \
  /path/to/llvm-project/compiler-rt

 # Note we don't compile compiler-rt in this workflow.
bin/llvm-lit -v test/path/to/generated/test_suite
```

A possible alternative approach is to configure the
`cmake/modules/LLVMConfig.cmake.in` file in the LLVM source tree
and then include it. This approach was not taken because it is more
complicated.

An interesting side benefit of this patch is that it is now
possible to configure on Darwin without `llvm-config` being available
by configuring with `-DLLVM_CONFIG_PATH=""`. This moves us a step
closer to a world where no LLVM build artefacts are required to
build compiler-rt.

rdar://76016632

Differential Revision: https://reviews.llvm.org/D99621
2021-04-06 08:31:18 -07:00
Dan Liew fa818bb035 [CMake][Compiler-rt] Compute `LLVM_MAIN_SRC_DIR` assuming the monorepo
layout.

When doing a standalone compiler-rt build we currently rely on
getting information from the `llvm-config` binary. Previously
we would rely on calling `llvm-config --src-root` to find the
LLVM sources. Unfortunately the returned path could easily be wrong
if the sources were built on another machine.

Now that compiler-rt is part of a monorepo we can easily fix this
problem by finding the LLVM source tree next to `compiler-rt` in
the monorepo. We do this regardless of whether or not the `llvm-config`
binary is available which moves us one step closer to not requiring
`llvm-config` to be available.

To try avoid anyone breaking anyone who relies on the current behavior,
if the path assuming the monorepo layout doesn't exist we invoke
`llvm-config --src-root` to get the path. A deprecation warning is
emitted if this path is taken because we should remove this path
in the future given that other runtimes already assume the monorepo
layout.

We also now emit a warning if `LLVM_MAIN_SRC_DIR` does not exist.
The intention is that this should be a hard error in future but
to avoid breaking existing users we'll keep this as a warning
for now.

rdar://76016632

Differential Revision: https://reviews.llvm.org/D99620
2021-04-06 08:31:18 -07:00
Martin Storsjö 8e11bede3a [compiler-rt] Produce the right arch suffix for arm libraries
If producing libraries with an arch suffix (i.e. if
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR isn't set), we append the
architecture name. However, for arm, clang doesn't look for libraries
with the full architecture name, but only looks for "arm" and "armhf".

Try to deduce what the full target triple might have been, and use
that for deciding between "arm" and "armhf".

This tries to reapply this bit from D98173, that had to be reverted
in 7b153b43d3 due to affecting how
the builtins themselves are compiled, not only affecting the output
file name.

Differential Revision: https://reviews.llvm.org/D98452
2021-03-18 14:58:58 +02:00
Martin Storsjö 7b153b43d3 [compiler-rt] Partially revert 8bd2722f65
Don't normalize arm architecture names; doing that loses the ability
to pick the right implementation of builtins for each architecture
variant. When building compiler-rt builtins as part of a
runtimes build, builtins for multiple armv* variants could be built
in the same directory, and with the simplified architecture name,
they'd all be built in the same directory, overlapping each other.
2021-03-11 22:41:10 +02:00
Martin Storsjö 8bd2722f65 [compiler-rt] Normalize i?86 to i386 and armv* to arm for COMPILER_RT_DEFAULT_TARGET_ARCH
This corresponds to getArchNameForCompilerRTLib in clang; any
32 bit x86 architecture triple (except on android, but those
exceptions are already handled in compiler-rt on a different level)
get the compiler rt library names with i386; arm targets get either
"arm" or "armhf". (Mapping to "armhf" is handled in the toplevel
CMakeLists.txt.)

Differential Revision: https://reviews.llvm.org/D98173
2021-03-10 08:43:54 +02:00
Petr Hosek 61a792b39b [CMake] Rename RUNTIMES_BUILD to LLVM_RUNTIMES_BUILD
This avoid potential conflict with other internal variables.

Differential Revision: https://reviews.llvm.org/D97838
2021-03-03 10:58:51 -08:00
Petr Hosek 6e3946c9f5 [runtimes] Use standalone build only for compiler-rt
compiler-rt needs to use standalone build because of the assumptions
made by its build, but other runtimes can use non-standalone build.

Differential Revision: https://reviews.llvm.org/D97575
2021-03-03 00:06:20 -08:00
Petr Hosek b3ac90da1d Revert "[runtimes] Use standalone build only for compiler-rt"
This reverts commit 4e421b2323 as this
seemed to have broke Python 3 executable detection on some builders.
2021-03-02 16:59:32 -08:00
Petr Hosek 4e421b2323 [runtimes] Use standalone build only for compiler-rt
compiler-rt needs to use standalone build because of the assumptions
made by its build, but other runtimes can use non-standalone build.

Differential Revision: https://reviews.llvm.org/D97575
2021-03-02 16:21:35 -08:00
Fangrui Song 7afdc89c20 [sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).

Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).

This patch makes `ninja asan cfi lsan msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:

* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)

Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).

Many sanitizer features are available now.

```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
  Unsupported      : 185
  Passed           : 512
  Expectedly Failed:   1
  Failed           :  12

% ninja check-ubsan check-ubsan-minimal check-memprof # all passed

% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
  Unsupported      : 264
  Passed           :  80
  Expectedly Failed:   8
  Failed           :  32

% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
  Unsupported:  7
  Passed     : 65
  Failed     : 22

% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
  Unsupported      :   6
  Passed           : 764
  Expectedly Failed:   2
  Failed           :  58

% ninja check-tsan
Testing Time: 23.21s
  Unsupported      :  86
  Passed           : 295
  Expectedly Failed:   1
  Failed           :  25
```

Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.

Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014

Note: we need to place `_FILE_OFFSET_BITS` above `#include "sanitizer_platform.h"` to avoid `#define __USE_FILE_OFFSET64 1` in 32-bit ARM `features.h`

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93848
2021-01-06 10:55:40 -08:00
Oliver Stannard 4839378ca0 Revert "[sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl"
This reverts commit b7718b6175, because it
is causing build failures on all 32-bit ARM bots which build
compiler-rt.
2021-01-06 10:31:59 +00:00
Fangrui Song b7718b6175 Reland D93848 "[sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl""
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).

Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).

This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:

* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)

Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).

Many sanitizer features are available now.

```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
  Unsupported      : 185
  Passed           : 512
  Expectedly Failed:   1
  Failed           :  12

% ninja check-ubsan check-ubsan-minimal check-memprof # all passed

% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
  Unsupported      : 264
  Passed           :  80
  Expectedly Failed:   8
  Failed           :  32

% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
  Unsupported:  7
  Passed     : 65
  Failed     : 22

% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
  Unsupported      :   6
  Passed           : 764
  Expectedly Failed:   2
  Failed           :  58

% ninja check-tsan
Testing Time: 23.21s
  Unsupported      :  86
  Passed           : 295
  Expectedly Failed:   1
  Failed           :  25
```

Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.

Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93848
2021-01-05 16:18:43 -08:00
Nico Weber fe9976c02c Revert "[sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl"
...and follow-ups. It still doesn't build on Android, see https://reviews.llvm.org/D93848#2476310

This reverts commit a92d01534f.
This reverts commit 52d7e183bf.
This reverts commit 34489da81b.
2021-01-02 18:59:26 -05:00
Fangrui Song a92d01534f [sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).

Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).

This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:

* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)

Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).

Many sanitizer features are available now.

```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
  Unsupported      : 185
  Passed           : 512
  Expectedly Failed:   1
  Failed           :  12

% ninja check-ubsan check-ubsan-minimal check-memprof # all passed

% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
  Unsupported      : 264
  Passed           :  80
  Expectedly Failed:   8
  Failed           :  32

% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
  Unsupported      :   6
  Passed           : 764
  Expectedly Failed:   2
  Failed           :  58

% ninja check-tsan
Testing Time: 23.21s
  Unsupported      :  86
  Passed           : 295
  Expectedly Failed:   1
  Failed           :  25
```

Used `ASAN_OPTIONS=verbosity=2` to verify no unneeded interceptors.

Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93848
2020-12-31 00:44:25 -08:00
Raul Tambre a72d462eb9 Re-apply "[CMake][compiler-rt][AArch64] Avoid preprocessing LSE builtins separately"
aa772fc85e (D92530) has landed fixing relocations on Darwin.
3000c19df6 (D93236) has landed working around an assembly parser bug on Darwin.
Previous quick-fix d9697c2e6b (D93198) included in this commit.

Invoking the preprocessor ourselves is fragile and would require us to replicate CMake's handling of definitions, compiler flags, etc for proper compatibility.
In my toolchain builds this notably resulted in a bunch of warnings from unused flags as my CMAKE_C_FLAGS includes CPU-specific optimization options.
Notably this part was already duplicating the logic for VISIBILITY_HIDDEN define.

Instead, symlink the files and set the proper set of defines on each.
This should also be faster as we avoid invoking the compiler multiple times.

Fixes https://llvm.org/PR48494

Differential Revision: https://reviews.llvm.org/D93278
2020-12-15 08:18:07 +02:00
Raul Tambre c21df2a79c Revert "Re-apply "[CMake][compiler-rt][AArch64] Avoid preprocessing LSE builtins separately""
This reverts commit 03ebe1937192c247c4a7b8ec19dde2cf9845c914.

It's still breaking bots, e.g. http://green.lab.llvm.org/green/job/clang-stage1-RA/17027/console although it doesn't change any actual code.
The compile errors don't make much sense either. Revert for now.

Differential Revision: https://reviews.llvm.org/D93228
2020-12-14 18:43:55 +02:00
Raul Tambre d0797e62fa Re-apply "[CMake][compiler-rt][AArch64] Avoid preprocessing LSE builtins separately"
aa772fc85e (D92530) has landed fixing Apple builds.
Previous quick-fix d9697c2e6b (D93198) included in this commit.

Invoking the preprocessor ourselves is fragile and would require us to replicate CMake's handling of definitions, compiler flags, etc for proper compatibility.
In my toolchain builds this notably resulted in a bunch of warnings from unused flags as my CMAKE_C_FLAGS includes CPU-specific optimization options.
Notably this part was already duplicating the logic for VISIBILITY_HIDDEN define.

Instead, symlink the files and set the proper set of defines on each.
This should also be faster as we avoid invoking the compiler multiple times.

Fixes https://llvm.org/PR48494

Differential Revision: https://reviews.llvm.org/D93211
2020-12-14 16:45:48 +02:00
Raul Tambre 617cd01a4b Revert "[CMake][compiler-rt][AArch64] Avoid preprocessing LSE builtins separately"
Causing issues on Apple buildbots.
http://green.lab.llvm.org/green/job/clang-stage1-RA/17019/console

This reverts commit 33b740f8dc.
This reverts commit d9697c2e6b.

Differential Revision: https://reviews.llvm.org/D93199
2020-12-14 11:42:28 +02:00
Raul Tambre d9697c2e6b [compiler-rt][CMake] Define HAS_ASM_LSE on Apple if available
Should hopefully fix 33b740f8dc (D93178) failing on bots.

Differential Revision: https://reviews.llvm.org/D93198
2020-12-14 11:26:24 +02:00
Robert Underwood 16de50895e honor Python2_EXECUTABLE and Python3_EXECUTABLE when they are passed to cmake
CMake's find_package(Python3) and find_package(Python2) packages have a PYTHON_EXECUTABLE, Python2_EXECUTABLE, and Python3_EXECUTABLE cmake variables which control which version of python is built against.  As far as I can tell, the rest of LLVM honors these variables. This can cause the build process to fail when  if the automatically selected version of Python can't run due to modifications of LD_LIBRARY_PATH when using spack.  The corresponding Spack issue is https://github.com/spack/spack/issues/19908.  The corresponding LLVM issue is 48180

I believe an appropriate fix is to add the variables to the list of PASSTHROUGH_VARIABLES in cmake/Modules/AddCompilerRT.cmake, and this fixed compilation errors for me.

This bug affects distributions like Gentoo and package managers like Spack which allow for combinatorial versioning.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D91536
2020-11-17 17:49:14 -06:00
Petr Hosek d11710dae6 [NFC][CMake] Move some COMPILER_RT variables setup
Part of D88922
2020-10-30 20:09:50 -07:00
Petr Hosek 220de1f32a Revert "[CMake] Avoid accidental C++ standard library dependency in sanitizers"
This reverts commit 287c318690 which broke
sanitizer tests that use C++ standard library.
2020-10-14 18:44:09 -07:00
Petr Hosek 287c318690 [CMake] Avoid accidental C++ standard library dependency in sanitizers
While sanitizers don't use C++ standard library, we could still end
up accidentally including or linking it just by the virtue of using
the C++ compiler. Pass -nostdinc++ and -nostdlib++ to avoid these
accidental dependencies.

Differential Revision: https://reviews.llvm.org/D88922
2020-10-14 18:26:56 -07:00
Petr Hosek 4540d66248 [CMake] Track TSan's dependency on C++ headers
TSan relies on C++ headers, so when libc++ is being built as part of
the runtimes build, include an explicit dependency on cxx-headers which
is the same approach that's already used for other sanitizers.

Differential Revision: https://reviews.llvm.org/D88912
2020-10-06 13:58:35 -07:00
David Tenty c455961479 [compiler-rt][AIX] Add CMake support for 32-bit Power builds
This patch enables support for building compiler-rt builtins for 32-bit
Power arch on AIX. For now, we leave out the specialized ppc builtin
implementations for 128-bit long double and friends since those will
need some special handling for AIX.

Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D87383
2020-09-22 16:08:58 -04:00
Raul Tambre 45344cf7ac [CMake][compiler-rt][libunwind] Compile assembly files as ASM not C, unify workarounds
It isn't very wise to pass an assembly file to the compiler and tell it to compile as a C file and hope that the compiler recognizes it as assembly instead.
Simply don't mark the file as C and CMake will recognize the rest.

This was attempted earlier in https://reviews.llvm.org/D85706, but reverted due to architecture issues on Apple.
Subsequent digging revealed a similar change was done earlier for libunwind in https://reviews.llvm.org/rGb780df052dd2b246a760d00e00f7de9ebdab9d09.
Afterwards workarounds were added for MinGW and Apple:
* https://reviews.llvm.org/rGb780df052dd2b246a760d00e00f7de9ebdab9d09
* https://reviews.llvm.org/rGd4ded05ba851304b26a437896bc3962ef56f62cb

The workarounds in libunwind and compiler-rt are unified and comments added pointing to each other.
The workaround is updated to only be used for MinGW for CMake versions before 3.17, which fixed the issue (https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4287).

Additionally fixed Clang not being passed as the assembly compiler for compiler-rt runtime build.

Example error:
[525/634] Building C object lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
FAILED: lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
/opt/tooling/drive/host/bin/clang --target=aarch64-linux-gnu -I/opt/tooling/drive/llvm/compiler-rt/lib/tsan/.. -isystem /opt/tooling/drive/toolchain/opt/drive/toolchain/include -x c -Wall -Wno-unused-parameter -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fPIE -fno-rtti -Wframe-larger-than=530 -Wglobal-constructors --sysroot=. -MD -MT lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -MF lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o.d -o lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -c /opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S
/opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S:29:1: error: expected identifier or '('
.section .text
^
1 error generated.

Differential Revision: https://reviews.llvm.org/D86308
2020-08-27 15:40:15 +03:00