when building with an IDE so that header files show up in the UI.
This massively improves the development workflow in IDEs.
To implement this a new function `compiler_rt_process_sources(...)` has
been added that adds header files to the list of sources when the
generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no
changes are made to the list of source files.
The function can be passed a list of headers via the
`ADDITIONAL_HEADERS` argument. For each runtime library a list of
explicit header files has been added and passed via
`ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of
headers was already present but it was stale and has been updated
to reflect the current state of the source tree.
The original version of this patch used file globbing (`*.{h,inc,def}`)
to find the headers but the approach was changed due to this being a
CMake anti-pattern (if the list of headers changes CMake won't
automatically re-generate if globbing is used).
The LLVM repo contains a similar function named `llvm_process_sources()`
but we don't use it here for several reasons:
* It depends on the `LLVM_ENABLE_OPTION` cache variable which is
not set in standalone compiler-rt builds.
* We would have to `include(LLVMProcessSources)` which I'd like to
avoid because it would include a bunch of stuff we don't need.
Differential Revision: https://reviews.llvm.org/D48422
llvm-svn: 336663
Summary:
This patch follows D48373.
The point is to be able to use Scudo with `-fsanitize-minimal-runtime`. For that
we need a runtime that doesn't embed the UBSan one. This results in binaries
that can be compiled with `-fsanitize=scudo,integer -fsanitize-minimal-runtime`.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: mgorny, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48377
llvm-svn: 335296
Summary:
The current `FailureHandler` mechanism was fairly opaque with regard to the
failure reason due to using `CHECK(0)`. Scudo is a bit different from the other
Sanitizers as it prefers to avoid spurious processing in its failure path. So
we just `dieWithMessage` using a somewhat explicit string.
Adapted the tests for the new strings.
While this takes care of the `OnBadRequest` & `OnOOM` failures, the next step
is probably to migrate the other Scudo failures in the same failes (header
corruption, invalid state and so on).
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: filcab, mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48199
llvm-svn: 334843
Summary:
Host symbolizer & stacktraces related code in their own RT:
`RTSanitizerCommonSymbolizer`, which is "libcdep" by nature. Symbolizer &
stacktraces specific code that used to live in common files is moved to a new
file `sanitizer_symbolizer_report.cc` as is.
The purpose of this is the enforce a separation between code that relies on
symbolization and code that doesn't. This saves the inclusion of spurious code
due to the interface functions with default visibility, and the extra data
associated.
The following sanitizers makefiles were modified & tested locally:
- dfsan: doesn't require the new symbolizer RT
- esan: requires it
- hwasan: requires it
- lsan: requires it
- msan: requires it
- safestack: doesn't require it
- xray: doesn't require it
- tsan: requires it
- ubsan: requires it
- ubsan_minimal: doesn't require it
- scudo: requires it (but not for Fuchsia that has a minimal runtime)
This was tested locally on Linux, Android, Fuchsia.
Reviewers: alekseyshl, eugenis, dberris, kubamracek, vitalybuka, dvyukov, mcgrathr
Reviewed By: alekseyshl, vitalybuka
Subscribers: srhines, kubamracek, mgorny, krytarowski, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D45457
llvm-svn: 330131
Summary:
Now that common options are propagated again for runtimes build with D45507,
the -f{data,function}-sections flags are now duplicates, remove them.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D45575
llvm-svn: 329925
Summary:
Fuchsia requires its Scudo shared runtime to not be C++ dependant. Since they
don't use UBSan in conjunction with Scudo, we can just remove the runtime,
and add the extra `nostdinc++` and `nostdlib++` flags. No need for Coverage
either. This allows to keep things going while working on additional splits
of sanitizer_commong and a more minimal runtime.
Reviewers: phosek, flowerhack, alekseyshl
Reviewed By: phosek, alekseyshl
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44791
llvm-svn: 328625
Summary:
`sanitizer_common`'s coverage support is fairly well separated, and libcdep by
default. Several sanitizers don't make use of coverage, and as far as I can
tell do no benefit from the extra dependencies pulled in by the coverage public
interface functions.
The following sanitizers call `InitializeCoverage` explicitely: MSan, ASan,
LSan, HWAsan, UBSan. On top of this, any sanitizer bundling RTUBSan should
add the coverage RT as well: ASan, Scudo, UBSan, CFI (diag), TSan, MSan, HWAsan.
So in the end the following have no need: DFSan, ESan, CFI, SafeStack (nolibc
anyway), XRay, and the upcoming Scudo minimal runtime.
I tested this with all the sanitizers check-* with gcc & clang, and in
standalone on Linux & Android, and there was no issue. I couldn't test this on
Mac, Fuchsia, BSDs, & Windows for lack of an environment, so adding a bunch of
people for additional scrunity. I couldn't test HWAsan either.
Reviewers: eugenis, vitalybuka, alekseyshl, flowerhack, kubamracek, dberris, rnk, krytarowski
Reviewed By: vitalybuka, alekseyshl, flowerhack, dberris
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44701
llvm-svn: 328204
Summary:
If not using `-Wl,--gc-sections`, a whole lot of unused `sanitizer_common` code
and related static variables are pulled into the shared library.
Keep the binary size smaller, and its memory footprint smaller as well, by
using the compiler flags `-ffunction-section` & `-fdata-sections` by default,
as well as the linker flags `-Wl,--gc-sections`.
Current experiments show a large discrepency between binary sizes generated
by gcc (big) and clang (small). I am not sure yet how I can make a test that
would encompass both, so it's an outstanding work item.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: mgorny, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D44121
llvm-svn: 326833
Summary:
This is a follow-up to D42506.
There are a few of things that bothered me about `scudo_interceptors.cpp`:
- the filename is a misnomer: it intercepts some functions, but the rest (C++)
is actually in `scudo_new_delete.cpp`. I feel like `scudo_malloc.cpp` is more
appropriate (ASan uses the same naming scheme);
- we do not need "full" interceptors, since we are never accessing the
unsanitized version of the functions, we just need the
`extern "C" INTERCEPTOR_ATTRIBUTE` part of it to just call our functions;
- a couple of functions where duplicated while they could just be `ALIAS`'d;
- use the `SANITIZER_INTERCEPT_*` defines to hide the unneeded interceptors;
- use `SIZE_T` instead of `uptr`: while it's the same behind the curtain,
the former is meant for this use case.
In the end there is no functional change on the currently supported platforms
(Linux, Android).
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: mgorny, hintonda, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42546
llvm-svn: 323464
Summary:
We somehow never did it, and it raised no issue until now, when trying to
enable Fuchsia as a supported Scudo platform in the cmake config.
So propagate SANITIZER_COMMON_LINK_FLAGS for now.
Reviewers: alekseyshl, flowerhack
Reviewed By: flowerhack
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42314
llvm-svn: 322999
Summary:
This is needed for the shared runtime since we are pulling RTUbsan in.
Otherwise some builds might fail with errors such as:
`error: undefined reference to '__dynamic_cast'`
Reviewers: alekseyshl, srhines
Reviewed By: srhines
Subscribers: kongyi, pirama, chh, mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D41995
llvm-svn: 322389
Summary:
Initially, Scudo had a monolithic design where both C and C++ functions were
living in the same library. This was not necessarily ideal, and with the work
on -fsanitize=scudo, it became more apparent that this needed to change.
We are splitting the new/delete interceptor in their own C++ library. This
allows more flexibility, notably with regard to std::bad_alloc when the work is
done. This also allows us to not link new & delete when using pure C.
Additionally, we add the UBSan runtimes with Scudo, in order to be able to have
a -fsanitize=scudo,undefined in Clang (see work in D39334).
The changes in this patch:
- split the cxx specific code in the scudo cmake file into a new library;
(remove the spurious foreach loop, that was not necessary)
- add the UBSan runtimes (both C and C++);
- change the test cmake file to allow for specific C & C++ tests;
- make C tests pure C, rename their extension accordingly.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D39461
llvm-svn: 317097
Summary:
Up to now, the Scudo cmake target only provided a static library that had to be
linked to an executable to benefit from the hardened allocator.
This introduces a shared library as well, that can be LD_PRELOAD'ed.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D38980
llvm-svn: 316342
Summary:
Previous parts: D38139, D38183.
In this part of the refactor, we abstract the Linux vs Android TSD dissociation
in favor of a Exclusive vs Shared one, allowing for easier platform introduction
and configuration.
Most of this change consist of shuffling the files around to reflect the new
organization.
We introduce `scudo_platform.h` where platform specific definition lie. This
involves the TSD model and the platform specific allocator parameters. In an
upcoming CL, those will be configurable via defines, but we currently stick
with conservative defaults.
Reviewers: alekseyshl, dvyukov
Reviewed By: alekseyshl, dvyukov
Subscribers: srhines, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D38244
llvm-svn: 314224
Summary:
This change adds Android support to the allocator (but doesn't yet enable it in
the cmake config), and should be the last fragment of the rewritten change
D31947.
Android has more memory constraints than other platforms, so the idea of a
unique context per thread would not have worked. The alternative chosen is to
allocate a set of contexts based on the number of cores on the machine, and
share those contexts within the threads. Contexts can be dynamically reassigned
to threads to prevent contention, based on a scheme suggested by @dvyuokv in
the initial review.
Additionally, given that Android doesn't support ELF TLS (only emutls for now),
we use the TSan TLS slot to make things faster: Scudo is mutually exclusive
with other sanitizers so this shouldn't cause any problem.
An additional change made here, is replacing `thread_local` by `THREADLOCAL`
and using the initial-exec thread model in the non-Android version to prevent
extraneous weak definition and checks on the relevant variables.
Reviewers: kcc, dvyukov, alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D32649
llvm-svn: 302300
Summary:
This change introduces scudo_tls.h & scudo_tls_linux.cpp, where we move the
thread local variables used by the allocator, namely the cache, quarantine
cache & prng. `ScudoThreadContext` will hold those. This patch doesn't
introduce any new platform support yet, this will be the object of a later
patch. This also changes the PRNG so that the structure can be POD.
Reviewers: kcc, dvyukov, alekseyshl
Reviewed By: dvyukov, alekseyshl
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D32440
llvm-svn: 301584
Summary:
ARM & AArch64 runtime detection for hardware support of CRC32 has been added
via check of the AT_HWVAL auxiliary vector.
Following Michal's suggestions in D28417, the CRC32 code has been further
changed and looks better now. When compiled with full relro (which is strongly
suggested to benefit from additional hardening), the weak symbol for
computeHardwareCRC32 is read-only and the assembly generated is fairly clean
and straight forward. As suggested, an additional optimization is to skip
the runtime check if SSE 4.2 has been enabled globally, as opposed to only
for scudo_crc32.cpp.
scudo_crc32.h has no purpose anymore and was removed.
Reviewers: alekseyshl, kcc, rengolin, mgorny, phosek
Reviewed By: rengolin, mgorny
Subscribers: aemerson, rengolin, llvm-commits
Differential Revision: https://reviews.llvm.org/D28574
llvm-svn: 292409
Summary:
As raised in D28304, enabling SSE 4.2 for the whole Scudo tree leads to the
emission of SSE 4.2 instructions everywhere, while the runtime checks only
applied to the CRC32 computing function.
This patch separates the CRC32 function taking advantage of the hardware into
its own file, and only enabled -msse4.2 for that file, if detected to be
supported by the compiler.
Another consequence of removing SSE4.2 globally is realizing that memcpy were
not being optimized, which turned out to be due to the -fno-builtin in
SANITIZER_COMMON_CFLAGS. So we now explicitely enable builtins for Scudo.
The resulting assembly looks good, with some CALLs are introduced instead of
the CRC32 code being inlined.
Reviewers: kcc, mgorny, alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D28417
llvm-svn: 291570
Disable the code appending -msse4.2 flag implicitly when the compiler
supports it. The compiler support for this flags do not indicate that
the underlying CPU will support SSE4.2, and passing it may result in
SSE4.2 code being emitted *implicitly*.
If the target platform supports SSE4.2 appropriately, the relevant bits
should be already enabled via -march= or equivalent. In this case
passing -msse4.2 is redundant.
If a runtime detection is desired (which seems to be a case with SCUDO),
then (as gcc manpage points out) the specific SSE4.2 needs to be
isolated into a separate file, the -msse4.2 flag can be forced only
for that file and the function defined in that file can only be called
when the CPU is determined to support SSE4.2.
This fixes SIGILL on SCUDO when it is compiled using gcc-5.4.
Differential Revision: https://reviews.llvm.org/D28304
llvm-svn: 291217
Summary:
This update introduces i386 support for the Scudo Hardened Allocator, and
offers software alternatives for functions that used to require hardware
specific instruction sets. This should make porting to new architectures
easier.
Among the changes:
- The chunk header has been changed to accomodate the size limitations
encountered on 32-bit architectures. We now fit everything in 64-bit. This
was achieved by storing the amount of unused bytes in an allocation rather
than the size itself, as one can be deduced from the other with the help
of the GetActuallyAllocatedSize function. As it turns out, this header can
be used for both 64 and 32 bit, and as such we dropped the requirement for
the 128-bit compare and exchange instruction support (cmpxchg16b).
- Add 32-bit support for the checksum and the PRNG functions: if the SSE 4.2
instruction set is supported, use the 32-bit CRC32 instruction, and in the
XorShift128, use a 32-bit based state instead of 64-bit.
- Add software support for CRC32: if SSE 4.2 is not supported, fallback on a
software implementation.
- Modify tests that were not 32-bit compliant, and expand them to cover more
allocation and alignment sizes. The random shuffle test has been deactivated
for linux-i386 & linux-i686 as the 32-bit sanitizer allocator doesn't
currently randomize chunks.
Reviewers: alekseyshl, kcc
Subscribers: filcab, llvm-commits, tberghammer, danalbert, srhines, mgorny, modocache
Differential Revision: https://reviews.llvm.org/D26358
llvm-svn: 288255
This patch builds on LLVM r279776.
In this patch I've done some cleanup and abstracted three common steps runtime components have in their CMakeLists files, and added a fourth.
The three steps I abstract are:
(1) Add a top-level target (i.e asan, msan, ...)
(2) Set the target properties for sorting files in IDE generators
(3) Make the compiler-rt target depend on the top-level target
The new step is to check if a command named "runtime_register_component" is defined, and to call it with the component name.
The runtime_register_component command is defined in llvm/runtimes/CMakeLists.txt, and presently just adds the component to a list of sub-components, which later gets used to generate target mappings.
With this patch a new workflow for runtimes builds is supported. The new workflow when building runtimes from the LLVM runtimes directory is:
> cmake [...]
> ninja runtimes-configure
> ninja asan
The "runtimes-configure" target builds all the dependencies for configuring the runtimes projects, and runs CMake on the runtimes projects. Running the runtimes CMake generates a list of targets to bind into the top-level CMake so subsequent build invocations will have access to some of Compiler-RT's targets through the top-level build.
Note: This patch does exclude some top-level targets from compiler-rt libraries because they either don't install files (sanitizer_common), or don't have a cooresponding `check` target (stats).
llvm-svn: 279863
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.
Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.
This patch doesn't contain behavior changes.
Reviewers: kubabrecka, rnk
Subscribers: wang0109, llvm-commits, kubabrecka, chrisha
Differential Revision: http://reviews.llvm.org/D21952
llvm-svn: 275111
Summary:
This is an initial implementation of a Hardened Allocator based on Sanitizer Common's CombinedAllocator.
It aims at mitigating heap based vulnerabilities by adding several features to the base allocator, while staying relatively fast.
The following were implemented:
- additional consistency checks on the allocation function parameters and on the heap chunks;
- use of checksum protected chunk header, to detect corruption;
- randomness to the allocator base;
- delayed freelist (quarantine), to mitigate use after free and overall determinism.
Additional mitigations are in the works.
Reviewers: eugenis, aizatsky, pcc, krasin, vitalybuka, glider, dvyukov, kcc
Subscribers: kubabrecka, filcab, llvm-commits
Differential Revision: http://reviews.llvm.org/D20084
llvm-svn: 271968
Dan Liew