Summary:
As reported on llvm-testers, during 8.0.0-rc1 testing I got errors while
building of `XRayTest`, during `check-all`:
```
[100%] Generating XRayTest-x86_64-Test
/home/dim/llvm/8.0.0/rc1/Phase3/Release/llvmCore-8.0.0-rc1.obj/./lib/libLLVMSupport.a(Signals.cpp.o): In function `llvm::sys::PrintStackTrace(llvm::raw_ostream&)':
Signals.cpp:(.text._ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x24): undefined reference to `backtrace'
Signals.cpp:(.text._ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x254): undefined reference to `llvm::itaniumDemangle(char const*, char*, unsigned long*, int*)'
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[3]: *** [projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/build.make:73: projects/compiler-rt/lib/xray/tests/unit/XRayTest-x86_64-Test] Error 1
gmake[3]: Target 'projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/build' not remade because of errors.
gmake[2]: *** [CMakeFiles/Makefile2:33513: projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/all] Error 2
gmake[2]: Target 'CMakeFiles/check-all.dir/all' not remade because of errors.
gmake[1]: *** [CMakeFiles/Makefile2:737: CMakeFiles/check-all.dir/rule] Error 2
gmake[1]: Target 'check-all' not remade because of errors.
gmake: *** [Makefile:277: check-all] Error 2
[Release Phase3] check-all failed
```
This is because the `backtrace` function requires `-lexecinfo` on BSD
platforms. To fix this, detect the `execinfo` library in
`cmake/config-ix.cmake`, and add it to the unit test link flags.
Additionally, since the code in `sys::PrintStackTrace` makes use of
`itaniumDemangle`, also add `-lLLVMDemangle`. (Note that this is more
of a general problem with libLLVMSupport, but I'm looking for a quick
fix now so it can be merged to the 8.0 branch.)
Reviewers: dberris, hans, mgorny, samsonov
Reviewed By: dberris
Subscribers: krytarowski, delcypher, erik.pilkington, #sanitizers, emaste, llvm-commits
Differential Revision: https://reviews.llvm.org/D57181
llvm-svn: 352234
Summary:
D57116 fails on the armv7 bots, which is I assume due to the timing of
the RSS check on the platform. While I don't have a platform to test
that change on, I assume this would do.
The test could be made more reliable by either delaying more the
allocations, or allocating more large-chunks, but both those options
have a somewhat non negligible impact (more memory used, longer test).
Hence me trying to keep the additional sleeping/allocating to a
minimum.
Reviewers: eugenis, yroux
Reviewed By: yroux
Subscribers: javed.absar, kristof.beyls, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D57241
llvm-svn: 352220
Summary:
Release memory pages for thread data (allocator cache, stack allocations
ring buffer, etc) when a thread exits. We can not simply munmap them
because this memory is custom allocated within a limited address range,
and it needs to stay "reserved".
This change alters thread storage layout by putting the ring buffer
before Thread instead of after it. This makes it possible to find the
start of the thread aux allocation given only the Thread pointer.
Reviewers: kcc, pcc
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56621
llvm-svn: 352151
Summary:
This tunes several of the default parameters used within the allocator:
- disable the deallocation type mismatch on Android by default; this
was causing too many issues with third party libraries;
- change the default `SizeClassMap` to `Dense`, it caches less entries
and is way more memory efficient overall;
- relax the timing of the RSS checks, 10 times per second was too much,
lower it to 4 times (every 250ms), and update the test so that it
passes with the new default.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: srhines, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D57116
llvm-svn: 352057
Summary:
UBSan wants to detect when unreachable code is actually reached, so it
adds instrumentation before every `unreachable` instruction. However,
the optimizer will remove code after calls to functions marked with
`noreturn`. To avoid this UBSan removes `noreturn` from both the call
instruction as well as from the function itself. Unfortunately, ASan
relies on this annotation to unpoison the stack by inserting calls to
`_asan_handle_no_return` before `noreturn` functions. This is important
for functions that do not return but access the the stack memory, e.g.,
unwinder functions *like* `longjmp` (`longjmp` itself is actually
"double-proofed" via its interceptor). The result is that when ASan and
UBSan are combined, the `noreturn` attributes are missing and ASan
cannot unpoison the stack, so it has false positives when stack
unwinding is used.
Changes:
# UBSan now adds the `expect_noreturn` attribute whenever it removes
the `noreturn` attribute from a function
# ASan additionally checks for the presence of this attribute
Generated code:
```
call void @__asan_handle_no_return // Additionally inserted to avoid false positives
call void @longjmp
call void @__asan_handle_no_return
call void @__ubsan_handle_builtin_unreachable
unreachable
```
The second call to `__asan_handle_no_return` is redundant. This will be
cleaned up in a follow-up patch.
rdar://problem/40723397
Reviewers: delcypher, eugenis
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56624
llvm-svn: 352003
This saves a cbz+cold call in the interceptor ABI, as well as a realign
in both ABIs, trading off a dcache entry against some branch predictor
entries and some code size.
Unfortunately the functionality is hidden behind a flag because ifunc is
known to be broken on static binaries on Android.
Differential Revision: https://reviews.llvm.org/D57084
llvm-svn: 351989
Each hwasan check requires emitting a small piece of code like this:
https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#memory-accesses
The problem with this is that these code blocks typically bloat code
size significantly.
An obvious solution is to outline these blocks of code. In fact, this
has already been implemented under the -hwasan-instrument-with-calls
flag. However, as currently implemented this has a number of problems:
- The functions use the same calling convention as regular C functions.
This means that the backend must spill all temporary registers as
required by the platform's C calling convention, even though the
check only needs two registers on the hot path.
- The functions take the address to be checked in a fixed register,
which increases register pressure.
Both of these factors can diminish the code size effect and increase
the performance hit of -hwasan-instrument-with-calls.
The solution that this patch implements is to involve the aarch64
backend in outlining the checks. An intrinsic and pseudo-instruction
are created to represent a hwasan check. The pseudo-instruction
is register allocated like any other instruction, and we allow the
register allocator to select almost any register for the address to
check. A particular combination of (register selection, type of check)
triggers the creation in the backend of a function to handle the check
for specifically that pair. The resulting functions are deduplicated by
the linker. The pseudo-instruction (really the function) is specified
to preserve all registers except for the registers that the AAPCS
specifies may be clobbered by a call.
To measure the code size and performance effect of this change, I
took a number of measurements using Chromium for Android on aarch64,
comparing a browser with inlined checks (the baseline) against a
browser with outlined checks.
Code size: Size of .text decreases from 243897420 to 171619972 bytes,
or a 30% decrease.
Performance: Using Chromium's blink_perf.layout microbenchmarks I
measured a median performance regression of 6.24%.
The fact that a perf/size tradeoff is evident here suggests that
we might want to make the new behaviour conditional on -Os/-Oz.
But for now I've enabled it unconditionally, my reasoning being that
hwasan users typically expect a relatively large perf hit, and ~6%
isn't really adding much. We may want to revisit this decision in
the future, though.
I also tried experimenting with varying the number of registers
selectable by the hwasan check pseudo-instruction (which would result
in fewer variants being created), on the hypothesis that creating
fewer variants of the function would expose another perf/size tradeoff
by reducing icache pressure from the check functions at the cost of
register pressure. Although I did observe a code size increase with
fewer registers, I did not observe a strong correlation between the
number of registers and the performance of the resulting browser on the
microbenchmarks, so I conclude that we might as well use ~all registers
to get the maximum code size improvement. My results are below:
Regs | .text size | Perf hit
-----+------------+---------
~all | 171619972 | 6.24%
16 | 171765192 | 7.03%
8 | 172917788 | 5.82%
4 | 177054016 | 6.89%
Differential Revision: https://reviews.llvm.org/D56954
llvm-svn: 351920
Bionic libc relies on an old libgcc behaviour which does not set hidden
visibility attribute. Keep exporting these symbols on Android for
compatibility.
Differential Revision: https://reviews.llvm.org/D56977
llvm-svn: 351915
Summary:
Enable building libFuzzer with MSVC.
* Don't try to include <endian.h> in FuzzerSHA1.cpp. MSVC
doesn't have this header, and WINDOWS is always little
endian (even on ARM)
Subscribers: srhines, mgorny, javed.absar, kristof.beyls
Differential Revision: https://reviews.llvm.org/D56510
llvm-svn: 351855
all missed!
Thanks to Alex Bradbury for pointing this out, and the fact that I never
added the intended `legacy` anchor to the developer policy. Add that
anchor too. With hope, this will cause the links to all resolve
successfully.
llvm-svn: 351731
Reports correct size and tags when either size is not power of two
or offset to bad granule is not zero.
Differential revision: https://reviews.llvm.org/D56603
llvm-svn: 351730
Summary:
Make Sanitizer Coverage work when compiled work when compiler-rt
is compiled with MSVC.
The previous solution did not work for MSVC because MSVC tried to
align the .SCOV$CZ section even though we used
__declspec(align(1)) on its only symbol:
__stop___sancov_cntrs.
Because the counter array is composed
of 1 byte elements, it does not always end on an 8 or 4 byte
boundary. This means that padding was sometimes added to
added to align the next section, .SCOV$CZ.
Use a different strategy now: instead of only instructing
the compiler not to align the symbol, make the section
one byte long by making its only symbol a uint8_t, so that
the linker won't try to align it.
Reviewers: morehouse, rnk
Reviewed By: rnk
Subscribers: kubamracek
Differential Revision: https://reviews.llvm.org/D56866
llvm-svn: 351714
This implements `mi_extra_init(...)` for the ASan allocator on
Darwin and uses the `__lsan::GetAllocatorGlobalRange(...)` function
to retrieve the allocator pointer and size.
rdar://problem/45284065
llvm-svn: 351713
`sanitizer_malloc_introspection_t` and initialize them to zero.
We allow sanitizer implementations to perform different initialization
by defining `COMMON_MALLOC_HAS_EXTRA_INTROSPECTION_INIT` to be `1`
and providing an implementation of `mi_extra_init(...)`.
We use these changes in future patches to implement malloc zone enumeration.
rdar://problem/45284065
llvm-svn: 351712
enumerator.
This is done by defining `COMMON_MALLOC_HAS_ZONE_ENUMERATOR` to `1` and
then by providing an implementation of the `mi_enumerator(...)` function.
If a custom implementation isn't desired the macro is set to `0` which
causes a stub version (that fails) to be used.
Currently all Darwin sanitizers that have malloc implementations define
this to be `0` so there is no functionality change.
rdar://problem/45284065
llvm-svn: 351711
We forgot to pass `AddressSpaceView` to the `CombinedAllocator`
which meant we would always use `LocalAddressSpaceView` for the
`CombinedAllocator` leading to a static_assert failing when we
tried to do `AsanAllocatorASVT<RemoteAddressSpaceView>` or
`AllocatorASVT<RemoteAddressSpaceView>`.
rdar://problem/45284065
llvm-svn: 351689
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351648
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
This installs the new developer policy and moves all of the license
files across all LLVM projects in the monorepo to the new license
structure. The remaining projects will be moved independently.
Note that I've left odd formatting and other idiosyncracies of the
legacy license structure text alone to make the diff easier to read.
Critically, note that we do not in any case *remove* the old license
notice or terms, as that remains necessary until we finish the
relicensing process.
I've updated a few license files that refer to the LLVM license to
instead simply refer generically to whatever license the LLVM project is
under, basically trying to minimize confusion.
This is really the culmination of so many people. Chris led the
community discussions, drafted the policy update and organized the
multi-year string of meeting between lawyers across the community to
figure out the strategy. Numerous lawyers at companies in the community
spent their time figuring out initial answers, and then the Foundation's
lawyer Heather Meeker has done *so* much to help refine and get us ready
here. I could keep going on, but I just want to make sure everyone
realizes what a huge community effort this has been from the begining.
Differential Revision: https://reviews.llvm.org/D56897
llvm-svn: 351631
Summary:
Whenever a large shadow region is tagged to zero, madvise(DONT_NEED)
as much of it as possible.
This reduces shadow RSS on Android by 45% or so, and total memory use
by 2-4%, probably even more on long running multithreaded programs.
CPU time seems to be in the noise.
Reviewers: kcc, pcc
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56757
llvm-svn: 351620
Summary:
SafeStack needs just few functions from there, but sanitizer_common
introduces conflicts with other runtimes, e.g. SCUDO.
Reviewers: eugenis, kcc, cryptoad
Subscribers: mgorny, krytarowski, fedor.sergeev, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D56886
llvm-svn: 351506
Summary:
This replaces the sanitizer tool list (used for generating
sanitizer_common configurations) with a tool list derived from
existing build system information.
Previously sanitizer_common had its own list of supported sanitizer
tools. This was bad because it was out of sync with the rest of the
build system. Notably it meant that the sanitizer_common runtime was
only being tested on Darwin the ASan dylib and not the other sanitizer
dylibs that are built for Darwin (LSan, TSan, and UBSan).
Unfortunately enabling the tests against other sanitizer dylibs has lead
to some test failures on Darwin. For now they've been marked as
XFAIL until the failures can investigated properly.
For Windows and Android we use the old sanitizer tool list to try avoid
bot breakages.
rdar://problem/47143078
Reviewers: kubamracek, george.karpenkov, yln, samsonov, vitalybuka, krytarowski
Subscribers: srhines, mgorny, fedor.sergeev, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55740
llvm-svn: 351398
Summary:
Small refactoring: replace some if-else cascades with switches so that the compiler warns us about missing cases.
Maybe found a small bug?
Reviewers: dcoughlin, kubamracek, dvyukov, delcypher, jfb
Reviewed By: dvyukov
Subscribers: llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56295
llvm-svn: 351288
Looks like the sanitizer-x86_64-linux-android bot started failing
because -pie is still needed when targeting API levels < 16 (which
is the case by default for arm and i686).
llvm-svn: 351270
Summary:
Remove code for handling unstable edges from libFuzzer since
it has not been found useful.
Differential Revision: https://reviews.llvm.org/D56730
llvm-svn: 351262
Kostya Serebryany