Summary:
The motivating use case is eliminating duplicate profile data registered
for the same inline function in two object files. Before this change,
users would observe multiple symbol definition errors with VC link, but
links with LLD would succeed.
Users (Mozilla) have reported that PGO works well with clang-cl and LLD,
but when using LLD without this static registration, we would get into a
"relocation against a discarded section" situation. I'm not sure what
happens in that situation, but I suspect that duplicate, unused profile
information was retained. If so, this change will reduce the size of
such binaries with LLD.
Now, Windows uses static registration and is in line with all the other
platforms.
Reviewers: davidxl, wmi, inglorion, void, calixte
Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D57929
llvm-svn: 353547
Current interception code does not cover all of the required registers
on Windows for a specific flavor of MOV, so this patch adds cases to
identify the following 5-byte instructions on 64-bit Windows:
mov QWORD PTR [rsp + XX], rdx <- second integer argument
mov QWORD PTR [rsp + XX], r9 <- third integer argument
mov QWORD PTR [rsp + XX], r8 <- fourth integer argument
The instruction for MOV [...] RCX is already covered in the previous
version.
Patch by Matthew McGovern!
Reviewers: rnk
Differential Revision: https://reviews.llvm.org/D57339
llvm-svn: 353483
Summary:
3rd party sysconf interceptor may crash if it's called before unsafe_stack_setup
However pageSize is not useful here. mmap should round up on it's own, SFS_CHECK can be removed.
Reviewers: eugenis, vlad.tsyrklevich
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D57924
llvm-svn: 353481
Summary:
From runtime side looks it's OK to RoundUpTo to needed alignment as buffer is
going to be RoundUpTo to page size anyway.
Reviewers: eugenis, pcc
Subscribers: #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D57866
llvm-svn: 353475
Summary:
The standalone Scudo version is being built with `-Werror` which can be
tripped by extraneous command line arguments. We have little control over
those as they can be passed down to us by `CMAKE_C(XX)_FLAGS`, the reported
scenario involving `-stdlib=libc++` (see https://reviews.llvm.org/D57412#1384504).
To work around this, disable `-Wunused-command-line-argument`.
Reviewers: eugenis, vitalybuka, Eugene.Zelenko
Reviewed By: eugenis
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D57757
llvm-svn: 353418
func_entry_exit.cc is for __tsan_func_entry/exit (spends ~75% there),
we don't yet have any.
mop.cc is for memory access functions, as compared to mini_bench_local/shared.cc
this benchmark passes through deduplication logic (ContainsSameAccess).
llvm-svn: 353407
- Manually unwind code in MemoryAccessImpl1() because clang do not optimize it
- Check for .rodata section only in read operations
- Place LIKELY/UNLIKELY on fast paths
This speeds up synthetic memory access benchmarks by 10-20%.
[dvyukov: fixed up consts in check_analyze.sh]
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57882
Context: https://reviews.llvm.org/D54889
llvm-svn: 353401
Replace bool workerthread flag with ThreadType enum.
This change is preparation for fiber support.
[dvyukov: fixed build of sanitizer_thread_registry_test.cc]
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57839
Context: https://reviews.llvm.org/D54889
llvm-svn: 353390
Summary:
Buffer should be referenced by results so used parts will be unpoisoned with unpoison_group and unpoison_passwd.
This fixes TSAN performance issue made us to disable this interceptors.
Reviewers: eugenis, dvyukov
Subscribers: srhines, kubamracek, krytarowski, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D57731
llvm-svn: 353351
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.
Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.
Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.
The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.
Reviewers: pcc, vitalybuka
Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D57720
llvm-svn: 353255
There is no reason for these sections to remain separate in the final
DLL or EXE.
I have not yet added a InstrProfilingPlatformWindows.c for these, since
avoiding dynamic profile data registration is a larger project for
later.
llvm-svn: 353221
Summary:
Use `_write` instead of the deprecated alias `write` on Windows.
Also, remove comment saying RawPrint is untested on Windows.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Subscribers: vitalybuka
Differential Revision: https://reviews.llvm.org/D57589
llvm-svn: 353108
Summary:
This is the initial check-in for the Standalone version of Scudo.
The project is initially going to live in scudo/standalone then will
replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html
for details.
This initial CL is meant to lay out the project structure, of both
code & tests, providing a minimal amount of functionalities, namely
various definitions, some atomic helpers and an intrusive list.
(empty.cc is just here to have a compilation unit, but will go away
in the upcoming CLs).
Initial support is restricted to Linux i386 & x86_64 in make files
and will be extended once things land & work.
We will grow organically from here, adding functionalities in limited
amounts.
Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack
Reviewed By: morehouse, vitalybuka
Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D57412
llvm-svn: 353055
The test seems to be failing because the module suppression file
contains a colon. I found that it was sufficient to just use the
basename of the suppression file.
While I was here, I noticed that we don't implement IsAbsolutePath for
Windows, so I added it.
llvm-svn: 352921
This function initializes enough of the runtime to be able to run
instrumented code in a statically linked executable. It replaces
__hwasan_shadow_init() which wasn't doing enough initialization for
instrumented code that uses either TLS or IFUNC to work.
Differential Revision: https://reviews.llvm.org/D57490
llvm-svn: 352816
Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is being
initialized (since OutputFile is not yet initialized).
Reviewers: kcc
Reviewed By: kcc
Differential Revision: https://reviews.llvm.org/D57531
llvm-svn: 352789
Summary:
Set default `ASAN_OPTIONS` when running libFuzzer tests. This allows us
to remove special casing in code for Darwin where we usually pass
`abort_on_error=0` to override platform defaults for tests.
A previous commit changed the code to make the tests pass:
7764a04af0
Adapted a few tests to use `%env_asan_opts=` instead of directly setting
the environment variable.
rdar://problem/47515276
Reviewers: kcc, george.karpenkov
Differential Revision: https://reviews.llvm.org/D57465
llvm-svn: 352711
This fixes most references to the paths:
llvm.org/svn/
llvm.org/git/
llvm.org/viewvc/
github.com/llvm-mirror/
github.com/llvm-project/
reviews.llvm.org/diffusion/
to instead point to https://github.com/llvm/llvm-project.
This is *not* a trivial substitution, because additionally, all the
checkout instructions had to be migrated to instruct users on how to
use the monorepo layout, setting LLVM_ENABLE_PROJECTS instead of
checking out various projects into various subdirectories.
I've attempted to not change any scripts here, only documentation. The
scripts will have to be addressed separately.
Additionally, I've deleted one document which appeared to be outdated
and unneeded:
lldb/docs/building-with-debug-llvm.txt
Differential Revision: https://reviews.llvm.org/D57330
llvm-svn: 352514
Summary:
Use dllexport for all declarations in FuzzerInterface.h Use it for clang
even though clang supports default visibility attribute to prevent a
warning from being thrown when LLVMFuzzerMutate is defined with dllexport.
This makes `FUZZER_INTERFACE_VISIBILITY` (FuzzerInterface.h) consistent with
`ATTRIBUTE_INTERFACE` (FuzzerDefs.h) when using clang on Windows.
Reviewers: vitalybuka, morehouse
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D57305
llvm-svn: 352395
Summary:
Remove this feature as it is unused, buggy, and not worth correcting
since the forkserver makes it difficult.
Reviewers: morehouse, jfb
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D57308
llvm-svn: 352392
Summary:
This makes `GetBlockBegin()` and `GetBlockBeginFastLocked()` work correctly with `RemoteAddressSpaceView`.
This has a knock on effect of also making the `PointerIsMine()` and
`GetMetaData()` methods behave correctly when `RemoteAddressSpaceView`
is used to instantiate the allocators.
This will be used by future out-of-process allocator enumeration
patches.
rdar://problem/45284065
Reviewers: kcc, vitalybuka, dvyukov, cryptoad, eugenis, george.karpenkov, yln
Subscribers: #sanitizers, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D56964
llvm-svn: 352335
Summary:
As reported on llvm-testers, during 8.0.0-rc1 testing I got errors while
building of `XRayTest`, during `check-all`:
```
[100%] Generating XRayTest-x86_64-Test
/home/dim/llvm/8.0.0/rc1/Phase3/Release/llvmCore-8.0.0-rc1.obj/./lib/libLLVMSupport.a(Signals.cpp.o): In function `llvm::sys::PrintStackTrace(llvm::raw_ostream&)':
Signals.cpp:(.text._ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x24): undefined reference to `backtrace'
Signals.cpp:(.text._ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x254): undefined reference to `llvm::itaniumDemangle(char const*, char*, unsigned long*, int*)'
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[3]: *** [projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/build.make:73: projects/compiler-rt/lib/xray/tests/unit/XRayTest-x86_64-Test] Error 1
gmake[3]: Target 'projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/build' not remade because of errors.
gmake[2]: *** [CMakeFiles/Makefile2:33513: projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/all] Error 2
gmake[2]: Target 'CMakeFiles/check-all.dir/all' not remade because of errors.
gmake[1]: *** [CMakeFiles/Makefile2:737: CMakeFiles/check-all.dir/rule] Error 2
gmake[1]: Target 'check-all' not remade because of errors.
gmake: *** [Makefile:277: check-all] Error 2
[Release Phase3] check-all failed
```
This is because the `backtrace` function requires `-lexecinfo` on BSD
platforms. To fix this, detect the `execinfo` library in
`cmake/config-ix.cmake`, and add it to the unit test link flags.
Additionally, since the code in `sys::PrintStackTrace` makes use of
`itaniumDemangle`, also add `-lLLVMDemangle`. (Note that this is more
of a general problem with libLLVMSupport, but I'm looking for a quick
fix now so it can be merged to the 8.0 branch.)
Reviewers: dberris, hans, mgorny, samsonov
Reviewed By: dberris
Subscribers: krytarowski, delcypher, erik.pilkington, #sanitizers, emaste, llvm-commits
Differential Revision: https://reviews.llvm.org/D57181
llvm-svn: 352234
Summary:
Release memory pages for thread data (allocator cache, stack allocations
ring buffer, etc) when a thread exits. We can not simply munmap them
because this memory is custom allocated within a limited address range,
and it needs to stay "reserved".
This change alters thread storage layout by putting the ring buffer
before Thread instead of after it. This makes it possible to find the
start of the thread aux allocation given only the Thread pointer.
Reviewers: kcc, pcc
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56621
llvm-svn: 352151
Summary:
This tunes several of the default parameters used within the allocator:
- disable the deallocation type mismatch on Android by default; this
was causing too many issues with third party libraries;
- change the default `SizeClassMap` to `Dense`, it caches less entries
and is way more memory efficient overall;
- relax the timing of the RSS checks, 10 times per second was too much,
lower it to 4 times (every 250ms), and update the test so that it
passes with the new default.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: srhines, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D57116
llvm-svn: 352057
Each hwasan check requires emitting a small piece of code like this:
https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#memory-accesses
The problem with this is that these code blocks typically bloat code
size significantly.
An obvious solution is to outline these blocks of code. In fact, this
has already been implemented under the -hwasan-instrument-with-calls
flag. However, as currently implemented this has a number of problems:
- The functions use the same calling convention as regular C functions.
This means that the backend must spill all temporary registers as
required by the platform's C calling convention, even though the
check only needs two registers on the hot path.
- The functions take the address to be checked in a fixed register,
which increases register pressure.
Both of these factors can diminish the code size effect and increase
the performance hit of -hwasan-instrument-with-calls.
The solution that this patch implements is to involve the aarch64
backend in outlining the checks. An intrinsic and pseudo-instruction
are created to represent a hwasan check. The pseudo-instruction
is register allocated like any other instruction, and we allow the
register allocator to select almost any register for the address to
check. A particular combination of (register selection, type of check)
triggers the creation in the backend of a function to handle the check
for specifically that pair. The resulting functions are deduplicated by
the linker. The pseudo-instruction (really the function) is specified
to preserve all registers except for the registers that the AAPCS
specifies may be clobbered by a call.
To measure the code size and performance effect of this change, I
took a number of measurements using Chromium for Android on aarch64,
comparing a browser with inlined checks (the baseline) against a
browser with outlined checks.
Code size: Size of .text decreases from 243897420 to 171619972 bytes,
or a 30% decrease.
Performance: Using Chromium's blink_perf.layout microbenchmarks I
measured a median performance regression of 6.24%.
The fact that a perf/size tradeoff is evident here suggests that
we might want to make the new behaviour conditional on -Os/-Oz.
But for now I've enabled it unconditionally, my reasoning being that
hwasan users typically expect a relatively large perf hit, and ~6%
isn't really adding much. We may want to revisit this decision in
the future, though.
I also tried experimenting with varying the number of registers
selectable by the hwasan check pseudo-instruction (which would result
in fewer variants being created), on the hypothesis that creating
fewer variants of the function would expose another perf/size tradeoff
by reducing icache pressure from the check functions at the cost of
register pressure. Although I did observe a code size increase with
fewer registers, I did not observe a strong correlation between the
number of registers and the performance of the resulting browser on the
microbenchmarks, so I conclude that we might as well use ~all registers
to get the maximum code size improvement. My results are below:
Regs | .text size | Perf hit
-----+------------+---------
~all | 171619972 | 6.24%
16 | 171765192 | 7.03%
8 | 172917788 | 5.82%
4 | 177054016 | 6.89%
Differential Revision: https://reviews.llvm.org/D56954
llvm-svn: 351920
Bionic libc relies on an old libgcc behaviour which does not set hidden
visibility attribute. Keep exporting these symbols on Android for
compatibility.
Differential Revision: https://reviews.llvm.org/D56977
llvm-svn: 351915
Summary:
Enable building libFuzzer with MSVC.
* Don't try to include <endian.h> in FuzzerSHA1.cpp. MSVC
doesn't have this header, and WINDOWS is always little
endian (even on ARM)
Subscribers: srhines, mgorny, javed.absar, kristof.beyls
Differential Revision: https://reviews.llvm.org/D56510
llvm-svn: 351855
Reports correct size and tags when either size is not power of two
or offset to bad granule is not zero.
Differential revision: https://reviews.llvm.org/D56603
llvm-svn: 351730
Summary:
Make Sanitizer Coverage work when compiled work when compiler-rt
is compiled with MSVC.
The previous solution did not work for MSVC because MSVC tried to
align the .SCOV$CZ section even though we used
__declspec(align(1)) on its only symbol:
__stop___sancov_cntrs.
Because the counter array is composed
of 1 byte elements, it does not always end on an 8 or 4 byte
boundary. This means that padding was sometimes added to
added to align the next section, .SCOV$CZ.
Use a different strategy now: instead of only instructing
the compiler not to align the symbol, make the section
one byte long by making its only symbol a uint8_t, so that
the linker won't try to align it.
Reviewers: morehouse, rnk
Reviewed By: rnk
Subscribers: kubamracek
Differential Revision: https://reviews.llvm.org/D56866
llvm-svn: 351714
This implements `mi_extra_init(...)` for the ASan allocator on
Darwin and uses the `__lsan::GetAllocatorGlobalRange(...)` function
to retrieve the allocator pointer and size.
rdar://problem/45284065
llvm-svn: 351713
`sanitizer_malloc_introspection_t` and initialize them to zero.
We allow sanitizer implementations to perform different initialization
by defining `COMMON_MALLOC_HAS_EXTRA_INTROSPECTION_INIT` to be `1`
and providing an implementation of `mi_extra_init(...)`.
We use these changes in future patches to implement malloc zone enumeration.
rdar://problem/45284065
llvm-svn: 351712
enumerator.
This is done by defining `COMMON_MALLOC_HAS_ZONE_ENUMERATOR` to `1` and
then by providing an implementation of the `mi_enumerator(...)` function.
If a custom implementation isn't desired the macro is set to `0` which
causes a stub version (that fails) to be used.
Currently all Darwin sanitizers that have malloc implementations define
this to be `0` so there is no functionality change.
rdar://problem/45284065
llvm-svn: 351711
We forgot to pass `AddressSpaceView` to the `CombinedAllocator`
which meant we would always use `LocalAddressSpaceView` for the
`CombinedAllocator` leading to a static_assert failing when we
tried to do `AsanAllocatorASVT<RemoteAddressSpaceView>` or
`AllocatorASVT<RemoteAddressSpaceView>`.
rdar://problem/45284065
llvm-svn: 351689
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351648
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
Whenever a large shadow region is tagged to zero, madvise(DONT_NEED)
as much of it as possible.
This reduces shadow RSS on Android by 45% or so, and total memory use
by 2-4%, probably even more on long running multithreaded programs.
CPU time seems to be in the noise.
Reviewers: kcc, pcc
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56757
llvm-svn: 351620
Summary:
SafeStack needs just few functions from there, but sanitizer_common
introduces conflicts with other runtimes, e.g. SCUDO.
Reviewers: eugenis, kcc, cryptoad
Subscribers: mgorny, krytarowski, fedor.sergeev, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D56886
llvm-svn: 351506
Summary:
Small refactoring: replace some if-else cascades with switches so that the compiler warns us about missing cases.
Maybe found a small bug?
Reviewers: dcoughlin, kubamracek, dvyukov, delcypher, jfb
Reviewed By: dvyukov
Subscribers: llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56295
llvm-svn: 351288
Summary:
Remove code for handling unstable edges from libFuzzer since
it has not been found useful.
Differential Revision: https://reviews.llvm.org/D56730
llvm-svn: 351262
Summary:
This is the compiler-rt part.
The clang part is D54589.
This is a second commit, the original one was r351106,
which was mass-reverted in r351159 because 2 compiler-rt tests were failing.
Now, i have fundamentally changed the testing approach:
i malloc a few bytes, intentionally mis-align the pointer
(increment it by one), and check that. Also, i have decreased
the expected alignment. This hopefully should be enough to pacify
all the bots. If not, i guess i might just drop the two 'bad' tests.
Reviewers: filcab, vsk, #sanitizers, vitalybuka, rsmith, morehouse
Reviewed By: morehouse
Subscribers: rjmccall, krytarowski, rsmith, kcc, srhines, kubamracek, dberris, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54590
llvm-svn: 351178
Summary:
Use alternatename for external functions only when using
MSVC since Clang doesn't support it and MSVC doesn't support
Clang's method (weak aliases).
Reviewers: morehouse
Reviewed By: morehouse
Subscribers: rnk, thakis, mgorny
Differential Revision: https://reviews.llvm.org/D56514
llvm-svn: 351152
Disable tests requiring sunrpc when the relevant headers are missing.
In order to accommodate that, move the header check
from sanitizer_common to base-config-ix, and define the check result
as a global variable there. Use it afterwards both for definition
needed by sanitizer_common, and to control 'sunrpc' test feature.
While at it, remove the append_have_file_definition macro that was used
only once, and no longer fits the split check-definition.
Bug report: https://github.com/google/sanitizers/issues/974
Differential Revision: https://reviews.llvm.org/D47819
llvm-svn: 351109
This reverts r350806 which marked some tests as UNSUPPORTED on ARM and
instead reintroduces the old code path only for Thumb, since that seems
to be the only target that broke.
It would still be nice to find the root cause of the breakage, but with
the branch point for LLVM 8.0 scheduled for next week it's better to put
things in a stable state while we investigate.
Differential Revision: https://reviews.llvm.org/D56594
llvm-svn: 351040
This makes the script a little more gn friendly; gn does not support
redirecting the output of a script.
Differential Revision: https://reviews.llvm.org/D56579
llvm-svn: 350980
Summary:
This fixes linker errors that occurs when the
`sanitizer_type_traits_test.cc` is built without optimizations.
The error occurs because the test tries to take a reference.
A possible workaround is to give the GTest macros take boolean rvalues
by doing something like:
```
ASSERT_TRUE(bool(is_same<uptr, uptr>::value));
```
However this only hides the problem. Unfortunately Using `constexpr`
won't fix the problem unless we are using C++17.
Reviewers: vitalybuka, kubamracek, george.karpenkov, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D56035
llvm-svn: 350940
- If entries are properly copied (there were a bug in FreeBSD implementation in earlier version), or list properly reset.
Reviewers: vitalybuka, krytarowski
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D56562
llvm-svn: 350919
Summary:
It has been superseded by the `ignore_noninstrumented_modules` flag and is no longer needed.
Also simplify a test that checks that `mmap_interceptor` respects ignore annotations (`thr->ignore_reads_and_writes `).
Relevant: https://reviews.llvm.org/rL269855
<rdar://problem/46263073> Remove obsolete Apple-specific suppression option
Reviewers: dcoughlin, kubamracek, dvyukov, delcypher
Reviewed By: dvyukov
Subscribers: jfb, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55075
llvm-svn: 350883
Remove the partial support for rpc/xdr.h from libtirpc. Since it is
an entirely external library, we ought to build it sanitized separately
and not attempt to intercept like the libc implementation. Besides,
the existing code for tirpc support was neither complete nor working.
Noted by @krytarowski.
Differential Revision: https://reviews.llvm.org/D47817
llvm-svn: 350881
This patch implements the long double __floattitf (int128_t) method for
PowerPC -- specifically to convert a 128 bit integer into a long double
(IBM double-double).
To invoke this method, one can do so by linking against compiler-rt, via the
--rtlib=compiler-rt command line option supplied to clang.
Differential Revision: https://reviews.llvm.org/D54313/
llvm-svn: 350818
This patch implements the __uint128_t __fixunstfti (long double) method for
PowerPC -- specifically to convert a long double (IBM double-double) to an
unsigned 128 bit integer.
The general approach of this algorithm is to convert the high and low doubles
of the long double and add them together if the doubles fit within 64 bits.
However, additional adjustments and scaling is performed when the high or low
double does not fit within a 64 bit integer.
To invoke this method, one can do so by linking against compiler-rt, via the
--rtlib=compiler-rt command line option supplied to clang.
Differential Revision: https://reviews.llvm.org/D54911
llvm-svn: 350815
Summary:
Replace calls to builtin functions with macros or functions that call the
Windows-equivalents when targeting windows and call the original
builtin functions everywhere else.
This change makes more parts of libFuzzer buildable with MSVC.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Subscribers: mgorny, rnk, thakis
Differential Revision: https://reviews.llvm.org/D56439
llvm-svn: 350766
Now that memory intrinsics are instrumented, it's more likely that
CheckAddressSized will be called with size 0. (It was possible before
with IR like:
%val = load [0 x i8], [0 x i8]* %ptr
but I don't think clang will generate IR like that and the optimizer
would normally remove it by the time it got anywhere near our pass
anyway). The right thing to do in both cases is to disable the
addressing checks (since the underlying memory intrinsic is a no-op),
so that's what we do.
Differential Revision: https://reviews.llvm.org/D56465
llvm-svn: 350683
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.
This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.
Reviewers: vitalybuka, pcc, krytarowski
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56141
llvm-svn: 350590
Summary:
Objective-C employs tagged pointers, that is, small objects/values may be encoded directly in the pointer bits. The resulting pointer is not backed by an allocation/does not point to a valid memory. TSan infrastructure requires a valid address for `Acquire/Release` and `Mutex{Lock/Unlock}`.
This patch establishes such a mapping via a "dummy allocation" for each encountered tagged pointer value.
Reviewers: dcoughlin, kubamracek, dvyukov, delcypher
Reviewed By: dvyukov
Subscribers: llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56238
llvm-svn: 350556
This is the deprecated legacy interface, replace it with the current
_zx_vmar_allocate one.
Differential Revision: https://reviews.llvm.org/D56360
llvm-svn: 350488
We still need the interceptor on non-aarch64 to untag the pthread_t
and pthread_attr_t pointers and disable tagging on allocations done
internally by glibc.
llvm-svn: 350445
Summary:
The default setting kTabSizeLog=20 results in an 8Mb global hash table,
almost all of it in private pages. That is not a sane setting in a
mobile, system-wide use case: with ~150 concurrent processes stack
depot will account for more than 1Gb of RAM.
Reviewers: kcc, pcc
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56333
llvm-svn: 350443
The problem is similar to D55986 but for threads: a process with the
interceptor hwasan library loaded might have some threads started by
instrumented libraries and some by uninstrumented libraries, and we
need to be able to run instrumented code on the latter.
The solution is to perform per-thread initialization lazily. If a
function needs to access shadow memory or add itself to the per-thread
ring buffer its prologue checks to see whether the value in the
sanitizer TLS slot is null, and if so it calls __hwasan_thread_enter
and reloads from the TLS slot. The runtime does the same thing if it
needs to access this data structure.
This change means that the code generator needs to know whether we
are targeting the interceptor runtime, since we don't want to pay
the cost of lazy initialization when targeting a platform with native
hwasan support. A flag -fsanitize-hwaddress-abi={interceptor,platform}
has been introduced for selecting the runtime ABI to target. The
default ABI is set to interceptor since it's assumed that it will
be more common that users will be compiling application code than
platform code.
Because we can no longer assume that the TLS slot is initialized,
the pthread_create interceptor is no longer necessary, so it has
been removed.
Ideally, lazy initialization should only cost one instruction in the
hot path, but at present the call may cause us to spill arguments
to the stack, which means more instructions in the hot path (or
theoretically in the cold path if the spills are moved with shrink
wrapping). With an appropriately chosen calling convention for
the per-thread initialization function (TODO) the hot path should
always need just one instruction and the cold path should need two
instructions with no spilling required.
Differential Revision: https://reviews.llvm.org/D56038
llvm-svn: 350429
The Android dynamic loader has a non-standard feature that allows
libraries such as the hwasan runtime to interpose symbols even after
the symbol already has a value. The new value of the symbol is used to
relocate libraries loaded after the interposing library, but existing
libraries keep the old value. This behaviour is activated by the
DF_1_GLOBAL flag in DT_FLAGS_1, which is set by passing -z global to
the linker, which is what we already do to link the hwasan runtime.
What this means in practice is that if we have .so files that depend
on interceptor-mode hwasan without the main executable depending on
it, some of the libraries in the process will be using the hwasan
allocator and some will be using the system allocator, and these
allocators need to interact somehow. For example, if an instrumented
library calls a function such as strdup that allocates memory on
behalf of the caller, the instrumented library can reasonably expect
to be able to call free to deallocate the memory.
We can handle that relatively easily with hwasan by using tag 0 to
represent allocations from the system allocator. If hwasan's realloc
or free functions are passed a pointer with tag 0, the system allocator
is called.
One limitation is that this scheme doesn't work in reverse: if an
instrumented library allocates memory, it must free the memory itself
and cannot pass ownership to a system library. In a future change,
we may want to expose an API for calling the system allocator so
that instrumented libraries can safely transfer ownership of memory
to system libraries.
Differential Revision: https://reviews.llvm.org/D55986
llvm-svn: 350427
Summary:
Replace the 32-bit allocator with a 64-bit one with a non-constant
base address, and reduce both the number of size classes and the maximum
size of per-thread caches.
As measured on [1], this reduces average weighted memory overhead
(MaxRSS) from 26% to 12% over stock android allocator. These numbers
include overhead from code instrumentation and hwasan shadow (i.e. not a
pure allocator benchmark).
This switch also enables release-to-OS functionality, which is not
implemented in the 32-bit allocator. I have not seen any effect from
that on the benchmark.
[1] https://android.googlesource.com/platform/system/extras/+/master/memory_replay/
Reviewers: vitalybuka, kcc
Subscribers: kubamracek, cryptoad, llvm-commits
Differential Revision: https://reviews.llvm.org/D56239
llvm-svn: 350370
Implement the interceptors for popen(), pclose() and popenve()
functions. The first two are POSIX, the third one is specific
to NetBSD. popen() spawns a process and creates a FILE object piping
data from/to that process. pclose() closes the pipe and waits for
the process to terminate appropriately.
For the purpose of popen(), the COMMON_INTERCEPTOR_FILE_OPEN macro is
modified to allow null path parameter.
Differential Revision: https://reviews.llvm.org/D56157
llvm-svn: 350232
Update the interceptor for devname_r() to account for correct return
types on different platforms. This function returns int on NetBSD
but char* on FreeBSD/OSX. Noticed by @krytarowski.
Differential Revision: https://reviews.llvm.org/D56150
llvm-svn: 350228
Summary:
Change the point of calling MaybeStartBackgroudThread() from AsanInitInternal()
that is too early on NetBSD to a constructor (with aid of C++11 lambda construct).
Enable the code for background thread as is for NetBSD.
Rename test/sanitizer_common/TestCases/Linux/hard_rss_limit_mb_test.cc
to test/sanitizer_common/TestCases/hard_rss_limit_mb_test.cc and allow runs
on NetBSD. This tests passes correctly.
Reviewers: vitalybuka, joerg, eugenis
Reviewed By: eugenis
Subscribers: eugenis, kubamracek, fedor.sergeev, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55887
llvm-svn: 350139
Summary:
This is a follow-up to r346956 (https://reviews.llvm.org/D53975).
The purpose of this change to allow implementers of the
`AddressSpaceView` to be able to distinguish between when a caller wants
read-only memory and when a caller wants writable memory. Being able
distinguish these cases allows implementations to optimize for the
different cases and also provides a way to workaround possible platform
restrictions (e.g. the low level platform interface for reading
out-of-process memory may place memory in read-only pages).
For allocator enumeration in almost all cases read-only is sufficient so
we make `Load(...)` take on this new requirement and introduce the
`LoadWritable(...)` variants for cases where memory needs to be
writable.
The behaviour of `LoadWritable(...)` documented in comments are
deliberately very restrictive so that it will be possible in the future
to implement a simple write-cache (i.e. just a map from target address
to a writable region of memory). These restrictions can be loosened in
the future if necessary by implementing a more sophisticated
write-cache.
rdar://problem/45284065
Reviewers: kcc, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54879
llvm-svn: 350136
- Disabled on purpose on Android and Darwin platform (for now).
- Darwin supports it, would need interception in its specific code before enabling it.
- Linux does not support it but only via third party library.
- Android supports it via bionic however it is known to have issue with older versions of the implementations. Can be enabled by an Android committer later on if necessary once there is more 'certainity'/been more tested.
Reviewers: krytarowski, vitalybuka
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D56125
llvm-svn: 350123
Revert r350104 "[asan] Fix build on windows."
Revert r350101 "[asan] Support running without /proc."
These changes break Mac build, too.
llvm-svn: 350112
Add SIOCSETHERCAP.
Add commented out NVMM (NetBSD Virtual Machine Monitor) operations
as this interface is still WIP and a subject to change.
llvm-svn: 350107
Summary: They happen to work out of the box.
Reviewers: rtrieu, vitalybuka
Subscribers: kubamracek, fedor.sergeev, krytarowski, llvm-commits
Differential Revision: https://reviews.llvm.org/D56088
llvm-svn: 350103
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the MSan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.
New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.
Reviewers: joerg, vitalybuka, jfb
Reviewed By: vitalybuka
Subscribers: emaste, jfb, llvm-commits, #sanitizers, mgorny
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55703
llvm-svn: 350102
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.
This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55874
llvm-svn: 350101
Summary: Assigning an array of bools to {false, false, false} can, in certain build configurations, lead to a memset call. Use internal_memset to avoid this.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D56107
llvm-svn: 350098
Add a CheckMPROTECT() routine to detect when pax MPROTECT is enabled
on NetBSD, and error xray out when it is. The solution is adapted
from existing CheckASLR().
Differential Revision: https://reviews.llvm.org/D56049
llvm-svn: 350030
Disable enforcing alignas() for structs that are used as thread_local
data on NetBSD. The NetBSD ld.so implementation is buggy and does
not enforce correct alignment; however, clang seems to take it for
granted and generates instructions that segv on wrongly aligned objects.
Therefore, disable those alignas() statements on NetBSD until we can
establish a better fix.
Apparently, std::aligned_storage<> does not have any real effect
at the moment, so we can leave it as-is.
Differential Revision: https://reviews.llvm.org/D56000
llvm-svn: 350029
Reviewers: krytarowski
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D56009
M lib/sanitizer_common/sanitizer_common_interceptors.inc
M lib/sanitizer_common/sanitizer_platform_interceptors.h
M lib/sanitizer_common/sanitizer_platform_limits_freebsd.cc
M lib/sanitizer_common/sanitizer_platform_limits_freebsd.h
D test/sanitizer_common/TestCases/NetBSD/regex.cc
A + test/sanitizer_common/TestCases/Posix/regex.cc
llvm-svn: 350002
Summary:
NetBSD uses typical UNIX interfaces.
All tests pass except instrprof-dlopen-dlclose-gcov.test, as there
is not supported semantics of atexit(3) in dlopen(3)ed+dlclose(3)d
DSO.
NetBSD also ships an older version of LLVM profile (ABI v.2 predating
ABI v.4 in upstream version) inside libc. That copy has been manually
removed during the porting and testing process of the upstream version
to NetBSD. Otherwise there were conflicts between them two.
Reviewers: joerg, vitalybuka, vsk
Subscribers: srhines, fedor.sergeev, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55997
llvm-svn: 349994
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.
This is a follow up to r349957.
rdar://problem/45284065
llvm-svn: 349960
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.
This is a follow up to r349138.
rdar://problem/45284065
llvm-svn: 349959
Summary:
This is a follow up to https://reviews.llvm.org/D55764 .
For the ASan and LSan allocatorsthe type declarations have been modified
so that it's possible to create a combined allocator type that
consistently uses a different type of `AddressSpaceView`. We intend to
use this in future patches. For the other sanitizers they just use
`LocalAddressSpaceView` by default because we have no plans to use these
allocators in an out-of-process manner.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov, yln
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55766
llvm-svn: 349957
Summary:
This is a follow up patch to r349138.
This patch makes a `AddressSpaceView` a type declaration in the
allocator parameters used by `SizeClassAllocator64`. For ASan, LSan, and
the unit tests the AP64 declarations have been made templated so that
`AddressSpaceView` can be changed at compile time. For the other
sanitizers we just hard-code `LocalAddressSpaceView` because we have no
plans to use these allocators in an out-of-process manner.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55764
llvm-svn: 349954
Add a code to properly test for presence of LLVMTestingSupport library
when performing a stand-alone build, and skip tests requiring it when
it is not present. Since the library is not installed, llvm-config
reported empty --libs for it and the tests failed to link with undefined
references. Skipping the two fdr_* test files is better than failing to
build, and should be good enough until we find a better solution.
NB: both installing LLVMTestingSupport and building it automatically
from within compiler-rt sources are non-trivial. The former due to
dependency on gtest, the latter due to tight integration with LLVM
source tree.
Differential Revision: https://reviews.llvm.org/D55891
llvm-svn: 349899
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.
(2nd attempt, this time without the sanitizer_rtems change)
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55801
llvm-svn: 349817
This is patch complements D55117 implementing __hwasan_mem*
functions in runtime
Differential revision: https://reviews.llvm.org/D55554
llvm-svn: 349730
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55801
llvm-svn: 349699
Summary:
An extra ';' at the end of a namespace triggers a pedantic warning:
```
.../sanitizer_common/sanitizer_type_traits.h:42:2: warning: extra ‘;’ [-Wpedantic]
}; // namespace __sanitizer
```
Reviewers: eugenis, delcypher
Reviewed By: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55849
llvm-svn: 349666
Fix the sha2 interceptor macros to use a constant for array parameter
length rather than referencing the extern variable. Since the digest
length is provided in hash name, reuse the macro parameter for it.
Verify that the calculated value matches the one provided by system
headers.
Differential Revision: https://reviews.llvm.org/D55811
llvm-svn: 349645
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the ASan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.
New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.
Reviewers: joerg, vitalybuka, jfb
Reviewed By: vitalybuka
Subscribers: dim, emaste, ro, jfb, devnexen, kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55596
llvm-svn: 349619
I tricked myself into thinking that deadlock detection is off by default in TSan by looking at the default value of the detect_deadlocks flag and outdated docs. (Created a pull request to update docs.)
I even managed to confuse others: https://groups.google.com/forum/#!topic/thread-sanitizer/xYvnAYwtoDk
However, the default value is overwritten in code (TSan_flags.cc:InitializeFlags). The TSan/deadlock tests also rely on this
This changes aligns the default value of the flag with the actual default behavior.
Author: yln (Julian Lettner)
Reviewed in: https://reviews.llvm.org/D55846
llvm-svn: 349609
As of r349413 it's now possible for a binary to contain an empty
hwasan frame section. Handle that case simply by doing nothing.
Differential Revision: https://reviews.llvm.org/D55796
llvm-svn: 349428
Summary:
This fixes the `ThreadSanitizer-x86_64-iossim` testsuite which broke
when r348770 (https://reviews.llvm.org/D55473) landed.
The root cause of the problem is that early-on during the iOS simulator
init process a call to `sysctlbyname` is issued. If the TSan initializer
is triggered at this point it will eventually trigger a call to
`__cxa_at_exit(...)`. This call then aborts because the library
implementing this function is not yet had its initialization function
called.
rdar://problem/46696934
Reviewers: kubamracek, george.karpenkov, devnexen, vitalybuka, krytarowski
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55711
llvm-svn: 349402
Summary:
This is a follow up patch to r346956 for the `SizeClassAllocator32`
allocator.
This patch makes `AddressSpaceView` a template parameter both to the
`ByteMap` implementations (but makes `LocalAddressSpaceView` the
default), some `AP32` implementations and is used in `SizeClassAllocator32`.
The actual changes to `ByteMap` implementations and
`SizeClassAllocator32` are very simple. However the patch is large
because it requires changing all the `AP32` definitions, and users of
those definitions.
For ASan and LSan we make `AP32` and `ByteMap` templateds type that take
a single `AddressSpaceView` argument. This has been done because we will
instantiate the allocator with a type that isn't `LocalAddressSpaceView`
in the future patches. For the allocators used in the other sanitizers
(i.e. HWAsan, MSan, Scudo, and TSan) use of `LocalAddressSpaceView` is
hard coded because we do not intend to instantiate the allocators with
any other type.
In the cases where untemplated types have become templated on a single
`AddressSpaceView` parameter (e.g. `PrimaryAllocator`) their name has
been changed to have a `ASVT` suffix (Address Space View Type) to
indicate they are templated. The only exception to this are the `AP32`
types due to the desire to keep the type name as short as possible.
In order to check that template is instantiated in the correct a way a
`static_assert(...)` has been added that checks that the
`AddressSpaceView` type used by `Params::ByteMap::AddressSpaceView` matches
the `Params::AddressSpaceView`. This uses the new `sanitizer_type_traits.h`
header.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D54904
llvm-svn: 349138
Summary:
A recent update to the VS toolchain in chromium [1] broke the windows
ASan bot because the new toolchain calls _msize_base() instead of
_msize() in a number of _aligned_* UCRT routines. Instrument
_msize_base() as well.
[1] https://crbug.com/914947
Reviewers: rnk, #sanitizers, vitalybuka
Reviewed By: rnk, #sanitizers, vitalybuka
Subscribers: vitalybuka, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55684
llvm-svn: 349115
Summary:
In particular we implement the `is_same<T,U>` templated type. This is
useful for doing compile-time comparison of types in `static_assert`s.
The plan is to use this in another patch (
https://reviews.llvm.org/D54904 ).
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54951
llvm-svn: 349077
Summary:
private and internal: should not trigger ODR at all.
unnamed_addr: current ODR checking approach fail and rereport false violation if
a linker merges such globals
linkonce_odr, weak_odr: could cause similar problems and they are already not
instrumented for ELF.
Reviewers: eugenis, kcc
Subscribers: kubamracek, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D55621
llvm-svn: 349015
Summary:
The TLS_SLOT_TSAN slot is available starting in N, but its location (8)
is incompatible with the proposed solution for implementing ELF TLS on
Android (i.e. bump ARM/AArch64 alignment to reserve an 8-word TCB).
Instead, starting in Q, Bionic replaced TLS_SLOT_DLERROR(6) with
TLS_SLOT_SANITIZER(6). Switch compiler-rt to the new slot.
Reviewers: eugenis, srhines, enh
Reviewed By: eugenis
Subscribers: ruiu, srhines, kubamracek, javed.absar, kristof.beyls, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D55581
llvm-svn: 348984
Summary:
Add a check that TLS_SLOT_TSAN / TLS_SLOT_SANITIZER, whichever
android_get_tls_slot is using, is not conflicting with
TLS_SLOT_DLERROR.
Reviewers: rprichard, vitalybuka
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55587
llvm-svn: 348979
Summary:
Fix bug where we'd try symbolize a second time with the same arguments even though symbolization failed the first time.
This looks like a long standing typo given that the guard for trying
symbolization again is to only try it if symbolization failed using
`binary` and `original_binary != binary`.
Reviewers: kubamracek, glider, samsonov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55504
llvm-svn: 348841
Summary:
Add new interceptors for a set of functions to reposition a stream:
fgetpos, fseek, fseeko, fsetpos, ftell, ftello, rewind .
Add a dedicated test.
Enable this interface on NetBSD.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55471
llvm-svn: 348743
- For the moment a subset of this api dealing with file descriptors permissions and ioctls.
Reviewers: vitalybuka, krytarowski
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D55368
llvm-svn: 348668
Summary:
strtoi/strtou converts string value to an intmax_t/uintmax_t integer.
Add a dedicated test.
Enable this API for NetBSD.
It's a reworked version of the original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, tomsun.0.7, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54702
llvm-svn: 348663
using `atos` symbolizer on Darwin when the binaries don't exist.
For now we just produce an unsymbolicated stackframe when the binary
doesn't exist.
llvm-svn: 348659
Summary:
fparseln - returns the next logical line from a stream.
Add a dedicated test for this API.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55019
llvm-svn: 348654
Summary:
strtonum(3) reliably convertss string value to an integer.
This function is used in OpenBSD compat namespace
and is located inside NetBSD's libc.
Add a dedicated test for this interface.
It's a reworked version of the original code by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54527
llvm-svn: 348651
Summary:
This change builds upon D54989, which removes memory allocation from the
critical path of the profiling implementation. This also changes the API
for the profile collection service, to take ownership of the memory and
associated data structures per-thread.
The consolidation of the memory allocation allows us to do two things:
- Limits the amount of memory used by the profiling implementation,
associating preallocated buffers instead of allocating memory
on-demand.
- Consolidate the memory initialisation and cleanup by relying on the
buffer queue's reference counting implementation.
We find a number of places which also display some problematic
behaviour, including:
- Off-by-factor bug in the allocator implementation.
- Unrolling semantics in cases of "memory exhausted" situations, when
managing the state of the function call trie.
We also add a few test cases which verify our understanding of the
behaviour of the system, with important edge-cases (especially for
memory-exhausted cases) in the segmented array and profile collector
unit tests.
Depends on D54989.
Reviewers: mboerger
Subscribers: dschuff, mgorny, dmgreen, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D55249
llvm-svn: 348568
This reverts commit r348455, with some additional changes:
- Work-around deficiency of gcc-4.8 by duplicating the implementation of
`AppendEmplace` in `Append`, but instead of using brace-init for the
copy construction, use a placement new explicitly calling the copy
constructor.
llvm-svn: 348563
Include CompilerRTCompile in fuzzer tests explicitly. Otherwise, when
building only libFuzzer, CMake fails due to:
CMake Error at cmake/Modules/AddCompilerRT.cmake:395 (sanitizer_test_compile):
Unknown CMake command "sanitizer_test_compile".
Call Stack (most recent call first):
lib/fuzzer/tests/CMakeLists.txt:53 (generate_compiler_rt_tests)
Differential Revision: https://reviews.llvm.org/D55378
llvm-svn: 348524
Summary:
The name of the function is based on `malloc_default_zone()` found
in Darwin's `malloc/malloc.h` header file.
Reviewers: kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55064
llvm-svn: 348481
This is a follow-up to D54989.
Work-around gcc-4.8 failing to handle brace-init for structs to imply
default-construction of an aggregate, and treats it as an initialiser
list instead.
llvm-svn: 348445
Continuation of D54989.
Additional changes:
- Use `.AppendEmplace(...)` instead of `.Append(Type{...})` to appease
GCC 4.8 with confusion on when an initializer_list is used as
opposed to a temporary aggregate initialized object.
llvm-svn: 348438
.. and also the follow-ups r348336 r348338.
It broke stand-alone compiler-rt builds with GCC 4.8:
In file included from /work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:20:0,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.h:21,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:15:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&}; T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’
/work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:517:54: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget’
new (AlignedOffset) T{std::forward<Args>(args)...};
^
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ThreadTrie&}; T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:98:34: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ThreadTrie&>((* & args#0))}’ from
‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ThreadTrie’
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ProfileBuffer&}; T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]
’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:244:44: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ProfileBuffer&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ProfileBuffer’
> Summary:
> This change makes the allocator and function call trie implementations
> move-aware and remove the FunctionCallTrie's reliance on a
> heap-allocated set of allocators.
>
> The change makes it possible to always have storage associated with
> Allocator instances, not necessarily having heap-allocated memory
> obtainable from these allocator instances. We also use thread-local
> uninitialised storage.
>
> We've also re-worked the segmented array implementation to have more
> precondition and post-condition checks when built in debug mode. This
> enables us to better implement some of the operations with surrounding
> documentation as well. The `trim` algorithm now has more documentation
> on the implementation, reducing the requirement to handle special
> conditions, and being more rigorous on the computations involved.
>
> In this change we also introduce an initialisation guard, through which
> we prevent an initialisation operation from racing with a cleanup
> operation.
>
> We also ensure that the ThreadTries array is not destroyed while copies
> into the elements are still being performed by other threads submitting
> profiles.
>
> Note that this change still has an issue with accessing thread-local
> storage from signal handlers that are instrumented with XRay. We also
> learn that with the testing of this patch, that there will be cases
> where calls to mmap(...) (through internal_mmap(...)) might be called in
> signal handlers, but are not async-signal-safe. Subsequent patches will
> address this, by re-using the `BufferQueue` type used in the FDR mode
> implementation for pre-allocated memory segments per active, tracing
> thread.
>
> We still want to land this change despite the known issues, with fixes
> forthcoming.
>
> Reviewers: mboerger, jfb
>
> Subscribers: jfb, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348346
Summary:
This change makes the allocator and function call trie implementations
move-aware and remove the FunctionCallTrie's reliance on a
heap-allocated set of allocators.
The change makes it possible to always have storage associated with
Allocator instances, not necessarily having heap-allocated memory
obtainable from these allocator instances. We also use thread-local
uninitialised storage.
We've also re-worked the segmented array implementation to have more
precondition and post-condition checks when built in debug mode. This
enables us to better implement some of the operations with surrounding
documentation as well. The `trim` algorithm now has more documentation
on the implementation, reducing the requirement to handle special
conditions, and being more rigorous on the computations involved.
In this change we also introduce an initialisation guard, through which
we prevent an initialisation operation from racing with a cleanup
operation.
We also ensure that the ThreadTries array is not destroyed while copies
into the elements are still being performed by other threads submitting
profiles.
Note that this change still has an issue with accessing thread-local
storage from signal handlers that are instrumented with XRay. We also
learn that with the testing of this patch, that there will be cases
where calls to mmap(...) (through internal_mmap(...)) might be called in
signal handlers, but are not async-signal-safe. Subsequent patches will
address this, by re-using the `BufferQueue` type used in the FDR mode
implementation for pre-allocated memory segments per active, tracing
thread.
We still want to land this change despite the known issues, with fixes
forthcoming.
Reviewers: mboerger, jfb
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348335
NetBSD 8.99.26 changed the layout of internal structure
returned by dlopen(3), switch to it.
Set new values for amd64 and i386 based on the results
of &((struct Struct_Obj_Entry*)0)->linkmap.
llvm-svn: 348329
Summary:
Flag was added for testing 3 years ago. Probably it's time
to simplify code and usage by removing it.
Reviewers: eugenis, m.ostapenko
Subscribers: mehdi_amini, kubamracek, steven_wu, dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D55254
llvm-svn: 348315
Summary:
Previously we weren't testing this function in the unit tests.
Reviewers: kcc, cryptoad, dvyukov, eugenis, kubamracek
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54861
llvm-svn: 348260
This finishes the soft-transition to the new primitive that implements
priority inheritance.
Differential Revision: https://reviews.llvm.org/D55244
llvm-svn: 348236
The res returned value might differ with REAL(strlen)(errbuf) + 1,
as the buffer's value is limited with errbuf_size.
Hot fix for D54584.
llvm-svn: 348231
Summary:
fts(3) is API to traverse a file hierarchy.
Cover this interface with interceptors.
Add a test to validate the interface reading
the number of regular files in /etc.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54247
llvm-svn: 348227
Summary:
The purpose of this option is provide a way for the ASan dylib
to be loaded via `dlopen()` without triggering most initialization
steps (e.g. shadow memory set up) that normally occur when the
ASan dylib is loaded.
This new functionality is exposed by
- A `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` macro which indicates if the
feature is supported. This only true for Darwin currently.
- A `HandleDlopenInit()` function which should return true if the library
is being loaded via `dlopen()` and
`SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` is supported. Platforms that
support this may perform any initialization they wish inside this
function.
Although disabling initialization is something that could potentially
apply to other sanitizers it appears to be unnecessary for other
sanitizers so this patch only makes the change for ASan.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov, kcc, eugenis, krytarowski
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54469
llvm-svn: 348078
These changed as a result of r347379. Unfortunately there was a
regression; filed PR39748 to track it.
Differential Revision: https://reviews.llvm.org/D54821
llvm-svn: 347442
Summary:
D48660 / rL335762 added a `silence_unsigned_overflow` env flag for [[ https://github.com/google/oss-fuzz/pull/1717 | oss-fuzz needs ]],
that allows to silence the reports from unsigned overflows.
It makes sense, it is there because `-fsanitize=integer` sanitizer is not enabled on oss-fuzz,
so this allows to still use it as an interestingness signal, without getting the actual reports.
However there is a slight problem here.
All types of unsigned overflows are ignored.
Even if `-fno-sanitize-recover=unsigned` was used (which means the program will die after the report)
there will still be no report, the program will just silently die.
At the moment there are just two projects on oss-fuzz that care:
* [[ 8eeffa627f/projects/llvm_libcxx/build.sh (L18-L20) | libc++ ]]
* [[ 8eeffa627f/projects/librawspeed/build.sh | RawSpeed ]] (me)
I suppose this could be overridden there ^, but i really don't think this is intended behavior in any case..
Reviewers: kcc, Dor1s, #sanitizers, filcab, vsk, kubamracek
Reviewed By: Dor1s
Subscribers: dberris, mclow.lists, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54771
llvm-svn: 347415
Add pthread_tryjoin_np() and pthread_timedjoin_np() interceptors on Linux,
so that ThreadSanitizer can handle programs using these functions.
Author: Yuri Per (yuri)
Reviewed in: https://reviews.llvm.org/D54521
llvm-svn: 347383
Use a more representative test of allocating small chunks for
oddly-sized (small) objects from an allocator that has a page's worth of
memory.
llvm-svn: 347286
Summary:
This change addresses an issue which shows up with the synchronised race
between threads writing into a buffer, and another thread reading the
buffer.
In a lot of cases, we cannot guarantee that threads will always see the
signal to finalise their buffers in time despite the grace periods and
state machine maintained through atomic variables. This change addresses
it by ensuring that the same instance being updated to indicate how much
of the buffer is "used" by the writing thread is the same instance being
read by the thread processing the buffer to be written out to disk or
handled through the iterators.
To do this, we ensure that all the "extents" instances live in their own
the backing store, in a different contiguous page from the
buffer-specific backing store. We also take precautions to ensure that
the atomic variables are cache-line-sized to prevent false-sharing from
unnecessarily causing cache contention on unrelated writes/reads.
It's feasible that we may in the future be able to move the storage of
the extents objects into the single backing store, slightly changing the
way to compute the size(s) of the buffers, but in the meantime we'll
settle for the isolation afforded by having a different backing store
for the extents instances.
Reviewers: mboerger
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54684
llvm-svn: 347280
Francis Visoiu Mistrih