Summary:
This is a follow-up to r346956 (https://reviews.llvm.org/D53975).
The purpose of this change to allow implementers of the
`AddressSpaceView` to be able to distinguish between when a caller wants
read-only memory and when a caller wants writable memory. Being able
distinguish these cases allows implementations to optimize for the
different cases and also provides a way to workaround possible platform
restrictions (e.g. the low level platform interface for reading
out-of-process memory may place memory in read-only pages).
For allocator enumeration in almost all cases read-only is sufficient so
we make `Load(...)` take on this new requirement and introduce the
`LoadWritable(...)` variants for cases where memory needs to be
writable.
The behaviour of `LoadWritable(...)` documented in comments are
deliberately very restrictive so that it will be possible in the future
to implement a simple write-cache (i.e. just a map from target address
to a writable region of memory). These restrictions can be loosened in
the future if necessary by implementing a more sophisticated
write-cache.
rdar://problem/45284065
Reviewers: kcc, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54879
llvm-svn: 350136
- Disabled on purpose on Android and Darwin platform (for now).
- Darwin supports it, would need interception in its specific code before enabling it.
- Linux does not support it but only via third party library.
- Android supports it via bionic however it is known to have issue with older versions of the implementations. Can be enabled by an Android committer later on if necessary once there is more 'certainity'/been more tested.
Reviewers: krytarowski, vitalybuka
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D56125
llvm-svn: 350123
Revert r350104 "[asan] Fix build on windows."
Revert r350101 "[asan] Support running without /proc."
These changes break Mac build, too.
llvm-svn: 350112
Add SIOCSETHERCAP.
Add commented out NVMM (NetBSD Virtual Machine Monitor) operations
as this interface is still WIP and a subject to change.
llvm-svn: 350107
Summary: They happen to work out of the box.
Reviewers: rtrieu, vitalybuka
Subscribers: kubamracek, fedor.sergeev, krytarowski, llvm-commits
Differential Revision: https://reviews.llvm.org/D56088
llvm-svn: 350103
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the MSan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.
New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.
Reviewers: joerg, vitalybuka, jfb
Reviewed By: vitalybuka
Subscribers: emaste, jfb, llvm-commits, #sanitizers, mgorny
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55703
llvm-svn: 350102
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.
This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55874
llvm-svn: 350101
Summary: Assigning an array of bools to {false, false, false} can, in certain build configurations, lead to a memset call. Use internal_memset to avoid this.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D56107
llvm-svn: 350098
Summary:
By an accident part of the tests contained hardcoded checksums
for external files that will differ between setups.
Reviewers: mgorny
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56104
llvm-svn: 350097
Summary:
By an accident part of the tests contained hardcoded checksums
for external files that will differ between setups.
Reviewers: mgorny
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56105
llvm-svn: 350096
Summary:
By an accident part of the tests contained hardcoded checksums
for external files that will differ between setups.
Reviewers: mgorny
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56106
llvm-svn: 350095
Add a CheckMPROTECT() routine to detect when pax MPROTECT is enabled
on NetBSD, and error xray out when it is. The solution is adapted
from existing CheckASLR().
Differential Revision: https://reviews.llvm.org/D56049
llvm-svn: 350030
Disable enforcing alignas() for structs that are used as thread_local
data on NetBSD. The NetBSD ld.so implementation is buggy and does
not enforce correct alignment; however, clang seems to take it for
granted and generates instructions that segv on wrongly aligned objects.
Therefore, disable those alignas() statements on NetBSD until we can
establish a better fix.
Apparently, std::aligned_storage<> does not have any real effect
at the moment, so we can leave it as-is.
Differential Revision: https://reviews.llvm.org/D56000
llvm-svn: 350029
Reviewers: krytarowski
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D56009
M lib/sanitizer_common/sanitizer_common_interceptors.inc
M lib/sanitizer_common/sanitizer_platform_interceptors.h
M lib/sanitizer_common/sanitizer_platform_limits_freebsd.cc
M lib/sanitizer_common/sanitizer_platform_limits_freebsd.h
D test/sanitizer_common/TestCases/NetBSD/regex.cc
A + test/sanitizer_common/TestCases/Posix/regex.cc
llvm-svn: 350002
Summary:
NetBSD uses typical UNIX interfaces.
All tests pass except instrprof-dlopen-dlclose-gcov.test, as there
is not supported semantics of atexit(3) in dlopen(3)ed+dlclose(3)d
DSO.
NetBSD also ships an older version of LLVM profile (ABI v.2 predating
ABI v.4 in upstream version) inside libc. That copy has been manually
removed during the porting and testing process of the upstream version
to NetBSD. Otherwise there were conflicts between them two.
Reviewers: joerg, vitalybuka, vsk
Subscribers: srhines, fedor.sergeev, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55997
llvm-svn: 349994
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.
This is a follow up to r349957.
rdar://problem/45284065
llvm-svn: 349960
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.
This is a follow up to r349138.
rdar://problem/45284065
llvm-svn: 349959
Summary:
This is a follow up to https://reviews.llvm.org/D55764 .
For the ASan and LSan allocatorsthe type declarations have been modified
so that it's possible to create a combined allocator type that
consistently uses a different type of `AddressSpaceView`. We intend to
use this in future patches. For the other sanitizers they just use
`LocalAddressSpaceView` by default because we have no plans to use these
allocators in an out-of-process manner.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov, yln
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55766
llvm-svn: 349957
Summary:
This is a follow up patch to r349138.
This patch makes a `AddressSpaceView` a type declaration in the
allocator parameters used by `SizeClassAllocator64`. For ASan, LSan, and
the unit tests the AP64 declarations have been made templated so that
`AddressSpaceView` can be changed at compile time. For the other
sanitizers we just hard-code `LocalAddressSpaceView` because we have no
plans to use these allocators in an out-of-process manner.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55764
llvm-svn: 349954
Add a code to properly test for presence of LLVMTestingSupport library
when performing a stand-alone build, and skip tests requiring it when
it is not present. Since the library is not installed, llvm-config
reported empty --libs for it and the tests failed to link with undefined
references. Skipping the two fdr_* test files is better than failing to
build, and should be good enough until we find a better solution.
NB: both installing LLVMTestingSupport and building it automatically
from within compiler-rt sources are non-trivial. The former due to
dependency on gtest, the latter due to tight integration with LLVM
source tree.
Differential Revision: https://reviews.llvm.org/D55891
llvm-svn: 349899
Summary:
This is a change requested by Vitaly Buka as prerequisite to landing
https://reviews.llvm.org/D55740.
Reviewers: vitalybuka, kubamracek
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55939
llvm-svn: 349897
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.
(2nd attempt, this time without the sanitizer_rtems change)
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55801
llvm-svn: 349817
This is patch complements D55117 implementing __hwasan_mem*
functions in runtime
Differential revision: https://reviews.llvm.org/D55554
llvm-svn: 349730
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55801
llvm-svn: 349699
Summary:
An extra ';' at the end of a namespace triggers a pedantic warning:
```
.../sanitizer_common/sanitizer_type_traits.h:42:2: warning: extra ‘;’ [-Wpedantic]
}; // namespace __sanitizer
```
Reviewers: eugenis, delcypher
Reviewed By: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55849
llvm-svn: 349666
Fix the sha2 interceptor macros to use a constant for array parameter
length rather than referencing the extern variable. Since the digest
length is provided in hash name, reuse the macro parameter for it.
Verify that the calculated value matches the one provided by system
headers.
Differential Revision: https://reviews.llvm.org/D55811
llvm-svn: 349645
This patch aims to enable the tests for the compiler-rt builtin functions (that
currently already exist within compiler-rt) for PowerPC 64bit LE (ppc64le).
Previously when unit tests are run, these tests would be reported as
UNSUPPORTED. This patch updates the REQUIRES line for each test (to enable for
ppc64le), and each test is linked against compiler-rt when running.
Differential Revision: https://reviews.llvm.org/D54449
llvm-svn: 349634
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the ASan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.
New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.
Reviewers: joerg, vitalybuka, jfb
Reviewed By: vitalybuka
Subscribers: dim, emaste, ro, jfb, devnexen, kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55596
llvm-svn: 349619
I tricked myself into thinking that deadlock detection is off by default in TSan by looking at the default value of the detect_deadlocks flag and outdated docs. (Created a pull request to update docs.)
I even managed to confuse others: https://groups.google.com/forum/#!topic/thread-sanitizer/xYvnAYwtoDk
However, the default value is overwritten in code (TSan_flags.cc:InitializeFlags). The TSan/deadlock tests also rely on this
This changes aligns the default value of the flag with the actual default behavior.
Author: yln (Julian Lettner)
Reviewed in: https://reviews.llvm.org/D55846
llvm-svn: 349609
Summary:
unnamed_addr is still useful for detecting of ODR violations on vtables
Still unnamed_addr with lld and --icf=safe or --icf=all can trigger false
reports which can be avoided with --icf=none or by using private aliases
with -fsanitize-address-use-odr-indicator
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: kubamracek, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D55799
llvm-svn: 349555
As of r349413 it's now possible for a binary to contain an empty
hwasan frame section. Handle that case simply by doing nothing.
Differential Revision: https://reviews.llvm.org/D55796
llvm-svn: 349428
Summary:
This fixes the `ThreadSanitizer-x86_64-iossim` testsuite which broke
when r348770 (https://reviews.llvm.org/D55473) landed.
The root cause of the problem is that early-on during the iOS simulator
init process a call to `sysctlbyname` is issued. If the TSan initializer
is triggered at this point it will eventually trigger a call to
`__cxa_at_exit(...)`. This call then aborts because the library
implementing this function is not yet had its initialization function
called.
rdar://problem/46696934
Reviewers: kubamracek, george.karpenkov, devnexen, vitalybuka, krytarowski
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55711
llvm-svn: 349402
Summary:
This is a follow up patch to r346956 for the `SizeClassAllocator32`
allocator.
This patch makes `AddressSpaceView` a template parameter both to the
`ByteMap` implementations (but makes `LocalAddressSpaceView` the
default), some `AP32` implementations and is used in `SizeClassAllocator32`.
The actual changes to `ByteMap` implementations and
`SizeClassAllocator32` are very simple. However the patch is large
because it requires changing all the `AP32` definitions, and users of
those definitions.
For ASan and LSan we make `AP32` and `ByteMap` templateds type that take
a single `AddressSpaceView` argument. This has been done because we will
instantiate the allocator with a type that isn't `LocalAddressSpaceView`
in the future patches. For the allocators used in the other sanitizers
(i.e. HWAsan, MSan, Scudo, and TSan) use of `LocalAddressSpaceView` is
hard coded because we do not intend to instantiate the allocators with
any other type.
In the cases where untemplated types have become templated on a single
`AddressSpaceView` parameter (e.g. `PrimaryAllocator`) their name has
been changed to have a `ASVT` suffix (Address Space View Type) to
indicate they are templated. The only exception to this are the `AP32`
types due to the desire to keep the type name as short as possible.
In order to check that template is instantiated in the correct a way a
`static_assert(...)` has been added that checks that the
`AddressSpaceView` type used by `Params::ByteMap::AddressSpaceView` matches
the `Params::AddressSpaceView`. This uses the new `sanitizer_type_traits.h`
header.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D54904
llvm-svn: 349138
Summary:
A recent update to the VS toolchain in chromium [1] broke the windows
ASan bot because the new toolchain calls _msize_base() instead of
_msize() in a number of _aligned_* UCRT routines. Instrument
_msize_base() as well.
[1] https://crbug.com/914947
Reviewers: rnk, #sanitizers, vitalybuka
Reviewed By: rnk, #sanitizers, vitalybuka
Subscribers: vitalybuka, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55684
llvm-svn: 349115
Summary:
In particular we implement the `is_same<T,U>` templated type. This is
useful for doing compile-time comparison of types in `static_assert`s.
The plan is to use this in another patch (
https://reviews.llvm.org/D54904 ).
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54951
llvm-svn: 349077
Summary:
private and internal: should not trigger ODR at all.
unnamed_addr: current ODR checking approach fail and rereport false violation if
a linker merges such globals
linkonce_odr, weak_odr: could cause similar problems and they are already not
instrumented for ELF.
Reviewers: eugenis, kcc
Subscribers: kubamracek, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D55621
llvm-svn: 349015
Summary:
The TLS_SLOT_TSAN slot is available starting in N, but its location (8)
is incompatible with the proposed solution for implementing ELF TLS on
Android (i.e. bump ARM/AArch64 alignment to reserve an 8-word TCB).
Instead, starting in Q, Bionic replaced TLS_SLOT_DLERROR(6) with
TLS_SLOT_SANITIZER(6). Switch compiler-rt to the new slot.
Reviewers: eugenis, srhines, enh
Reviewed By: eugenis
Subscribers: ruiu, srhines, kubamracek, javed.absar, kristof.beyls, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D55581
llvm-svn: 348984
Summary:
Add a check that TLS_SLOT_TSAN / TLS_SLOT_SANITIZER, whichever
android_get_tls_slot is using, is not conflicting with
TLS_SLOT_DLERROR.
Reviewers: rprichard, vitalybuka
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55587
llvm-svn: 348979
Summary:
Fix bug where we'd try symbolize a second time with the same arguments even though symbolization failed the first time.
This looks like a long standing typo given that the guard for trying
symbolization again is to only try it if symbolization failed using
`binary` and `original_binary != binary`.
Reviewers: kubamracek, glider, samsonov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55504
llvm-svn: 348841
Summary:
Add new interceptors for a set of functions to reposition a stream:
fgetpos, fseek, fseeko, fsetpos, ftell, ftello, rewind .
Add a dedicated test.
Enable this interface on NetBSD.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55471
llvm-svn: 348743
- For the moment a subset of this api dealing with file descriptors permissions and ioctls.
Reviewers: vitalybuka, krytarowski
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D55368
llvm-svn: 348668
Summary:
strtoi/strtou converts string value to an intmax_t/uintmax_t integer.
Add a dedicated test.
Enable this API for NetBSD.
It's a reworked version of the original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, tomsun.0.7, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54702
llvm-svn: 348663
using `atos` symbolizer on Darwin when the binaries don't exist.
For now we just produce an unsymbolicated stackframe when the binary
doesn't exist.
llvm-svn: 348659
Summary:
fparseln - returns the next logical line from a stream.
Add a dedicated test for this API.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55019
llvm-svn: 348654
Summary:
strtonum(3) reliably convertss string value to an integer.
This function is used in OpenBSD compat namespace
and is located inside NetBSD's libc.
Add a dedicated test for this interface.
It's a reworked version of the original code by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54527
llvm-svn: 348651
Summary:
This change builds upon D54989, which removes memory allocation from the
critical path of the profiling implementation. This also changes the API
for the profile collection service, to take ownership of the memory and
associated data structures per-thread.
The consolidation of the memory allocation allows us to do two things:
- Limits the amount of memory used by the profiling implementation,
associating preallocated buffers instead of allocating memory
on-demand.
- Consolidate the memory initialisation and cleanup by relying on the
buffer queue's reference counting implementation.
We find a number of places which also display some problematic
behaviour, including:
- Off-by-factor bug in the allocator implementation.
- Unrolling semantics in cases of "memory exhausted" situations, when
managing the state of the function call trie.
We also add a few test cases which verify our understanding of the
behaviour of the system, with important edge-cases (especially for
memory-exhausted cases) in the segmented array and profile collector
unit tests.
Depends on D54989.
Reviewers: mboerger
Subscribers: dschuff, mgorny, dmgreen, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D55249
llvm-svn: 348568
This reverts commit r348455, with some additional changes:
- Work-around deficiency of gcc-4.8 by duplicating the implementation of
`AppendEmplace` in `Append`, but instead of using brace-init for the
copy construction, use a placement new explicitly calling the copy
constructor.
llvm-svn: 348563
Include CompilerRTCompile in fuzzer tests explicitly. Otherwise, when
building only libFuzzer, CMake fails due to:
CMake Error at cmake/Modules/AddCompilerRT.cmake:395 (sanitizer_test_compile):
Unknown CMake command "sanitizer_test_compile".
Call Stack (most recent call first):
lib/fuzzer/tests/CMakeLists.txt:53 (generate_compiler_rt_tests)
Differential Revision: https://reviews.llvm.org/D55378
llvm-svn: 348524
Summary:
The name of the function is based on `malloc_default_zone()` found
in Darwin's `malloc/malloc.h` header file.
Reviewers: kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55064
llvm-svn: 348481
This is a follow-up to D54989.
Work-around gcc-4.8 failing to handle brace-init for structs to imply
default-construction of an aggregate, and treats it as an initialiser
list instead.
llvm-svn: 348445
Continuation of D54989.
Additional changes:
- Use `.AppendEmplace(...)` instead of `.Append(Type{...})` to appease
GCC 4.8 with confusion on when an initializer_list is used as
opposed to a temporary aggregate initialized object.
llvm-svn: 348438
.. and also the follow-ups r348336 r348338.
It broke stand-alone compiler-rt builds with GCC 4.8:
In file included from /work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:20:0,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.h:21,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:15:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&}; T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’
/work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:517:54: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget’
new (AlignedOffset) T{std::forward<Args>(args)...};
^
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ThreadTrie&}; T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:98:34: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ThreadTrie&>((* & args#0))}’ from
‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ThreadTrie’
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ProfileBuffer&}; T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]
’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:244:44: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ProfileBuffer&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ProfileBuffer’
> Summary:
> This change makes the allocator and function call trie implementations
> move-aware and remove the FunctionCallTrie's reliance on a
> heap-allocated set of allocators.
>
> The change makes it possible to always have storage associated with
> Allocator instances, not necessarily having heap-allocated memory
> obtainable from these allocator instances. We also use thread-local
> uninitialised storage.
>
> We've also re-worked the segmented array implementation to have more
> precondition and post-condition checks when built in debug mode. This
> enables us to better implement some of the operations with surrounding
> documentation as well. The `trim` algorithm now has more documentation
> on the implementation, reducing the requirement to handle special
> conditions, and being more rigorous on the computations involved.
>
> In this change we also introduce an initialisation guard, through which
> we prevent an initialisation operation from racing with a cleanup
> operation.
>
> We also ensure that the ThreadTries array is not destroyed while copies
> into the elements are still being performed by other threads submitting
> profiles.
>
> Note that this change still has an issue with accessing thread-local
> storage from signal handlers that are instrumented with XRay. We also
> learn that with the testing of this patch, that there will be cases
> where calls to mmap(...) (through internal_mmap(...)) might be called in
> signal handlers, but are not async-signal-safe. Subsequent patches will
> address this, by re-using the `BufferQueue` type used in the FDR mode
> implementation for pre-allocated memory segments per active, tracing
> thread.
>
> We still want to land this change despite the known issues, with fixes
> forthcoming.
>
> Reviewers: mboerger, jfb
>
> Subscribers: jfb, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348346
Summary:
This change makes the allocator and function call trie implementations
move-aware and remove the FunctionCallTrie's reliance on a
heap-allocated set of allocators.
The change makes it possible to always have storage associated with
Allocator instances, not necessarily having heap-allocated memory
obtainable from these allocator instances. We also use thread-local
uninitialised storage.
We've also re-worked the segmented array implementation to have more
precondition and post-condition checks when built in debug mode. This
enables us to better implement some of the operations with surrounding
documentation as well. The `trim` algorithm now has more documentation
on the implementation, reducing the requirement to handle special
conditions, and being more rigorous on the computations involved.
In this change we also introduce an initialisation guard, through which
we prevent an initialisation operation from racing with a cleanup
operation.
We also ensure that the ThreadTries array is not destroyed while copies
into the elements are still being performed by other threads submitting
profiles.
Note that this change still has an issue with accessing thread-local
storage from signal handlers that are instrumented with XRay. We also
learn that with the testing of this patch, that there will be cases
where calls to mmap(...) (through internal_mmap(...)) might be called in
signal handlers, but are not async-signal-safe. Subsequent patches will
address this, by re-using the `BufferQueue` type used in the FDR mode
implementation for pre-allocated memory segments per active, tracing
thread.
We still want to land this change despite the known issues, with fixes
forthcoming.
Reviewers: mboerger, jfb
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348335
NetBSD 8.99.26 changed the layout of internal structure
returned by dlopen(3), switch to it.
Set new values for amd64 and i386 based on the results
of &((struct Struct_Obj_Entry*)0)->linkmap.
llvm-svn: 348329
Summary:
Flag was added for testing 3 years ago. Probably it's time
to simplify code and usage by removing it.
Reviewers: eugenis, m.ostapenko
Subscribers: mehdi_amini, kubamracek, steven_wu, dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D55254
llvm-svn: 348315
Summary:
Previously we weren't testing this function in the unit tests.
Reviewers: kcc, cryptoad, dvyukov, eugenis, kubamracek
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54861
llvm-svn: 348260
This finishes the soft-transition to the new primitive that implements
priority inheritance.
Differential Revision: https://reviews.llvm.org/D55244
llvm-svn: 348236
The res returned value might differ with REAL(strlen)(errbuf) + 1,
as the buffer's value is limited with errbuf_size.
Hot fix for D54584.
llvm-svn: 348231
Summary:
fts(3) is API to traverse a file hierarchy.
Cover this interface with interceptors.
Add a test to validate the interface reading
the number of regular files in /etc.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54247
llvm-svn: 348227
Make sure that symbols needed to implement runtime support for gcov are
exported when using an export list on Darwin.
Without the clang driver exporting these symbols, the linker hides them,
resulting in tapi verification failures.
rdar://45944768
Differential Revision: https://reviews.llvm.org/D55151
llvm-svn: 348187
Summary:
The purpose of this option is provide a way for the ASan dylib
to be loaded via `dlopen()` without triggering most initialization
steps (e.g. shadow memory set up) that normally occur when the
ASan dylib is loaded.
This new functionality is exposed by
- A `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` macro which indicates if the
feature is supported. This only true for Darwin currently.
- A `HandleDlopenInit()` function which should return true if the library
is being loaded via `dlopen()` and
`SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` is supported. Platforms that
support this may perform any initialization they wish inside this
function.
Although disabling initialization is something that could potentially
apply to other sanitizers it appears to be unnecessary for other
sanitizers so this patch only makes the change for ASan.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov, kcc, eugenis, krytarowski
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54469
llvm-svn: 348078
These changed as a result of r347379. Unfortunately there was a
regression; filed PR39748 to track it.
Differential Revision: https://reviews.llvm.org/D54821
llvm-svn: 347442
Summary:
D48660 / rL335762 added a `silence_unsigned_overflow` env flag for [[ https://github.com/google/oss-fuzz/pull/1717 | oss-fuzz needs ]],
that allows to silence the reports from unsigned overflows.
It makes sense, it is there because `-fsanitize=integer` sanitizer is not enabled on oss-fuzz,
so this allows to still use it as an interestingness signal, without getting the actual reports.
However there is a slight problem here.
All types of unsigned overflows are ignored.
Even if `-fno-sanitize-recover=unsigned` was used (which means the program will die after the report)
there will still be no report, the program will just silently die.
At the moment there are just two projects on oss-fuzz that care:
* [[ 8eeffa627f/projects/llvm_libcxx/build.sh (L18-L20) | libc++ ]]
* [[ 8eeffa627f/projects/librawspeed/build.sh | RawSpeed ]] (me)
I suppose this could be overridden there ^, but i really don't think this is intended behavior in any case..
Reviewers: kcc, Dor1s, #sanitizers, filcab, vsk, kubamracek
Reviewed By: Dor1s
Subscribers: dberris, mclow.lists, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54771
llvm-svn: 347415
Add pthread_tryjoin_np() and pthread_timedjoin_np() interceptors on Linux,
so that ThreadSanitizer can handle programs using these functions.
Author: Yuri Per (yuri)
Reviewed in: https://reviews.llvm.org/D54521
llvm-svn: 347383
Copy the fix for determining the correct terminfo library from LLVM --
use distinct variables for check_library_exists() calls. Otherwise,
the first check (for -ltinfo) populates the variable and no other checks
are performed. Effectively, systems with other libraries than the first
one listed are presumed not to have terminfo routines at all.
Also sync the check order to include the NetBSD fix from r347156.
This partially fixes undefined symbols when linking XRay tests. It's
probably not the best solution to the problem there but as long
as the terminfo check stays in config-ix, I thnk it's worth fixing.
Differential Revision: https://reviews.llvm.org/D54641
llvm-svn: 347338
Use a more representative test of allocating small chunks for
oddly-sized (small) objects from an allocator that has a page's worth of
memory.
llvm-svn: 347286
Summary:
This change addresses an issue which shows up with the synchronised race
between threads writing into a buffer, and another thread reading the
buffer.
In a lot of cases, we cannot guarantee that threads will always see the
signal to finalise their buffers in time despite the grace periods and
state machine maintained through atomic variables. This change addresses
it by ensuring that the same instance being updated to indicate how much
of the buffer is "used" by the writing thread is the same instance being
read by the thread processing the buffer to be written out to disk or
handled through the iterators.
To do this, we ensure that all the "extents" instances live in their own
the backing store, in a different contiguous page from the
buffer-specific backing store. We also take precautions to ensure that
the atomic variables are cache-line-sized to prevent false-sharing from
unnecessarily causing cache contention on unrelated writes/reads.
It's feasible that we may in the future be able to move the storage of
the extents objects into the single backing store, slightly changing the
way to compute the size(s) of the buffers, but in the meantime we'll
settle for the isolation afforded by having a different backing store
for the extents instances.
Reviewers: mboerger
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54684
llvm-svn: 347280
This change is part of the soft-transition to the new synchronization
primitives which implement priority inheritance.
Differential Revision: https://reviews.llvm.org/D54727
llvm-svn: 347279
Summary:
setvbuf(3) is a routine to setup stream buffering.
Enable the interceptor for NetBSD.
Add dedicated tests for setvbuf(3) and functions
on top of this interface: setbuf, setbuffer, setlinebuf.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: devnexen, tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54548
llvm-svn: 347270
Summary:
With free_checks_tail_magic=1 (default) HWASAN
writes magic bytes to the tail of every heap allocation
(last bytes of the last granule, if the last granule is not fully used)
and checks these bytes on free().
This feature will detect buffer overwires within the last granule
at the time of free().
This is an alternative to malloc_align_right=[1289] that should have
fewer compatibility issues. It is also weaker since it doesn't
detect read overflows and reports bugs at free() instead of at access.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54656
llvm-svn: 347116
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.
No functional change intended for others.
Cherry-picked chunk from D33878.
llvm-svn: 347102
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.
No functional change intended for others.
Cherry-picked chunk from D33878.
llvm-svn: 347101
Summary:
mi_vector_hash(3) provides fast 32bit hash functions.
Add a test for this interface.
Enable the API for NetBSD.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54530
llvm-svn: 347088
Summary:
getmntinfo gets information about mounted file systems.
Add a dedicated test for new interceptor.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54611
llvm-svn: 347083
Summary:
... so that we can find intra-granule buffer overflows.
The default is still to always align left.
It remains to be seen wether we can enable this mode at scale.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: jfb, dvyukov, kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53789
llvm-svn: 347082
When building for default target only, use exact target spelling
when deriving the name for the per-target runtime directory. This
is necessary for AArch32 where the CMake build by default rewrites
the architecture which leads to unexpected results.
Differential Revision: https://reviews.llvm.org/D54612
llvm-svn: 347022
enumeration.
Summary:
This patch introduces the local portion (`LocalAddressSpaceView`) of the
`AddressSpaceView` abstraction and modifies the secondary allocator
so that the `ForEachChunk()` method (and its callees) would work in the
out-of-process case when `AddressSpaceView` is `RemoteAddressSpaceView`.
The `AddressSpaceView` abstraction simply maps pointers from a target
process to a pointer in the local process (via its `Load()` method). For
the local (in-process) case this is a no-op. For the remote
(out-of-process) case this is not a no-op. The implementation of the
out-of-process `RemoteAddressSpaceView` is not included in this patch
and will be introduced later.
This patch is considerably simpler than the `ObjectView` abstraction
used in previous patches but lacks the type safety and stricter memory
management of the `ObjectView` abstraction.
This patch does not introduce any tests because with
`LocalAddressSpaceView` it should be a non functional change and unit
tests already cover the secondary allocator. When
`RemoteAddressSpaceView` is landed tests will be added to ensure that it
functions as expected.
rdar://problem/45284065
Reviewers: kcc, kubamracek, dvyukov, vitalybuka, cryptoad,
george.karpenkov, morehouse
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53975
llvm-svn: 346956
This is needed when cross-compiling for a different target since
CFLAGS may contain additional flags like -resource-dir which
change the location in which compiler-rt builtins are found.
Differential Revision: https://reviews.llvm.org/D54371
llvm-svn: 346820
After the change, the tests started failing, as skipped sections can be
equal in size to kMaxSegName.
Changing `<` to `<=` to address the off-by-one problem.
llvm-svn: 346804
Calling ARRAY_SIZE on a char* will not actually compute it's size, but just the pointer size.
A new Clang warning enabled by default warns about this.
Replaced the call with internal_strnlen.
Differential Revision: https://reviews.llvm.org/D54484
llvm-svn: 346792
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.
Changes:
- Split paths of handling intercepted __cxa_atexit() and atexit(3).
This affects all supported Operating Systems.
- Add a local stack-like structure to hold the __cxa_atexit() context.
atexit(3) is documented in the C standard as calling callback from the
earliest to the oldest entry. This path also fixes potential ABI
problem of passing an argument to a function from the atexit(3)
callback mechanism.
- Allow usage of global vars with ctors in interceptors.
This allows to use Vector without automatic cleaning up the structures.
This code has been modeled after TSan implementation for the same functions.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc
Reviewed By: vitalybuka
Subscribers: delcypher, devnexen, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40714
llvm-svn: 346579
This change was reverted because it caused some nacl tests in chromium
to fail. I attempted to reproduce those problems locally, but I was
unable to. Let's reland this and let Chromium's test infrastructure
discover any problems.
llvm-svn: 346560
Summary:
When reporting a fatal error, collect and add the entire report text to
android_set_abort_message so that it can be found in the tombstone.
Reviewers: kcc, vitalybuka
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D54284
llvm-svn: 346557
This change adds a static check to ensure that all data metadata record
payloads don't go past the available buffers in Metadata records.
llvm-svn: 346476
Summary:
Before this change, we could run into a situation where we may try to
undo tail exit records after writing metadata records before a function
enter event. This change rectifies that by resetting the tail exit
counter after writing the metadata records.
Reviewers: mboerger
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D54292
llvm-svn: 346475
Summary:
We need these fences to ensure that other threads attempting to read
bytes in the buffer will see thw writes committed before the extents are
updated. Without these, the writes can be un-committed by the time the
buffer extents counter is updated -- the fences should ensure that the
records written into the log have completed by the time we observe the
buffer extents from different threads.
Reviewers: mboerger
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54291
llvm-svn: 346474
Summary:
This change covers a number of things spanning LLVM and compiler-rt,
which are related in a non-trivial way.
In LLVM, we have a library that handles the FDR mode even log loading,
which uses C++'s runtime polymorphism feature to better faithfully
represent the events that are written down by the FDR mode runtime. We
do this by interpreting a trace that's serliased in a common format
agreed upon by both the trace loading library and the FDR mode runtime.
This library is under active development, which consists of features
allowing us to reconstitute a higher-level event log.
This event log is used by the conversion and visualisation tools we have
for interpreting XRay traces.
One of the tools we have is a diagnostic tool in llvm-xray called
`fdr-dump` which we've been using to debug our expectations of what the
FDR runtime should be writing and what the logical FDR event log
structures are. We use this fairly extensively to reason about why some
non-trivial traces we're generating with FDR mode runtimes fail to
convert or fail to parse correctly.
One of these failures we've found in manual debugging of some of the
traces we've seen involve an inconsistency between the buffer extents (a
record indicating how many bytes to follow are part of a logical
thread's event log) and the record of the bytes written into the log --
sometimes it turns out the data could be garbage, due to buffers being
recycled, but sometimes we're seeing the buffer extent indicating a log
is "shorter" than the actual records associated with the buffer. This
case happens particularly with function entry records with a call
argument.
This change for now updates the FDR mode runtime to write the bytes for
the function call and arg record before updating the buffer extents
atomically, allowing multiple threads to see a consistent view of the
data in the buffer using the atomic counter associated with a buffer.
What we're trying to prevent here is partial updates where we see the
intermediary updates to the buffer extents (function record size then
call argument record size) becoming observable from another thread, for
instance, one doing the serialization/flushing.
To do both diagnose this issue properly, we need to be able to honour
the extents being set in the `BufferExtents` records marking the
beginning of the logical buffers when reading an FDR trace. Since LLVM
doesn't use C++'s RTTI mechanism, we instead follow the advice in the
documentation for LLVM Style RTTI
(https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html). We then rely on
this RTTI feature to ensure that our file-based record producer (our
streaming "deserializer") can honour the extents of individual buffers
as we interpret traces.
This also sets us up to be able to eventually do smart
skipping/continuation of FDR logs, seeking instead to find BufferExtents
records in cases where we find potentially recoverable errors. In the
meantime, we make this change to operate in a strict mode when reading
logical buffers with extent records.
Reviewers: mboerger
Subscribers: hiraditya, llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D54201
llvm-svn: 346473
Summary:
Currently `sanitizer_malloc_introspection_t` just adds a version field
which is used to version the allocator ABI. The current allocator ABI
version is returned by the new `GetMallocZoneAllocatorEnumerationVersion()` function.
The motivation behind this change is to allow external processes to
determine the allocator ABI of a sanitized process.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54045
llvm-svn: 346420
Summary: Remove the XFAIL for arm since it seems to be ok
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: javed.absar, kristof.beyls, delcypher, chrib, llvm-commits, #sanitizers, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D54263
llvm-svn: 346409
Summary:
Introduce SANITIZER_INTERCEPT_LGAMMAL
dedicated for lgammal(). Disable it for NetBSD
as this routine is not implemented in this OS.
Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54054
llvm-svn: 346352
Summary:
Introduce SANITIZER_INTERCEPT_REMQUOL
dedicated for remquol(). Disable for for NetBSD
as this routine is not implemented in this OS.
Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54051
llvm-svn: 346351
We have seen failing builds due to a race condition between
RTAsan_dynamic and libc++ headers builds, specifically libc++
headers depend on __config and if this header hasn't been copied
into the final location, including other headers will typically
result in failure. To avoid this race, we add an explicit dependency
on libc++ headers which ensures that they've been copied into place
before the sanitizer object library build starts.
Differential Revision: https://reviews.llvm.org/D54198
llvm-svn: 346339
Dan Liew