Implement the interceptors for popen(), pclose() and popenve()
functions. The first two are POSIX, the third one is specific
to NetBSD. popen() spawns a process and creates a FILE object piping
data from/to that process. pclose() closes the pipe and waits for
the process to terminate appropriately.
For the purpose of popen(), the COMMON_INTERCEPTOR_FILE_OPEN macro is
modified to allow null path parameter.
Differential Revision: https://reviews.llvm.org/D56157
llvm-svn: 350232
Add tests for the more character-oriented functions, that is:
- fputc(), putc() and putchar()
- getc_unlocked()
- putc_unlocked() and putchar_unlocked()
Differential Revision: https://reviews.llvm.org/D56152
llvm-svn: 350229
Update the interceptor for devname_r() to account for correct return
types on different platforms. This function returns int on NetBSD
but char* on FreeBSD/OSX. Noticed by @krytarowski.
Differential Revision: https://reviews.llvm.org/D56150
llvm-svn: 350228
Rewrite the tests for Posix functions that silently 'return 1'
or 'exit(1)' on error, to instead verbosely report the error using
assert. This is based on requests made in review of D56136.
Differential Revision: https://reviews.llvm.org/D56149
llvm-svn: 350227
Add two new test cases that test the following stdio.h functions:
- clearerr()
- feof()
- ferror()
- fileno()
- fgetc()
- getc()
- ungetc()
Differential Revision: https://reviews.llvm.org/D56136
llvm-svn: 350225
Summary:
Change the point of calling MaybeStartBackgroudThread() from AsanInitInternal()
that is too early on NetBSD to a constructor (with aid of C++11 lambda construct).
Enable the code for background thread as is for NetBSD.
Rename test/sanitizer_common/TestCases/Linux/hard_rss_limit_mb_test.cc
to test/sanitizer_common/TestCases/hard_rss_limit_mb_test.cc and allow runs
on NetBSD. This tests passes correctly.
Reviewers: vitalybuka, joerg, eugenis
Reviewed By: eugenis
Subscribers: eugenis, kubamracek, fedor.sergeev, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55887
llvm-svn: 350139
Summary:
This is a follow-up to r346956 (https://reviews.llvm.org/D53975).
The purpose of this change to allow implementers of the
`AddressSpaceView` to be able to distinguish between when a caller wants
read-only memory and when a caller wants writable memory. Being able
distinguish these cases allows implementations to optimize for the
different cases and also provides a way to workaround possible platform
restrictions (e.g. the low level platform interface for reading
out-of-process memory may place memory in read-only pages).
For allocator enumeration in almost all cases read-only is sufficient so
we make `Load(...)` take on this new requirement and introduce the
`LoadWritable(...)` variants for cases where memory needs to be
writable.
The behaviour of `LoadWritable(...)` documented in comments are
deliberately very restrictive so that it will be possible in the future
to implement a simple write-cache (i.e. just a map from target address
to a writable region of memory). These restrictions can be loosened in
the future if necessary by implementing a more sophisticated
write-cache.
rdar://problem/45284065
Reviewers: kcc, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54879
llvm-svn: 350136
- Disabled on purpose on Android and Darwin platform (for now).
- Darwin supports it, would need interception in its specific code before enabling it.
- Linux does not support it but only via third party library.
- Android supports it via bionic however it is known to have issue with older versions of the implementations. Can be enabled by an Android committer later on if necessary once there is more 'certainity'/been more tested.
Reviewers: krytarowski, vitalybuka
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D56125
llvm-svn: 350123
Revert r350104 "[asan] Fix build on windows."
Revert r350101 "[asan] Support running without /proc."
These changes break Mac build, too.
llvm-svn: 350112
Add SIOCSETHERCAP.
Add commented out NVMM (NetBSD Virtual Machine Monitor) operations
as this interface is still WIP and a subject to change.
llvm-svn: 350107
Summary: They happen to work out of the box.
Reviewers: rtrieu, vitalybuka
Subscribers: kubamracek, fedor.sergeev, krytarowski, llvm-commits
Differential Revision: https://reviews.llvm.org/D56088
llvm-svn: 350103
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the MSan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.
New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.
Reviewers: joerg, vitalybuka, jfb
Reviewed By: vitalybuka
Subscribers: emaste, jfb, llvm-commits, #sanitizers, mgorny
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55703
llvm-svn: 350102
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.
This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55874
llvm-svn: 350101
Summary: Assigning an array of bools to {false, false, false} can, in certain build configurations, lead to a memset call. Use internal_memset to avoid this.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D56107
llvm-svn: 350098
Summary:
By an accident part of the tests contained hardcoded checksums
for external files that will differ between setups.
Reviewers: mgorny
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56104
llvm-svn: 350097
Summary:
By an accident part of the tests contained hardcoded checksums
for external files that will differ between setups.
Reviewers: mgorny
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56105
llvm-svn: 350096
Summary:
By an accident part of the tests contained hardcoded checksums
for external files that will differ between setups.
Reviewers: mgorny
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56106
llvm-svn: 350095
Add a CheckMPROTECT() routine to detect when pax MPROTECT is enabled
on NetBSD, and error xray out when it is. The solution is adapted
from existing CheckASLR().
Differential Revision: https://reviews.llvm.org/D56049
llvm-svn: 350030
Disable enforcing alignas() for structs that are used as thread_local
data on NetBSD. The NetBSD ld.so implementation is buggy and does
not enforce correct alignment; however, clang seems to take it for
granted and generates instructions that segv on wrongly aligned objects.
Therefore, disable those alignas() statements on NetBSD until we can
establish a better fix.
Apparently, std::aligned_storage<> does not have any real effect
at the moment, so we can leave it as-is.
Differential Revision: https://reviews.llvm.org/D56000
llvm-svn: 350029
Reviewers: krytarowski
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D56009
M lib/sanitizer_common/sanitizer_common_interceptors.inc
M lib/sanitizer_common/sanitizer_platform_interceptors.h
M lib/sanitizer_common/sanitizer_platform_limits_freebsd.cc
M lib/sanitizer_common/sanitizer_platform_limits_freebsd.h
D test/sanitizer_common/TestCases/NetBSD/regex.cc
A + test/sanitizer_common/TestCases/Posix/regex.cc
llvm-svn: 350002
Summary:
NetBSD uses typical UNIX interfaces.
All tests pass except instrprof-dlopen-dlclose-gcov.test, as there
is not supported semantics of atexit(3) in dlopen(3)ed+dlclose(3)d
DSO.
NetBSD also ships an older version of LLVM profile (ABI v.2 predating
ABI v.4 in upstream version) inside libc. That copy has been manually
removed during the porting and testing process of the upstream version
to NetBSD. Otherwise there were conflicts between them two.
Reviewers: joerg, vitalybuka, vsk
Subscribers: srhines, fedor.sergeev, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55997
llvm-svn: 349994
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.
This is a follow up to r349957.
rdar://problem/45284065
llvm-svn: 349960
It should be at the class scope and not inside the `Init(...)` function
because we want to error out as soon as the wrong type is constructed.
At the function scope the `static_assert` is only checked if the
function might be called.
This is a follow up to r349138.
rdar://problem/45284065
llvm-svn: 349959
Summary:
This is a follow up to https://reviews.llvm.org/D55764 .
For the ASan and LSan allocatorsthe type declarations have been modified
so that it's possible to create a combined allocator type that
consistently uses a different type of `AddressSpaceView`. We intend to
use this in future patches. For the other sanitizers they just use
`LocalAddressSpaceView` by default because we have no plans to use these
allocators in an out-of-process manner.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov, yln
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55766
llvm-svn: 349957
Summary:
This is a follow up patch to r349138.
This patch makes a `AddressSpaceView` a type declaration in the
allocator parameters used by `SizeClassAllocator64`. For ASan, LSan, and
the unit tests the AP64 declarations have been made templated so that
`AddressSpaceView` can be changed at compile time. For the other
sanitizers we just hard-code `LocalAddressSpaceView` because we have no
plans to use these allocators in an out-of-process manner.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55764
llvm-svn: 349954
Add a code to properly test for presence of LLVMTestingSupport library
when performing a stand-alone build, and skip tests requiring it when
it is not present. Since the library is not installed, llvm-config
reported empty --libs for it and the tests failed to link with undefined
references. Skipping the two fdr_* test files is better than failing to
build, and should be good enough until we find a better solution.
NB: both installing LLVMTestingSupport and building it automatically
from within compiler-rt sources are non-trivial. The former due to
dependency on gtest, the latter due to tight integration with LLVM
source tree.
Differential Revision: https://reviews.llvm.org/D55891
llvm-svn: 349899
Summary:
This is a change requested by Vitaly Buka as prerequisite to landing
https://reviews.llvm.org/D55740.
Reviewers: vitalybuka, kubamracek
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55939
llvm-svn: 349897
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.
(2nd attempt, this time without the sanitizer_rtems change)
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55801
llvm-svn: 349817
This is patch complements D55117 implementing __hwasan_mem*
functions in runtime
Differential revision: https://reviews.llvm.org/D55554
llvm-svn: 349730
Summary:
Support running with no open file descriptors (as may happen to
"init" process on linux).
* Remove a check that writing to stderr succeeds.
* When opening a file (ex. for log_path option), dup the new fd out of
[0, 2] range to avoid confusing the program.
Reviewers: pcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55801
llvm-svn: 349699
Summary:
An extra ';' at the end of a namespace triggers a pedantic warning:
```
.../sanitizer_common/sanitizer_type_traits.h:42:2: warning: extra ‘;’ [-Wpedantic]
}; // namespace __sanitizer
```
Reviewers: eugenis, delcypher
Reviewed By: eugenis
Subscribers: kubamracek, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55849
llvm-svn: 349666
Fix the sha2 interceptor macros to use a constant for array parameter
length rather than referencing the extern variable. Since the digest
length is provided in hash name, reuse the macro parameter for it.
Verify that the calculated value matches the one provided by system
headers.
Differential Revision: https://reviews.llvm.org/D55811
llvm-svn: 349645
This patch aims to enable the tests for the compiler-rt builtin functions (that
currently already exist within compiler-rt) for PowerPC 64bit LE (ppc64le).
Previously when unit tests are run, these tests would be reported as
UNSUPPORTED. This patch updates the REQUIRES line for each test (to enable for
ppc64le), and each test is linked against compiler-rt when running.
Differential Revision: https://reviews.llvm.org/D54449
llvm-svn: 349634
Summary:
Thread Static Data cannot be used in early init on NetBSD
and FreeBSD. Reuse the ASan TSD API for compatibility with
existing code with an alternative implementation using Thread
Local Storage.
New version uses Thread Local Storage to store a pointer
with thread specific data. The destructor from TSD has been
replaced with a TLS destrucutor that is called upon thread
exit.
Reviewers: joerg, vitalybuka, jfb
Reviewed By: vitalybuka
Subscribers: dim, emaste, ro, jfb, devnexen, kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55596
llvm-svn: 349619
I tricked myself into thinking that deadlock detection is off by default in TSan by looking at the default value of the detect_deadlocks flag and outdated docs. (Created a pull request to update docs.)
I even managed to confuse others: https://groups.google.com/forum/#!topic/thread-sanitizer/xYvnAYwtoDk
However, the default value is overwritten in code (TSan_flags.cc:InitializeFlags). The TSan/deadlock tests also rely on this
This changes aligns the default value of the flag with the actual default behavior.
Author: yln (Julian Lettner)
Reviewed in: https://reviews.llvm.org/D55846
llvm-svn: 349609
Summary:
unnamed_addr is still useful for detecting of ODR violations on vtables
Still unnamed_addr with lld and --icf=safe or --icf=all can trigger false
reports which can be avoided with --icf=none or by using private aliases
with -fsanitize-address-use-odr-indicator
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: kubamracek, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D55799
llvm-svn: 349555
As of r349413 it's now possible for a binary to contain an empty
hwasan frame section. Handle that case simply by doing nothing.
Differential Revision: https://reviews.llvm.org/D55796
llvm-svn: 349428
Summary:
This fixes the `ThreadSanitizer-x86_64-iossim` testsuite which broke
when r348770 (https://reviews.llvm.org/D55473) landed.
The root cause of the problem is that early-on during the iOS simulator
init process a call to `sysctlbyname` is issued. If the TSan initializer
is triggered at this point it will eventually trigger a call to
`__cxa_at_exit(...)`. This call then aborts because the library
implementing this function is not yet had its initialization function
called.
rdar://problem/46696934
Reviewers: kubamracek, george.karpenkov, devnexen, vitalybuka, krytarowski
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55711
llvm-svn: 349402
Summary:
This is a follow up patch to r346956 for the `SizeClassAllocator32`
allocator.
This patch makes `AddressSpaceView` a template parameter both to the
`ByteMap` implementations (but makes `LocalAddressSpaceView` the
default), some `AP32` implementations and is used in `SizeClassAllocator32`.
The actual changes to `ByteMap` implementations and
`SizeClassAllocator32` are very simple. However the patch is large
because it requires changing all the `AP32` definitions, and users of
those definitions.
For ASan and LSan we make `AP32` and `ByteMap` templateds type that take
a single `AddressSpaceView` argument. This has been done because we will
instantiate the allocator with a type that isn't `LocalAddressSpaceView`
in the future patches. For the allocators used in the other sanitizers
(i.e. HWAsan, MSan, Scudo, and TSan) use of `LocalAddressSpaceView` is
hard coded because we do not intend to instantiate the allocators with
any other type.
In the cases where untemplated types have become templated on a single
`AddressSpaceView` parameter (e.g. `PrimaryAllocator`) their name has
been changed to have a `ASVT` suffix (Address Space View Type) to
indicate they are templated. The only exception to this are the `AP32`
types due to the desire to keep the type name as short as possible.
In order to check that template is instantiated in the correct a way a
`static_assert(...)` has been added that checks that the
`AddressSpaceView` type used by `Params::ByteMap::AddressSpaceView` matches
the `Params::AddressSpaceView`. This uses the new `sanitizer_type_traits.h`
header.
rdar://problem/45284065
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D54904
llvm-svn: 349138
Summary:
A recent update to the VS toolchain in chromium [1] broke the windows
ASan bot because the new toolchain calls _msize_base() instead of
_msize() in a number of _aligned_* UCRT routines. Instrument
_msize_base() as well.
[1] https://crbug.com/914947
Reviewers: rnk, #sanitizers, vitalybuka
Reviewed By: rnk, #sanitizers, vitalybuka
Subscribers: vitalybuka, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55684
llvm-svn: 349115
Summary:
In particular we implement the `is_same<T,U>` templated type. This is
useful for doing compile-time comparison of types in `static_assert`s.
The plan is to use this in another patch (
https://reviews.llvm.org/D54904 ).
Reviewers: kcc, dvyukov, vitalybuka, cryptoad, eugenis, kubamracek, george.karpenkov
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54951
llvm-svn: 349077
Summary:
private and internal: should not trigger ODR at all.
unnamed_addr: current ODR checking approach fail and rereport false violation if
a linker merges such globals
linkonce_odr, weak_odr: could cause similar problems and they are already not
instrumented for ELF.
Reviewers: eugenis, kcc
Subscribers: kubamracek, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D55621
llvm-svn: 349015
Summary:
The TLS_SLOT_TSAN slot is available starting in N, but its location (8)
is incompatible with the proposed solution for implementing ELF TLS on
Android (i.e. bump ARM/AArch64 alignment to reserve an 8-word TCB).
Instead, starting in Q, Bionic replaced TLS_SLOT_DLERROR(6) with
TLS_SLOT_SANITIZER(6). Switch compiler-rt to the new slot.
Reviewers: eugenis, srhines, enh
Reviewed By: eugenis
Subscribers: ruiu, srhines, kubamracek, javed.absar, kristof.beyls, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D55581
llvm-svn: 348984
Summary:
Add a check that TLS_SLOT_TSAN / TLS_SLOT_SANITIZER, whichever
android_get_tls_slot is using, is not conflicting with
TLS_SLOT_DLERROR.
Reviewers: rprichard, vitalybuka
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D55587
llvm-svn: 348979
Summary:
Fix bug where we'd try symbolize a second time with the same arguments even though symbolization failed the first time.
This looks like a long standing typo given that the guard for trying
symbolization again is to only try it if symbolization failed using
`binary` and `original_binary != binary`.
Reviewers: kubamracek, glider, samsonov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55504
llvm-svn: 348841
Summary:
Add new interceptors for a set of functions to reposition a stream:
fgetpos, fseek, fseeko, fsetpos, ftell, ftello, rewind .
Add a dedicated test.
Enable this interface on NetBSD.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55471
llvm-svn: 348743
- For the moment a subset of this api dealing with file descriptors permissions and ioctls.
Reviewers: vitalybuka, krytarowski
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D55368
llvm-svn: 348668
Summary:
strtoi/strtou converts string value to an intmax_t/uintmax_t integer.
Add a dedicated test.
Enable this API for NetBSD.
It's a reworked version of the original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, tomsun.0.7, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54702
llvm-svn: 348663
using `atos` symbolizer on Darwin when the binaries don't exist.
For now we just produce an unsymbolicated stackframe when the binary
doesn't exist.
llvm-svn: 348659
Summary:
fparseln - returns the next logical line from a stream.
Add a dedicated test for this API.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55019
llvm-svn: 348654
Summary:
strtonum(3) reliably convertss string value to an integer.
This function is used in OpenBSD compat namespace
and is located inside NetBSD's libc.
Add a dedicated test for this interface.
It's a reworked version of the original code by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54527
llvm-svn: 348651
Summary:
This change builds upon D54989, which removes memory allocation from the
critical path of the profiling implementation. This also changes the API
for the profile collection service, to take ownership of the memory and
associated data structures per-thread.
The consolidation of the memory allocation allows us to do two things:
- Limits the amount of memory used by the profiling implementation,
associating preallocated buffers instead of allocating memory
on-demand.
- Consolidate the memory initialisation and cleanup by relying on the
buffer queue's reference counting implementation.
We find a number of places which also display some problematic
behaviour, including:
- Off-by-factor bug in the allocator implementation.
- Unrolling semantics in cases of "memory exhausted" situations, when
managing the state of the function call trie.
We also add a few test cases which verify our understanding of the
behaviour of the system, with important edge-cases (especially for
memory-exhausted cases) in the segmented array and profile collector
unit tests.
Depends on D54989.
Reviewers: mboerger
Subscribers: dschuff, mgorny, dmgreen, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D55249
llvm-svn: 348568
This reverts commit r348455, with some additional changes:
- Work-around deficiency of gcc-4.8 by duplicating the implementation of
`AppendEmplace` in `Append`, but instead of using brace-init for the
copy construction, use a placement new explicitly calling the copy
constructor.
llvm-svn: 348563
Include CompilerRTCompile in fuzzer tests explicitly. Otherwise, when
building only libFuzzer, CMake fails due to:
CMake Error at cmake/Modules/AddCompilerRT.cmake:395 (sanitizer_test_compile):
Unknown CMake command "sanitizer_test_compile".
Call Stack (most recent call first):
lib/fuzzer/tests/CMakeLists.txt:53 (generate_compiler_rt_tests)
Differential Revision: https://reviews.llvm.org/D55378
llvm-svn: 348524
Summary:
The name of the function is based on `malloc_default_zone()` found
in Darwin's `malloc/malloc.h` header file.
Reviewers: kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55064
llvm-svn: 348481
This is a follow-up to D54989.
Work-around gcc-4.8 failing to handle brace-init for structs to imply
default-construction of an aggregate, and treats it as an initialiser
list instead.
llvm-svn: 348445
Continuation of D54989.
Additional changes:
- Use `.AppendEmplace(...)` instead of `.Append(Type{...})` to appease
GCC 4.8 with confusion on when an initializer_list is used as
opposed to a temporary aggregate initialized object.
llvm-svn: 348438
.. and also the follow-ups r348336 r348338.
It broke stand-alone compiler-rt builds with GCC 4.8:
In file included from /work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:20:0,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.h:21,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:15:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&}; T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’
/work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:517:54: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget’
new (AlignedOffset) T{std::forward<Args>(args)...};
^
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ThreadTrie&}; T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:98:34: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ThreadTrie&>((* & args#0))}’ from
‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ThreadTrie’
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ProfileBuffer&}; T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]
’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:244:44: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ProfileBuffer&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ProfileBuffer’
> Summary:
> This change makes the allocator and function call trie implementations
> move-aware and remove the FunctionCallTrie's reliance on a
> heap-allocated set of allocators.
>
> The change makes it possible to always have storage associated with
> Allocator instances, not necessarily having heap-allocated memory
> obtainable from these allocator instances. We also use thread-local
> uninitialised storage.
>
> We've also re-worked the segmented array implementation to have more
> precondition and post-condition checks when built in debug mode. This
> enables us to better implement some of the operations with surrounding
> documentation as well. The `trim` algorithm now has more documentation
> on the implementation, reducing the requirement to handle special
> conditions, and being more rigorous on the computations involved.
>
> In this change we also introduce an initialisation guard, through which
> we prevent an initialisation operation from racing with a cleanup
> operation.
>
> We also ensure that the ThreadTries array is not destroyed while copies
> into the elements are still being performed by other threads submitting
> profiles.
>
> Note that this change still has an issue with accessing thread-local
> storage from signal handlers that are instrumented with XRay. We also
> learn that with the testing of this patch, that there will be cases
> where calls to mmap(...) (through internal_mmap(...)) might be called in
> signal handlers, but are not async-signal-safe. Subsequent patches will
> address this, by re-using the `BufferQueue` type used in the FDR mode
> implementation for pre-allocated memory segments per active, tracing
> thread.
>
> We still want to land this change despite the known issues, with fixes
> forthcoming.
>
> Reviewers: mboerger, jfb
>
> Subscribers: jfb, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348346
Summary:
This change makes the allocator and function call trie implementations
move-aware and remove the FunctionCallTrie's reliance on a
heap-allocated set of allocators.
The change makes it possible to always have storage associated with
Allocator instances, not necessarily having heap-allocated memory
obtainable from these allocator instances. We also use thread-local
uninitialised storage.
We've also re-worked the segmented array implementation to have more
precondition and post-condition checks when built in debug mode. This
enables us to better implement some of the operations with surrounding
documentation as well. The `trim` algorithm now has more documentation
on the implementation, reducing the requirement to handle special
conditions, and being more rigorous on the computations involved.
In this change we also introduce an initialisation guard, through which
we prevent an initialisation operation from racing with a cleanup
operation.
We also ensure that the ThreadTries array is not destroyed while copies
into the elements are still being performed by other threads submitting
profiles.
Note that this change still has an issue with accessing thread-local
storage from signal handlers that are instrumented with XRay. We also
learn that with the testing of this patch, that there will be cases
where calls to mmap(...) (through internal_mmap(...)) might be called in
signal handlers, but are not async-signal-safe. Subsequent patches will
address this, by re-using the `BufferQueue` type used in the FDR mode
implementation for pre-allocated memory segments per active, tracing
thread.
We still want to land this change despite the known issues, with fixes
forthcoming.
Reviewers: mboerger, jfb
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348335
NetBSD 8.99.26 changed the layout of internal structure
returned by dlopen(3), switch to it.
Set new values for amd64 and i386 based on the results
of &((struct Struct_Obj_Entry*)0)->linkmap.
llvm-svn: 348329
Summary:
Flag was added for testing 3 years ago. Probably it's time
to simplify code and usage by removing it.
Reviewers: eugenis, m.ostapenko
Subscribers: mehdi_amini, kubamracek, steven_wu, dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D55254
llvm-svn: 348315
Summary:
Previously we weren't testing this function in the unit tests.
Reviewers: kcc, cryptoad, dvyukov, eugenis, kubamracek
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54861
llvm-svn: 348260
This finishes the soft-transition to the new primitive that implements
priority inheritance.
Differential Revision: https://reviews.llvm.org/D55244
llvm-svn: 348236
The res returned value might differ with REAL(strlen)(errbuf) + 1,
as the buffer's value is limited with errbuf_size.
Hot fix for D54584.
llvm-svn: 348231
Summary:
fts(3) is API to traverse a file hierarchy.
Cover this interface with interceptors.
Add a test to validate the interface reading
the number of regular files in /etc.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54247
llvm-svn: 348227
Make sure that symbols needed to implement runtime support for gcov are
exported when using an export list on Darwin.
Without the clang driver exporting these symbols, the linker hides them,
resulting in tapi verification failures.
rdar://45944768
Differential Revision: https://reviews.llvm.org/D55151
llvm-svn: 348187
Summary:
The purpose of this option is provide a way for the ASan dylib
to be loaded via `dlopen()` without triggering most initialization
steps (e.g. shadow memory set up) that normally occur when the
ASan dylib is loaded.
This new functionality is exposed by
- A `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` macro which indicates if the
feature is supported. This only true for Darwin currently.
- A `HandleDlopenInit()` function which should return true if the library
is being loaded via `dlopen()` and
`SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` is supported. Platforms that
support this may perform any initialization they wish inside this
function.
Although disabling initialization is something that could potentially
apply to other sanitizers it appears to be unnecessary for other
sanitizers so this patch only makes the change for ASan.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov, kcc, eugenis, krytarowski
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54469
llvm-svn: 348078
These changed as a result of r347379. Unfortunately there was a
regression; filed PR39748 to track it.
Differential Revision: https://reviews.llvm.org/D54821
llvm-svn: 347442
Summary:
D48660 / rL335762 added a `silence_unsigned_overflow` env flag for [[ https://github.com/google/oss-fuzz/pull/1717 | oss-fuzz needs ]],
that allows to silence the reports from unsigned overflows.
It makes sense, it is there because `-fsanitize=integer` sanitizer is not enabled on oss-fuzz,
so this allows to still use it as an interestingness signal, without getting the actual reports.
However there is a slight problem here.
All types of unsigned overflows are ignored.
Even if `-fno-sanitize-recover=unsigned` was used (which means the program will die after the report)
there will still be no report, the program will just silently die.
At the moment there are just two projects on oss-fuzz that care:
* [[ 8eeffa627f/projects/llvm_libcxx/build.sh (L18-L20) | libc++ ]]
* [[ 8eeffa627f/projects/librawspeed/build.sh | RawSpeed ]] (me)
I suppose this could be overridden there ^, but i really don't think this is intended behavior in any case..
Reviewers: kcc, Dor1s, #sanitizers, filcab, vsk, kubamracek
Reviewed By: Dor1s
Subscribers: dberris, mclow.lists, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54771
llvm-svn: 347415
Add pthread_tryjoin_np() and pthread_timedjoin_np() interceptors on Linux,
so that ThreadSanitizer can handle programs using these functions.
Author: Yuri Per (yuri)
Reviewed in: https://reviews.llvm.org/D54521
llvm-svn: 347383
Copy the fix for determining the correct terminfo library from LLVM --
use distinct variables for check_library_exists() calls. Otherwise,
the first check (for -ltinfo) populates the variable and no other checks
are performed. Effectively, systems with other libraries than the first
one listed are presumed not to have terminfo routines at all.
Also sync the check order to include the NetBSD fix from r347156.
This partially fixes undefined symbols when linking XRay tests. It's
probably not the best solution to the problem there but as long
as the terminfo check stays in config-ix, I thnk it's worth fixing.
Differential Revision: https://reviews.llvm.org/D54641
llvm-svn: 347338
Use a more representative test of allocating small chunks for
oddly-sized (small) objects from an allocator that has a page's worth of
memory.
llvm-svn: 347286
Summary:
This change addresses an issue which shows up with the synchronised race
between threads writing into a buffer, and another thread reading the
buffer.
In a lot of cases, we cannot guarantee that threads will always see the
signal to finalise their buffers in time despite the grace periods and
state machine maintained through atomic variables. This change addresses
it by ensuring that the same instance being updated to indicate how much
of the buffer is "used" by the writing thread is the same instance being
read by the thread processing the buffer to be written out to disk or
handled through the iterators.
To do this, we ensure that all the "extents" instances live in their own
the backing store, in a different contiguous page from the
buffer-specific backing store. We also take precautions to ensure that
the atomic variables are cache-line-sized to prevent false-sharing from
unnecessarily causing cache contention on unrelated writes/reads.
It's feasible that we may in the future be able to move the storage of
the extents objects into the single backing store, slightly changing the
way to compute the size(s) of the buffers, but in the meantime we'll
settle for the isolation afforded by having a different backing store
for the extents instances.
Reviewers: mboerger
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54684
llvm-svn: 347280
This change is part of the soft-transition to the new synchronization
primitives which implement priority inheritance.
Differential Revision: https://reviews.llvm.org/D54727
llvm-svn: 347279
Summary:
setvbuf(3) is a routine to setup stream buffering.
Enable the interceptor for NetBSD.
Add dedicated tests for setvbuf(3) and functions
on top of this interface: setbuf, setbuffer, setlinebuf.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: devnexen, tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54548
llvm-svn: 347270
Summary:
With free_checks_tail_magic=1 (default) HWASAN
writes magic bytes to the tail of every heap allocation
(last bytes of the last granule, if the last granule is not fully used)
and checks these bytes on free().
This feature will detect buffer overwires within the last granule
at the time of free().
This is an alternative to malloc_align_right=[1289] that should have
fewer compatibility issues. It is also weaker since it doesn't
detect read overflows and reports bugs at free() instead of at access.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54656
llvm-svn: 347116
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.
No functional change intended for others.
Cherry-picked chunk from D33878.
llvm-svn: 347102
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.
No functional change intended for others.
Cherry-picked chunk from D33878.
llvm-svn: 347101
Summary:
mi_vector_hash(3) provides fast 32bit hash functions.
Add a test for this interface.
Enable the API for NetBSD.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54530
llvm-svn: 347088
Summary:
getmntinfo gets information about mounted file systems.
Add a dedicated test for new interceptor.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54611
llvm-svn: 347083
Summary:
... so that we can find intra-granule buffer overflows.
The default is still to always align left.
It remains to be seen wether we can enable this mode at scale.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: jfb, dvyukov, kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53789
llvm-svn: 347082
When building for default target only, use exact target spelling
when deriving the name for the per-target runtime directory. This
is necessary for AArch32 where the CMake build by default rewrites
the architecture which leads to unexpected results.
Differential Revision: https://reviews.llvm.org/D54612
llvm-svn: 347022
enumeration.
Summary:
This patch introduces the local portion (`LocalAddressSpaceView`) of the
`AddressSpaceView` abstraction and modifies the secondary allocator
so that the `ForEachChunk()` method (and its callees) would work in the
out-of-process case when `AddressSpaceView` is `RemoteAddressSpaceView`.
The `AddressSpaceView` abstraction simply maps pointers from a target
process to a pointer in the local process (via its `Load()` method). For
the local (in-process) case this is a no-op. For the remote
(out-of-process) case this is not a no-op. The implementation of the
out-of-process `RemoteAddressSpaceView` is not included in this patch
and will be introduced later.
This patch is considerably simpler than the `ObjectView` abstraction
used in previous patches but lacks the type safety and stricter memory
management of the `ObjectView` abstraction.
This patch does not introduce any tests because with
`LocalAddressSpaceView` it should be a non functional change and unit
tests already cover the secondary allocator. When
`RemoteAddressSpaceView` is landed tests will be added to ensure that it
functions as expected.
rdar://problem/45284065
Reviewers: kcc, kubamracek, dvyukov, vitalybuka, cryptoad,
george.karpenkov, morehouse
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53975
llvm-svn: 346956
This is needed when cross-compiling for a different target since
CFLAGS may contain additional flags like -resource-dir which
change the location in which compiler-rt builtins are found.
Differential Revision: https://reviews.llvm.org/D54371
llvm-svn: 346820
After the change, the tests started failing, as skipped sections can be
equal in size to kMaxSegName.
Changing `<` to `<=` to address the off-by-one problem.
llvm-svn: 346804
Calling ARRAY_SIZE on a char* will not actually compute it's size, but just the pointer size.
A new Clang warning enabled by default warns about this.
Replaced the call with internal_strnlen.
Differential Revision: https://reviews.llvm.org/D54484
llvm-svn: 346792
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.
Changes:
- Split paths of handling intercepted __cxa_atexit() and atexit(3).
This affects all supported Operating Systems.
- Add a local stack-like structure to hold the __cxa_atexit() context.
atexit(3) is documented in the C standard as calling callback from the
earliest to the oldest entry. This path also fixes potential ABI
problem of passing an argument to a function from the atexit(3)
callback mechanism.
- Allow usage of global vars with ctors in interceptors.
This allows to use Vector without automatic cleaning up the structures.
This code has been modeled after TSan implementation for the same functions.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc
Reviewed By: vitalybuka
Subscribers: delcypher, devnexen, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40714
llvm-svn: 346579
This change was reverted because it caused some nacl tests in chromium
to fail. I attempted to reproduce those problems locally, but I was
unable to. Let's reland this and let Chromium's test infrastructure
discover any problems.
llvm-svn: 346560
Summary:
When reporting a fatal error, collect and add the entire report text to
android_set_abort_message so that it can be found in the tombstone.
Reviewers: kcc, vitalybuka
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D54284
llvm-svn: 346557
This change adds a static check to ensure that all data metadata record
payloads don't go past the available buffers in Metadata records.
llvm-svn: 346476
Summary:
Before this change, we could run into a situation where we may try to
undo tail exit records after writing metadata records before a function
enter event. This change rectifies that by resetting the tail exit
counter after writing the metadata records.
Reviewers: mboerger
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D54292
llvm-svn: 346475
Summary:
We need these fences to ensure that other threads attempting to read
bytes in the buffer will see thw writes committed before the extents are
updated. Without these, the writes can be un-committed by the time the
buffer extents counter is updated -- the fences should ensure that the
records written into the log have completed by the time we observe the
buffer extents from different threads.
Reviewers: mboerger
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54291
llvm-svn: 346474
Summary:
This change covers a number of things spanning LLVM and compiler-rt,
which are related in a non-trivial way.
In LLVM, we have a library that handles the FDR mode even log loading,
which uses C++'s runtime polymorphism feature to better faithfully
represent the events that are written down by the FDR mode runtime. We
do this by interpreting a trace that's serliased in a common format
agreed upon by both the trace loading library and the FDR mode runtime.
This library is under active development, which consists of features
allowing us to reconstitute a higher-level event log.
This event log is used by the conversion and visualisation tools we have
for interpreting XRay traces.
One of the tools we have is a diagnostic tool in llvm-xray called
`fdr-dump` which we've been using to debug our expectations of what the
FDR runtime should be writing and what the logical FDR event log
structures are. We use this fairly extensively to reason about why some
non-trivial traces we're generating with FDR mode runtimes fail to
convert or fail to parse correctly.
One of these failures we've found in manual debugging of some of the
traces we've seen involve an inconsistency between the buffer extents (a
record indicating how many bytes to follow are part of a logical
thread's event log) and the record of the bytes written into the log --
sometimes it turns out the data could be garbage, due to buffers being
recycled, but sometimes we're seeing the buffer extent indicating a log
is "shorter" than the actual records associated with the buffer. This
case happens particularly with function entry records with a call
argument.
This change for now updates the FDR mode runtime to write the bytes for
the function call and arg record before updating the buffer extents
atomically, allowing multiple threads to see a consistent view of the
data in the buffer using the atomic counter associated with a buffer.
What we're trying to prevent here is partial updates where we see the
intermediary updates to the buffer extents (function record size then
call argument record size) becoming observable from another thread, for
instance, one doing the serialization/flushing.
To do both diagnose this issue properly, we need to be able to honour
the extents being set in the `BufferExtents` records marking the
beginning of the logical buffers when reading an FDR trace. Since LLVM
doesn't use C++'s RTTI mechanism, we instead follow the advice in the
documentation for LLVM Style RTTI
(https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html). We then rely on
this RTTI feature to ensure that our file-based record producer (our
streaming "deserializer") can honour the extents of individual buffers
as we interpret traces.
This also sets us up to be able to eventually do smart
skipping/continuation of FDR logs, seeking instead to find BufferExtents
records in cases where we find potentially recoverable errors. In the
meantime, we make this change to operate in a strict mode when reading
logical buffers with extent records.
Reviewers: mboerger
Subscribers: hiraditya, llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D54201
llvm-svn: 346473
Summary:
Currently `sanitizer_malloc_introspection_t` just adds a version field
which is used to version the allocator ABI. The current allocator ABI
version is returned by the new `GetMallocZoneAllocatorEnumerationVersion()` function.
The motivation behind this change is to allow external processes to
determine the allocator ABI of a sanitized process.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54045
llvm-svn: 346420
Summary: Remove the XFAIL for arm since it seems to be ok
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: javed.absar, kristof.beyls, delcypher, chrib, llvm-commits, #sanitizers, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D54263
llvm-svn: 346409
Summary:
Introduce SANITIZER_INTERCEPT_LGAMMAL
dedicated for lgammal(). Disable it for NetBSD
as this routine is not implemented in this OS.
Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54054
llvm-svn: 346352
Summary:
Introduce SANITIZER_INTERCEPT_REMQUOL
dedicated for remquol(). Disable for for NetBSD
as this routine is not implemented in this OS.
Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54051
llvm-svn: 346351
We have seen failing builds due to a race condition between
RTAsan_dynamic and libc++ headers builds, specifically libc++
headers depend on __config and if this header hasn't been copied
into the final location, including other headers will typically
result in failure. To avoid this race, we add an explicit dependency
on libc++ headers which ensures that they've been copied into place
before the sanitizer object library build starts.
Differential Revision: https://reviews.llvm.org/D54198
llvm-svn: 346339
Summary: instrprof-gcov-fork.test is failing with arm so mark it as XFAIL
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: javed.absar, kristof.beyls, delcypher, chrib, llvm-commits, sylvestre.ledru, #sanitizers
Differential Revision: https://reviews.llvm.org/D54209
llvm-svn: 346329
Summary:
This change updates the version number for FDR logs to 5, and update the
trace processing to support changes in the custom event records.
In the runtime, since we're already writing down the record preamble to
handle CPU migrations and TSC wraparound, we can use the same TSC delta
encoding in the custom event and typed event records that we use in
function event records. We do the same change to typed events (which
were unsupported before this change in the trace processing) which now
show up in the trace.
Future changes should increase our testing coverage to make custom and
typed events as first class entities in the FDR mode log processing
tools.
This change is also a good example of how we end up supporting new
record types in the FDR mode implementation. This shows the places where
new record types are added and supported.
Depends on D54139.
Reviewers: mboerger
Subscribers: hiraditya, arphaman, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54140
llvm-svn: 346293
Summary: If user wants to raise it back, let them.
Reviewers: kcc, vitalybuka
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D54190
llvm-svn: 346284
Some weak external symbols were added to the profile runtime in D49953,
and on Darwin, these need to be exported for tapi verification purposes.
I've tightened the test so that future breakages can be caught earlier.
rdar://45831054
llvm-svn: 346276
This speeds up process startup and teardown and also reduces lock contention when running multiple ASanified/TSanified processes simultaneously. Should greatly improve lit testing time.
Differential Revision: https://reviews.llvm.org/D48445
llvm-svn: 346262
Summary:
The NetBSD headers use internal indirect type for
standard *int*_t definitions. The internal type is unrolled
inside the sanitizer into e.g. __int32_t from int32_t.
This symbol mangling causes pattern mismatch in
the interger truncation tests as they expect exact
types such as 'int32_t'.
Change the pattern rules so every acceptable internal
form of *int*_t will be accepted flawlessly.
Reviewers: lebedev.ri, vitalybuka, joerg
Reviewed By: lebedev.ri
Subscribers: kubamracek, dmgreen, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54150
llvm-svn: 346228
Summary:
For platforms without preinit support (such as NetBSD/amd64) the
initialization routine __xray_init() was called in non-deterministic order
compared to other constructors. This caused breakage failures
as xray routines attempted to execute code with assumption of
being initialized, which was no always true.
Use GCC/Clang extension to set maximal priority to the constructor
calling __xray_init(). This code switches away from C++ lambda form,
as it did not allow to specify this compiler extension.
Reviewers: dberris, joerg
Reviewed By: dberris
Subscribers: llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54136
llvm-svn: 346222
execve("/proc/self/exe") will not work if the binary relies on
$EXEC_ORIGIN in an rpath. Query AT_EXECFN instead, which will give the
same string that the current binary was exec'd with.
Differential Revision: https://reviews.llvm.org/D54113
llvm-svn: 346215
Summary:
This change cuts across LLVM and compiler-rt to add support for
rendering custom events in the XRayRecord type, to allow for including
user-provided annotations in the output YAML (as raw bytes).
This work enables us to add custom event and typed event records into
the `llvm::xray::Trace` type for user-provided events. This can then be
programmatically handled through the C++ API and can be included in some
of the tooling as well. For now we support printing the raw data we
encounter in the custom events in the converted output.
Future work will allow us to start interpreting these custom and typed
events through a yet-to-be-defined API for extending the trace analysis
library.
Reviewers: mboerger
Subscribers: hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D54139
llvm-svn: 346214
value-profile-cmp was always flaky, and OOM fails to work in some environments.
counters.test fails when vectorization is enabled, and it is more likely
to kick in when compiling for ARM.
Differential Revision: https://reviews.llvm.org/D54004
llvm-svn: 346193
While testing my to-be-submitted Solaris sanitizer support on gcc mainline, I ran into
an issue on Solaris/SPARC (sparc-sun-solaris2.11). Initially libasan.so failed to link:
Undefined first referenced
symbol in file
__sanitizer::BufferedStackTrace::FastUnwindStack(unsigned long, unsigned long, unsigned long, unsigned long, unsigned int) /var/gcc/gcc-9.0.0-20181024/11.5-gcc-gas/sparc-sun-solaris2.11/./libsanitizer/asan/.libs/libasan.so
This happens because SANITIZER_CAN_FAST_UNWIND is enabled on non-Linux
SPARC targets (cf. sanitizer_stacktrace.h), but the guard around the SPARCv8-only
definition in sanitizer_stacktrace_sparc.cc only works with clang:
clang predefines __sparcv8__ on non-Solaris, and __sparcv8 only on Solaris
gcc predefines __sparcv8 on Solaris, but __sparc_v8__ on non-Solaris
The attached patch allows for all three variants.
However, disabling SANITIZER_CAN_FAST_UNWIND on all SPARC targets
fixes a couple of testsuite failures in the Solaris asan testsuite, so for now it's better
to keep it disabled everywhere.
This allowed the libsanitizer build to complete and gave reasonable (though slightly
worse than on Solaris/x86) testsuite results.
Differential Revision: https://reviews.llvm.org/D54099
llvm-svn: 346155
While testing the Solaris libsanitizer port on GCC mainline, I found that
I'd messed up
the largefile checks in various ways, some of which showed as compile failures
(wrong structure sizes and member offsets), others at runtime, some of those only
on sparc as a big-endian target.
This patch fixes all of them:
- OFF_T is now correctly defined for 32-bit largefile and traditional
environments, and 64-bit.
- The definition of __sanitizer_dirent now checks the correct conditionals.
- sanitizer_procmaps_solaris.cc undefines _FILE_OFFSET_BITS: before
Solaris 11.4 <procfs.h> doesn't even compile with largefile support
enabled, but the use at hand doesn't need it anyway while g++ 9 will
define _FILE_OFFSET_BITS=64 out of the box.
- With full largefile support enabled, one needs to use e.g. mmap64
instead of mmap; this is hidden behind macros.
With this patch I could bootstrap gcc mainline on both sparc-sun-solaris2.11 and
i386-pc-solaris2.11. In addition, I've successfully built llvm on
i386-pc-solaris2.11.
Differential Revision: https://reviews.llvm.org/D54101
llvm-svn: 346153
Summary:
Prior to this change, we can run into situations where the TSC we're
getting when exiting a function is less than the TSC we got when
entering it. This would sometimes cause the counter for cumulative call
times overflow, which was erroneously also being stored as a signed
64-bit integer.
This change addresses both these issues while adding provisions for
tracking CPU migrations. We do this because moving from one CPU to
another doesn't guarantee that the timestamp counter for some
architectures aren't guaranteed to be synchronised. For the moment, we
leave the provisions there until we can update the data format to
include the counting of CPU migrations we can catch.
We update the necessary tests as well, ensuring that our expectations
for the cycle accounting to be met in case of counter wraparound.
Reviewers: mboerger
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D54088
llvm-svn: 346116
This fixes the issue introduced in r345765 which changed the way in
which the embedded libc++ is being built but omitted tests.
Differential Revision: https://reviews.llvm.org/D54058
llvm-svn: 346052
We're using an old NDK and a new Clang. New Clangs default to
`-stdlib=libc++` for Android, but those libraries cannot be found by
default with an old NDK. Use an explicit `-stdlib=libstdc++` in the
cxx_mode_flags.
llvm-svn: 346051
Include the build of unpoison_passwd() and unpoison_group() for
SANITIZER_INTERCEPT_FGETPWENT_R and SANITIZER_INTERCEPT_FGETGRENT_R.
static void unpoison_passwd(
llvm-svn: 346042
Summary:
NetBSD does not ship with fgetpwent_r() and fgetgrent_r().
Split their interceptors from getpwent_r() and getgrent_r()
and disable for this OS.
Installation of supernumerary interceptors causes leaking of
errors to dlsym(3)-like operations.
No functional change for other OSes.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: srhines, kubamracek, fedor.sergeev, llvm-commits, #sanitizers, mgorny
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54041
llvm-svn: 346038
Recent versions of Ubuntu (17.04 and 18.04) on PowerPC have introduced changes
to Address Space Layout Randomization (ASLR) that is causing 500+ sanitizer
failures. This patch disables ASLR when running the sanitizers on PowerPC 64bit
LE.
Differential Revision: https://reviews.llvm.org/D52900
llvm-svn: 346030
Summary:
The motivation here is to be able support initialisation of the
malloc zone on other code paths.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov
Subscribers: llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D54037
llvm-svn: 345983
Summary:
Fix some issues discovered from mostly manual inspection of outputs from
the `llvm-xray fdr-dump` tool.
It turns out we haven't been writing the deltas properly, and have been
writing down zeros for deltas of some records. This change fixes this
oversight born by the recent refactoring.
Reviewers: mboerger
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D54022
llvm-svn: 345954
Summary:
This is a follow-on change to D53858 which turns out to have had a TSC
accounting bug when writing out function exit records in FDR mode.
This change adds a number of tests to ensure that:
- We are handling the delta between the exit TSC and the last TSC we've
seen.
- We are writing the custom event and typed event records as a single
update to the buffer extents.
- We are able to catch boundary conditions when loading FDR logs.
We introduce a TSC matcher to the test helpers, which we use in the
testing/verification of the TSC accounting change.
Reviewers: mboerger
Subscribers: mgorny, hiraditya, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53967
llvm-svn: 345905
Summary:
This change cuts across compiler-rt and llvm, to increment the FDR log
version number to 4, and include the CPU ID in the custom event records.
This is a step towards allowing us to change the `llvm::xray::Trace`
object to start representing both custom and typed events in the stream
of records. Follow-on changes will allow us to change the kinds of
records we're presenting in the stream of traces, to incorporate the
data in custom/typed events.
A follow-on change will handle the typed event case, where it may not
fit within the 15-byte buffer for metadata records.
This work is part of the larger effort to enable writing analysis and
processing tools using a common in-memory representation of the events
found in traces. The work will focus on porting existing tools in LLVM
to use the common representation and informing the design of a
library/framework for expressing trace event analysis as C++ programs.
Reviewers: mboerger, eizan
Subscribers: hiraditya, mgrang, llvm-commits
Differential Revision: https://reviews.llvm.org/D53920
llvm-svn: 345798
Summary:
Windows has always been LLP64, not LP64, so the macros were incorrect.
Check for _WIN64, since AArch64 sets that too.
The tests have to be fixed up in two main ways:
1. Use 'ULL' suffixes to avoid sign extension when passing hex literals
with the sign bit set to signed 128 arguments. Clang
-fms-compatibility makes such hex literals signed, not unsigned.
2. Disable various tests for 80-bit long double interoperation with i128
values.
Reviewers: joerg, mstorsjo
Subscribers: javed.absar, kristof.beyls, hiraditya, aheejin, llvm-commits
Differential Revision: https://reviews.llvm.org/D53918
llvm-svn: 345796
When building the custom libc++ that's used by libFuzzer as well as
MSan and TSan tests, passthrough the C and C++ flags that were passed
to the compiler-rt CMake build. These may be needed to successfuly
compile the library on a particular platform.
Differential Revision: https://reviews.llvm.org/D53862
llvm-svn: 345788
libc++ now supports customizing the ABI namespace directly from the
CMake build so we no longer need to rely on custom CFLAGS.
Differential Revision: https://reviews.llvm.org/D53939
llvm-svn: 345765
Eliminate dropped operations, add new operations.
Update included headers for newer need.
Add a fallback definition of nvlist_ref_t, becaue this type
is internal to libnpf and the kernel, not exported into public
headers.
llvm-svn: 345726
Summary:
In r81552, the HAS_80_BIT_LONG_DOUBLE macro was added to the unit test
only version of int_lib.h. One month later in r85260 the duplicate
int_lib.h was removed, but the tests still passed because we don't build
with -Werror.
This is the minimal change to bring it back, and I decided to put the
configuration macro next to our 128-bit integer support macro.
Reviewers: joerg, compnerd, mstorsjo
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53838
llvm-svn: 345645
When -ffreestanding is used, main() isn't considered special and
when compiled as C++ code it'll get mangled which makes the
compilation fail since main() will be undefined so this check will
never succeed.
Differential Revision: https://reviews.llvm.org/D49332
llvm-svn: 345632
Summary:
Add a new helper macro TSAN_INTERCEPTOR_NETBSD_ALIAS_THR2
that handles pthread(3)/libc aliases in scenarios when a name in both
libraries differs not just in prefix namespace.
Handle TSan pthread_sigmask mangling accordingly into __libc_thr_sigsetmask.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: kubamracek, jfb, llvm-commits, #sanitizers, mgorny
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D53863
llvm-svn: 345627
Summary: Add a test for coverage for function definition like void foo() { }.
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: delcypher, llvm-commits, #sanitizers, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D53601
llvm-svn: 345625
Sort the headers more correctly according to NetBSD style.
Prevent in this code part clang-format, as shuffling the order
will cause build failures.
llvm-svn: 345586
Register new syscall getsockopt2.
Drop removed syscalls pmc_get_info and pmc_control.
While there address compiler warnings about potentially
unused variables.
llvm-svn: 345582
Register new syscall getsockopt2.
Drop removed syscalls pmc_get_info and pmc_control.
While there address compiler warnings about potentially
unused variables.
llvm-svn: 345580
Summary:
This change completes the refactoring of the FDR runtime to support the
following:
- Generational buffer management.
- Centralised and well-tested controller implementation.
In this change we've had to:
- Greatly simplify the code in xray_fdr_logging.cc to only implement the
glue code for calling into the controller.
- Implement the custom and typed event logging functions in the
FDRLogWriter.
- Imbue the `XRAY_NEVER_INSTRUMENT` attribute onto all functions in the
controller implementation.
Reviewers: mboerger, eizan, jfb
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53858
llvm-svn: 345568
Summary:
Prepend minimal when UBsan is mentioned and delete a dead comment
Patch by Igor Sugak
Reviewers: eugenis, pcc
Reviewed By: eugenis
Subscribers: mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D53826
llvm-svn: 345557
This breaks some configurations, so just forward declare everything that we need.
rdar://35943793
Differential revision: https://reviews.llvm.org/D50269
llvm-svn: 345551
Revert older change that was incorrect in this test.
It was already reverted in the past after an attempt to port it to Darwin.
While there, mark FreeBSD as unsupported as well.
llvm-svn: 345492
Summary:
Some cases where `postCurrentThreadFCT()` are not guarded by our
recursion guard. We've observed that sometimes these can lead to
deadlocks when some functions (like memcpy()) gets outlined and the
version of memcpy is XRay-instrumented, which can be materialised by the
compiler in the implementation of lower-level components used by the
profiling runtime.
This change ensures that all calls to `postCurrentThreadFCT` are guarded
by our thread-recursion guard, to prevent deadlocks.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53805
llvm-svn: 345489
Summary:
In D53560, we assumed a specific layout for memory without using an
explicit structure. This follow-up change uses more portable layout
control by using unions in a struct, and consolidating the memory
management code in the buffer queue.
We also take the opportunity to improve the documentation on the types
and operations, along with simplifying some of the logic in the buffer
queue implementation.
Reviewers: mboerger, eizan
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53802
llvm-svn: 345485
Summary:
This change implements the ref-counting for backing stores associated
with generational buffer management. We do this as an implementation
detail of the buffer queue, instead of exposing this to the interface.
This change allows us to keep the buffer queue interface and usage model
the same.
Depends on D53551.
Reviewers: mboerger, eizan
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53560
llvm-svn: 345471
Summary:
This is an intermediary step in the full support for generational buffer
management in the FDR runtime. This change makes the FDR controller
aware of the new generation number in the buffers handed out by the
BufferQueue type.
In the process of making this change, we've realised that the cleanest
way of ensuring that the backing store per generation is live while all
the threads that need access to it will need reference counting to tie
the backing store to the lifetime of all threads that have a handle on
buffers associated with the memory.
We also learn that we're missing the edge-case in the function exit
handler's implementation where the first record being written into the
buffer is a function exit, which is caught/fixed by the test for
generational buffer management.
We still haven't wired the controller into the FDR mode runtime, which
will need the reference counting on the backing store implemented to
ensure that we're being conservatively thread-safe with this approach.
Depends on D52974.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53551
llvm-svn: 345445
There's a TSan startup crash on Linux when used in Swift programs, where MonotonicNanoTime will try to call real_clock_gettime and then jump to NULL because interceptors are not yet initialized. This is on Ubuntu 18.04. Looks like TSan's main Initialize() function is called at a point where __progname is already set, but interceptors aren't yet set up. Let's fix this by checking whether interceptors are initialized in MonotonicNanoTime.
Differential Revision: https://reviews.llvm.org/D53528
llvm-svn: 345174
Summary:
At compile-time, create an array of {PC,HumanReadableStackFrameDescription}
for every function that has an instrumented frame, and pass this array
to the run-time at the module-init time.
Similar to how we handle pc-table in SanitizerCoverage.
The run-time is dummy, will add the actual logic in later commits.
Reviewers: morehouse, eugenis
Reviewed By: eugenis
Subscribers: srhines, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D53227
llvm-svn: 344985
David Carlier