Summary:
fparseln - returns the next logical line from a stream.
Add a dedicated test for this API.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55019
llvm-svn: 348654
Summary:
strtonum(3) reliably convertss string value to an integer.
This function is used in OpenBSD compat namespace
and is located inside NetBSD's libc.
Add a dedicated test for this interface.
It's a reworked version of the original code by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54527
llvm-svn: 348651
Summary:
This change builds upon D54989, which removes memory allocation from the
critical path of the profiling implementation. This also changes the API
for the profile collection service, to take ownership of the memory and
associated data structures per-thread.
The consolidation of the memory allocation allows us to do two things:
- Limits the amount of memory used by the profiling implementation,
associating preallocated buffers instead of allocating memory
on-demand.
- Consolidate the memory initialisation and cleanup by relying on the
buffer queue's reference counting implementation.
We find a number of places which also display some problematic
behaviour, including:
- Off-by-factor bug in the allocator implementation.
- Unrolling semantics in cases of "memory exhausted" situations, when
managing the state of the function call trie.
We also add a few test cases which verify our understanding of the
behaviour of the system, with important edge-cases (especially for
memory-exhausted cases) in the segmented array and profile collector
unit tests.
Depends on D54989.
Reviewers: mboerger
Subscribers: dschuff, mgorny, dmgreen, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D55249
llvm-svn: 348568
This reverts commit r348455, with some additional changes:
- Work-around deficiency of gcc-4.8 by duplicating the implementation of
`AppendEmplace` in `Append`, but instead of using brace-init for the
copy construction, use a placement new explicitly calling the copy
constructor.
llvm-svn: 348563
Include CompilerRTCompile in fuzzer tests explicitly. Otherwise, when
building only libFuzzer, CMake fails due to:
CMake Error at cmake/Modules/AddCompilerRT.cmake:395 (sanitizer_test_compile):
Unknown CMake command "sanitizer_test_compile".
Call Stack (most recent call first):
lib/fuzzer/tests/CMakeLists.txt:53 (generate_compiler_rt_tests)
Differential Revision: https://reviews.llvm.org/D55378
llvm-svn: 348524
Summary:
The name of the function is based on `malloc_default_zone()` found
in Darwin's `malloc/malloc.h` header file.
Reviewers: kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D55064
llvm-svn: 348481
This is a follow-up to D54989.
Work-around gcc-4.8 failing to handle brace-init for structs to imply
default-construction of an aggregate, and treats it as an initialiser
list instead.
llvm-svn: 348445
Continuation of D54989.
Additional changes:
- Use `.AppendEmplace(...)` instead of `.Append(Type{...})` to appease
GCC 4.8 with confusion on when an initializer_list is used as
opposed to a temporary aggregate initialized object.
llvm-svn: 348438
.. and also the follow-ups r348336 r348338.
It broke stand-alone compiler-rt builds with GCC 4.8:
In file included from /work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:20:0,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.h:21,
from /work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:15:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&}; T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget]’
/work/llvm/projects/compiler-rt/lib/xray/xray_function_call_trie.h:517:54: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::FunctionCallTrie::mergeInto(__xray::FunctionCallTrie&) const::NodeAndTarget’
new (AlignedOffset) T{std::forward<Args>(args)...};
^
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ThreadTrie&}; T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ThreadTrie]’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:98:34: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ThreadTrie&>((* & args#0))}’ from
‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ThreadTrie’
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h: In instantiation of ‘T* __xray::Array<T>::AppendEmplace(Args&& ...) [with Args = {const __xray::profileCollectorService::{anonymous}::ProfileBuffer&}; T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]’:
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:383:71: required from ‘T* __xray::Array<T>::Append(const T&) [with T = __xray::profileCollectorService::{anonymous}::ProfileBuffer]
’
/work/llvm/projects/compiler-rt/lib/xray/xray_profile_collector.cc:244:44: required from here
/work/llvm/projects/compiler-rt/lib/xray/xray_segmented_array.h:378:5: error: could not convert ‘{std::forward<const __xray::profileCollectorService::{anonymous}::ProfileBuffer&>((* & args#0))}’ from ‘<brace-enclosed initializer list>’ to ‘__xray::profileCollectorService::{anonymous}::ProfileBuffer’
> Summary:
> This change makes the allocator and function call trie implementations
> move-aware and remove the FunctionCallTrie's reliance on a
> heap-allocated set of allocators.
>
> The change makes it possible to always have storage associated with
> Allocator instances, not necessarily having heap-allocated memory
> obtainable from these allocator instances. We also use thread-local
> uninitialised storage.
>
> We've also re-worked the segmented array implementation to have more
> precondition and post-condition checks when built in debug mode. This
> enables us to better implement some of the operations with surrounding
> documentation as well. The `trim` algorithm now has more documentation
> on the implementation, reducing the requirement to handle special
> conditions, and being more rigorous on the computations involved.
>
> In this change we also introduce an initialisation guard, through which
> we prevent an initialisation operation from racing with a cleanup
> operation.
>
> We also ensure that the ThreadTries array is not destroyed while copies
> into the elements are still being performed by other threads submitting
> profiles.
>
> Note that this change still has an issue with accessing thread-local
> storage from signal handlers that are instrumented with XRay. We also
> learn that with the testing of this patch, that there will be cases
> where calls to mmap(...) (through internal_mmap(...)) might be called in
> signal handlers, but are not async-signal-safe. Subsequent patches will
> address this, by re-using the `BufferQueue` type used in the FDR mode
> implementation for pre-allocated memory segments per active, tracing
> thread.
>
> We still want to land this change despite the known issues, with fixes
> forthcoming.
>
> Reviewers: mboerger, jfb
>
> Subscribers: jfb, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348346
Summary:
This change makes the allocator and function call trie implementations
move-aware and remove the FunctionCallTrie's reliance on a
heap-allocated set of allocators.
The change makes it possible to always have storage associated with
Allocator instances, not necessarily having heap-allocated memory
obtainable from these allocator instances. We also use thread-local
uninitialised storage.
We've also re-worked the segmented array implementation to have more
precondition and post-condition checks when built in debug mode. This
enables us to better implement some of the operations with surrounding
documentation as well. The `trim` algorithm now has more documentation
on the implementation, reducing the requirement to handle special
conditions, and being more rigorous on the computations involved.
In this change we also introduce an initialisation guard, through which
we prevent an initialisation operation from racing with a cleanup
operation.
We also ensure that the ThreadTries array is not destroyed while copies
into the elements are still being performed by other threads submitting
profiles.
Note that this change still has an issue with accessing thread-local
storage from signal handlers that are instrumented with XRay. We also
learn that with the testing of this patch, that there will be cases
where calls to mmap(...) (through internal_mmap(...)) might be called in
signal handlers, but are not async-signal-safe. Subsequent patches will
address this, by re-using the `BufferQueue` type used in the FDR mode
implementation for pre-allocated memory segments per active, tracing
thread.
We still want to land this change despite the known issues, with fixes
forthcoming.
Reviewers: mboerger, jfb
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54989
llvm-svn: 348335
NetBSD 8.99.26 changed the layout of internal structure
returned by dlopen(3), switch to it.
Set new values for amd64 and i386 based on the results
of &((struct Struct_Obj_Entry*)0)->linkmap.
llvm-svn: 348329
Summary:
Flag was added for testing 3 years ago. Probably it's time
to simplify code and usage by removing it.
Reviewers: eugenis, m.ostapenko
Subscribers: mehdi_amini, kubamracek, steven_wu, dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D55254
llvm-svn: 348315
Summary:
Previously we weren't testing this function in the unit tests.
Reviewers: kcc, cryptoad, dvyukov, eugenis, kubamracek
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54861
llvm-svn: 348260
This finishes the soft-transition to the new primitive that implements
priority inheritance.
Differential Revision: https://reviews.llvm.org/D55244
llvm-svn: 348236
The res returned value might differ with REAL(strlen)(errbuf) + 1,
as the buffer's value is limited with errbuf_size.
Hot fix for D54584.
llvm-svn: 348231
Summary:
fts(3) is API to traverse a file hierarchy.
Cover this interface with interceptors.
Add a test to validate the interface reading
the number of regular files in /etc.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54247
llvm-svn: 348227
Summary:
The purpose of this option is provide a way for the ASan dylib
to be loaded via `dlopen()` without triggering most initialization
steps (e.g. shadow memory set up) that normally occur when the
ASan dylib is loaded.
This new functionality is exposed by
- A `SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` macro which indicates if the
feature is supported. This only true for Darwin currently.
- A `HandleDlopenInit()` function which should return true if the library
is being loaded via `dlopen()` and
`SANITIZER_SUPPORTS_INIT_FOR_DLOPEN` is supported. Platforms that
support this may perform any initialization they wish inside this
function.
Although disabling initialization is something that could potentially
apply to other sanitizers it appears to be unnecessary for other
sanitizers so this patch only makes the change for ASan.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov, kcc, eugenis, krytarowski
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54469
llvm-svn: 348078
These changed as a result of r347379. Unfortunately there was a
regression; filed PR39748 to track it.
Differential Revision: https://reviews.llvm.org/D54821
llvm-svn: 347442
Summary:
D48660 / rL335762 added a `silence_unsigned_overflow` env flag for [[ https://github.com/google/oss-fuzz/pull/1717 | oss-fuzz needs ]],
that allows to silence the reports from unsigned overflows.
It makes sense, it is there because `-fsanitize=integer` sanitizer is not enabled on oss-fuzz,
so this allows to still use it as an interestingness signal, without getting the actual reports.
However there is a slight problem here.
All types of unsigned overflows are ignored.
Even if `-fno-sanitize-recover=unsigned` was used (which means the program will die after the report)
there will still be no report, the program will just silently die.
At the moment there are just two projects on oss-fuzz that care:
* [[ 8eeffa627f/projects/llvm_libcxx/build.sh (L18-L20) | libc++ ]]
* [[ 8eeffa627f/projects/librawspeed/build.sh | RawSpeed ]] (me)
I suppose this could be overridden there ^, but i really don't think this is intended behavior in any case..
Reviewers: kcc, Dor1s, #sanitizers, filcab, vsk, kubamracek
Reviewed By: Dor1s
Subscribers: dberris, mclow.lists, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54771
llvm-svn: 347415
Add pthread_tryjoin_np() and pthread_timedjoin_np() interceptors on Linux,
so that ThreadSanitizer can handle programs using these functions.
Author: Yuri Per (yuri)
Reviewed in: https://reviews.llvm.org/D54521
llvm-svn: 347383
Use a more representative test of allocating small chunks for
oddly-sized (small) objects from an allocator that has a page's worth of
memory.
llvm-svn: 347286
Summary:
This change addresses an issue which shows up with the synchronised race
between threads writing into a buffer, and another thread reading the
buffer.
In a lot of cases, we cannot guarantee that threads will always see the
signal to finalise their buffers in time despite the grace periods and
state machine maintained through atomic variables. This change addresses
it by ensuring that the same instance being updated to indicate how much
of the buffer is "used" by the writing thread is the same instance being
read by the thread processing the buffer to be written out to disk or
handled through the iterators.
To do this, we ensure that all the "extents" instances live in their own
the backing store, in a different contiguous page from the
buffer-specific backing store. We also take precautions to ensure that
the atomic variables are cache-line-sized to prevent false-sharing from
unnecessarily causing cache contention on unrelated writes/reads.
It's feasible that we may in the future be able to move the storage of
the extents objects into the single backing store, slightly changing the
way to compute the size(s) of the buffers, but in the meantime we'll
settle for the isolation afforded by having a different backing store
for the extents instances.
Reviewers: mboerger
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54684
llvm-svn: 347280
This change is part of the soft-transition to the new synchronization
primitives which implement priority inheritance.
Differential Revision: https://reviews.llvm.org/D54727
llvm-svn: 347279
Summary:
setvbuf(3) is a routine to setup stream buffering.
Enable the interceptor for NetBSD.
Add dedicated tests for setvbuf(3) and functions
on top of this interface: setbuf, setbuffer, setlinebuf.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: devnexen, tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54548
llvm-svn: 347270
Summary:
With free_checks_tail_magic=1 (default) HWASAN
writes magic bytes to the tail of every heap allocation
(last bytes of the last granule, if the last granule is not fully used)
and checks these bytes on free().
This feature will detect buffer overwires within the last granule
at the time of free().
This is an alternative to malloc_align_right=[1289] that should have
fewer compatibility issues. It is also weaker since it doesn't
detect read overflows and reports bugs at free() instead of at access.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54656
llvm-svn: 347116
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.
No functional change intended for others.
Cherry-picked chunk from D33878.
llvm-svn: 347102
This modification is require for NetBSD with GCC, as there is a
custom unwind.h header implementation with different types.
No functional change intended for others.
Cherry-picked chunk from D33878.
llvm-svn: 347101
Summary:
mi_vector_hash(3) provides fast 32bit hash functions.
Add a test for this interface.
Enable the API for NetBSD.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54530
llvm-svn: 347088
Summary:
getmntinfo gets information about mounted file systems.
Add a dedicated test for new interceptor.
Based on original work by Yang Zheng.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: tomsun.0.7, kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54611
llvm-svn: 347083
Summary:
... so that we can find intra-granule buffer overflows.
The default is still to always align left.
It remains to be seen wether we can enable this mode at scale.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: jfb, dvyukov, kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53789
llvm-svn: 347082
enumeration.
Summary:
This patch introduces the local portion (`LocalAddressSpaceView`) of the
`AddressSpaceView` abstraction and modifies the secondary allocator
so that the `ForEachChunk()` method (and its callees) would work in the
out-of-process case when `AddressSpaceView` is `RemoteAddressSpaceView`.
The `AddressSpaceView` abstraction simply maps pointers from a target
process to a pointer in the local process (via its `Load()` method). For
the local (in-process) case this is a no-op. For the remote
(out-of-process) case this is not a no-op. The implementation of the
out-of-process `RemoteAddressSpaceView` is not included in this patch
and will be introduced later.
This patch is considerably simpler than the `ObjectView` abstraction
used in previous patches but lacks the type safety and stricter memory
management of the `ObjectView` abstraction.
This patch does not introduce any tests because with
`LocalAddressSpaceView` it should be a non functional change and unit
tests already cover the secondary allocator. When
`RemoteAddressSpaceView` is landed tests will be added to ensure that it
functions as expected.
rdar://problem/45284065
Reviewers: kcc, kubamracek, dvyukov, vitalybuka, cryptoad,
george.karpenkov, morehouse
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53975
llvm-svn: 346956
After the change, the tests started failing, as skipped sections can be
equal in size to kMaxSegName.
Changing `<` to `<=` to address the off-by-one problem.
llvm-svn: 346804
Calling ARRAY_SIZE on a char* will not actually compute it's size, but just the pointer size.
A new Clang warning enabled by default warns about this.
Replaced the call with internal_strnlen.
Differential Revision: https://reviews.llvm.org/D54484
llvm-svn: 346792
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.
Changes:
- Split paths of handling intercepted __cxa_atexit() and atexit(3).
This affects all supported Operating Systems.
- Add a local stack-like structure to hold the __cxa_atexit() context.
atexit(3) is documented in the C standard as calling callback from the
earliest to the oldest entry. This path also fixes potential ABI
problem of passing an argument to a function from the atexit(3)
callback mechanism.
- Allow usage of global vars with ctors in interceptors.
This allows to use Vector without automatic cleaning up the structures.
This code has been modeled after TSan implementation for the same functions.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc
Reviewed By: vitalybuka
Subscribers: delcypher, devnexen, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40714
llvm-svn: 346579
This change was reverted because it caused some nacl tests in chromium
to fail. I attempted to reproduce those problems locally, but I was
unable to. Let's reland this and let Chromium's test infrastructure
discover any problems.
llvm-svn: 346560
Summary:
When reporting a fatal error, collect and add the entire report text to
android_set_abort_message so that it can be found in the tombstone.
Reviewers: kcc, vitalybuka
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D54284
llvm-svn: 346557
This change adds a static check to ensure that all data metadata record
payloads don't go past the available buffers in Metadata records.
llvm-svn: 346476
Summary:
Before this change, we could run into a situation where we may try to
undo tail exit records after writing metadata records before a function
enter event. This change rectifies that by resetting the tail exit
counter after writing the metadata records.
Reviewers: mboerger
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D54292
llvm-svn: 346475
Summary:
We need these fences to ensure that other threads attempting to read
bytes in the buffer will see thw writes committed before the extents are
updated. Without these, the writes can be un-committed by the time the
buffer extents counter is updated -- the fences should ensure that the
records written into the log have completed by the time we observe the
buffer extents from different threads.
Reviewers: mboerger
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54291
llvm-svn: 346474
Summary:
This change covers a number of things spanning LLVM and compiler-rt,
which are related in a non-trivial way.
In LLVM, we have a library that handles the FDR mode even log loading,
which uses C++'s runtime polymorphism feature to better faithfully
represent the events that are written down by the FDR mode runtime. We
do this by interpreting a trace that's serliased in a common format
agreed upon by both the trace loading library and the FDR mode runtime.
This library is under active development, which consists of features
allowing us to reconstitute a higher-level event log.
This event log is used by the conversion and visualisation tools we have
for interpreting XRay traces.
One of the tools we have is a diagnostic tool in llvm-xray called
`fdr-dump` which we've been using to debug our expectations of what the
FDR runtime should be writing and what the logical FDR event log
structures are. We use this fairly extensively to reason about why some
non-trivial traces we're generating with FDR mode runtimes fail to
convert or fail to parse correctly.
One of these failures we've found in manual debugging of some of the
traces we've seen involve an inconsistency between the buffer extents (a
record indicating how many bytes to follow are part of a logical
thread's event log) and the record of the bytes written into the log --
sometimes it turns out the data could be garbage, due to buffers being
recycled, but sometimes we're seeing the buffer extent indicating a log
is "shorter" than the actual records associated with the buffer. This
case happens particularly with function entry records with a call
argument.
This change for now updates the FDR mode runtime to write the bytes for
the function call and arg record before updating the buffer extents
atomically, allowing multiple threads to see a consistent view of the
data in the buffer using the atomic counter associated with a buffer.
What we're trying to prevent here is partial updates where we see the
intermediary updates to the buffer extents (function record size then
call argument record size) becoming observable from another thread, for
instance, one doing the serialization/flushing.
To do both diagnose this issue properly, we need to be able to honour
the extents being set in the `BufferExtents` records marking the
beginning of the logical buffers when reading an FDR trace. Since LLVM
doesn't use C++'s RTTI mechanism, we instead follow the advice in the
documentation for LLVM Style RTTI
(https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html). We then rely on
this RTTI feature to ensure that our file-based record producer (our
streaming "deserializer") can honour the extents of individual buffers
as we interpret traces.
This also sets us up to be able to eventually do smart
skipping/continuation of FDR logs, seeking instead to find BufferExtents
records in cases where we find potentially recoverable errors. In the
meantime, we make this change to operate in a strict mode when reading
logical buffers with extent records.
Reviewers: mboerger
Subscribers: hiraditya, llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D54201
llvm-svn: 346473
Summary:
Currently `sanitizer_malloc_introspection_t` just adds a version field
which is used to version the allocator ABI. The current allocator ABI
version is returned by the new `GetMallocZoneAllocatorEnumerationVersion()` function.
The motivation behind this change is to allow external processes to
determine the allocator ABI of a sanitized process.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D54045
llvm-svn: 346420
Summary:
Introduce SANITIZER_INTERCEPT_LGAMMAL
dedicated for lgammal(). Disable it for NetBSD
as this routine is not implemented in this OS.
Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54054
llvm-svn: 346352
Summary:
Introduce SANITIZER_INTERCEPT_REMQUOL
dedicated for remquol(). Disable for for NetBSD
as this routine is not implemented in this OS.
Installation of supernumerary interceptors causes
leaking of errors to dlsym(3)-like operations.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54051
llvm-svn: 346351
We have seen failing builds due to a race condition between
RTAsan_dynamic and libc++ headers builds, specifically libc++
headers depend on __config and if this header hasn't been copied
into the final location, including other headers will typically
result in failure. To avoid this race, we add an explicit dependency
on libc++ headers which ensures that they've been copied into place
before the sanitizer object library build starts.
Differential Revision: https://reviews.llvm.org/D54198
llvm-svn: 346339
Summary:
This change updates the version number for FDR logs to 5, and update the
trace processing to support changes in the custom event records.
In the runtime, since we're already writing down the record preamble to
handle CPU migrations and TSC wraparound, we can use the same TSC delta
encoding in the custom event and typed event records that we use in
function event records. We do the same change to typed events (which
were unsupported before this change in the trace processing) which now
show up in the trace.
Future changes should increase our testing coverage to make custom and
typed events as first class entities in the FDR mode log processing
tools.
This change is also a good example of how we end up supporting new
record types in the FDR mode implementation. This shows the places where
new record types are added and supported.
Depends on D54139.
Reviewers: mboerger
Subscribers: hiraditya, arphaman, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D54140
llvm-svn: 346293
Summary: If user wants to raise it back, let them.
Reviewers: kcc, vitalybuka
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D54190
llvm-svn: 346284
This speeds up process startup and teardown and also reduces lock contention when running multiple ASanified/TSanified processes simultaneously. Should greatly improve lit testing time.
Differential Revision: https://reviews.llvm.org/D48445
llvm-svn: 346262
Summary:
For platforms without preinit support (such as NetBSD/amd64) the
initialization routine __xray_init() was called in non-deterministic order
compared to other constructors. This caused breakage failures
as xray routines attempted to execute code with assumption of
being initialized, which was no always true.
Use GCC/Clang extension to set maximal priority to the constructor
calling __xray_init(). This code switches away from C++ lambda form,
as it did not allow to specify this compiler extension.
Reviewers: dberris, joerg
Reviewed By: dberris
Subscribers: llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54136
llvm-svn: 346222
execve("/proc/self/exe") will not work if the binary relies on
$EXEC_ORIGIN in an rpath. Query AT_EXECFN instead, which will give the
same string that the current binary was exec'd with.
Differential Revision: https://reviews.llvm.org/D54113
llvm-svn: 346215
Summary:
This change cuts across LLVM and compiler-rt to add support for
rendering custom events in the XRayRecord type, to allow for including
user-provided annotations in the output YAML (as raw bytes).
This work enables us to add custom event and typed event records into
the `llvm::xray::Trace` type for user-provided events. This can then be
programmatically handled through the C++ API and can be included in some
of the tooling as well. For now we support printing the raw data we
encounter in the custom events in the converted output.
Future work will allow us to start interpreting these custom and typed
events through a yet-to-be-defined API for extending the trace analysis
library.
Reviewers: mboerger
Subscribers: hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D54139
llvm-svn: 346214
While testing my to-be-submitted Solaris sanitizer support on gcc mainline, I ran into
an issue on Solaris/SPARC (sparc-sun-solaris2.11). Initially libasan.so failed to link:
Undefined first referenced
symbol in file
__sanitizer::BufferedStackTrace::FastUnwindStack(unsigned long, unsigned long, unsigned long, unsigned long, unsigned int) /var/gcc/gcc-9.0.0-20181024/11.5-gcc-gas/sparc-sun-solaris2.11/./libsanitizer/asan/.libs/libasan.so
This happens because SANITIZER_CAN_FAST_UNWIND is enabled on non-Linux
SPARC targets (cf. sanitizer_stacktrace.h), but the guard around the SPARCv8-only
definition in sanitizer_stacktrace_sparc.cc only works with clang:
clang predefines __sparcv8__ on non-Solaris, and __sparcv8 only on Solaris
gcc predefines __sparcv8 on Solaris, but __sparc_v8__ on non-Solaris
The attached patch allows for all three variants.
However, disabling SANITIZER_CAN_FAST_UNWIND on all SPARC targets
fixes a couple of testsuite failures in the Solaris asan testsuite, so for now it's better
to keep it disabled everywhere.
This allowed the libsanitizer build to complete and gave reasonable (though slightly
worse than on Solaris/x86) testsuite results.
Differential Revision: https://reviews.llvm.org/D54099
llvm-svn: 346155
While testing the Solaris libsanitizer port on GCC mainline, I found that
I'd messed up
the largefile checks in various ways, some of which showed as compile failures
(wrong structure sizes and member offsets), others at runtime, some of those only
on sparc as a big-endian target.
This patch fixes all of them:
- OFF_T is now correctly defined for 32-bit largefile and traditional
environments, and 64-bit.
- The definition of __sanitizer_dirent now checks the correct conditionals.
- sanitizer_procmaps_solaris.cc undefines _FILE_OFFSET_BITS: before
Solaris 11.4 <procfs.h> doesn't even compile with largefile support
enabled, but the use at hand doesn't need it anyway while g++ 9 will
define _FILE_OFFSET_BITS=64 out of the box.
- With full largefile support enabled, one needs to use e.g. mmap64
instead of mmap; this is hidden behind macros.
With this patch I could bootstrap gcc mainline on both sparc-sun-solaris2.11 and
i386-pc-solaris2.11. In addition, I've successfully built llvm on
i386-pc-solaris2.11.
Differential Revision: https://reviews.llvm.org/D54101
llvm-svn: 346153
Summary:
Prior to this change, we can run into situations where the TSC we're
getting when exiting a function is less than the TSC we got when
entering it. This would sometimes cause the counter for cumulative call
times overflow, which was erroneously also being stored as a signed
64-bit integer.
This change addresses both these issues while adding provisions for
tracking CPU migrations. We do this because moving from one CPU to
another doesn't guarantee that the timestamp counter for some
architectures aren't guaranteed to be synchronised. For the moment, we
leave the provisions there until we can update the data format to
include the counting of CPU migrations we can catch.
We update the necessary tests as well, ensuring that our expectations
for the cycle accounting to be met in case of counter wraparound.
Reviewers: mboerger
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D54088
llvm-svn: 346116
This fixes the issue introduced in r345765 which changed the way in
which the embedded libc++ is being built but omitted tests.
Differential Revision: https://reviews.llvm.org/D54058
llvm-svn: 346052
Include the build of unpoison_passwd() and unpoison_group() for
SANITIZER_INTERCEPT_FGETPWENT_R and SANITIZER_INTERCEPT_FGETGRENT_R.
static void unpoison_passwd(
llvm-svn: 346042
Summary:
NetBSD does not ship with fgetpwent_r() and fgetgrent_r().
Split their interceptors from getpwent_r() and getgrent_r()
and disable for this OS.
Installation of supernumerary interceptors causes leaking of
errors to dlsym(3)-like operations.
No functional change for other OSes.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: srhines, kubamracek, fedor.sergeev, llvm-commits, #sanitizers, mgorny
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54041
llvm-svn: 346038
Recent versions of Ubuntu (17.04 and 18.04) on PowerPC have introduced changes
to Address Space Layout Randomization (ASLR) that is causing 500+ sanitizer
failures. This patch disables ASLR when running the sanitizers on PowerPC 64bit
LE.
Differential Revision: https://reviews.llvm.org/D52900
llvm-svn: 346030
Summary:
The motivation here is to be able support initialisation of the
malloc zone on other code paths.
rdar://problem/45284065
Reviewers: kubamracek, george.karpenkov
Subscribers: llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D54037
llvm-svn: 345983
Summary:
Fix some issues discovered from mostly manual inspection of outputs from
the `llvm-xray fdr-dump` tool.
It turns out we haven't been writing the deltas properly, and have been
writing down zeros for deltas of some records. This change fixes this
oversight born by the recent refactoring.
Reviewers: mboerger
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D54022
llvm-svn: 345954
Summary:
This is a follow-on change to D53858 which turns out to have had a TSC
accounting bug when writing out function exit records in FDR mode.
This change adds a number of tests to ensure that:
- We are handling the delta between the exit TSC and the last TSC we've
seen.
- We are writing the custom event and typed event records as a single
update to the buffer extents.
- We are able to catch boundary conditions when loading FDR logs.
We introduce a TSC matcher to the test helpers, which we use in the
testing/verification of the TSC accounting change.
Reviewers: mboerger
Subscribers: mgorny, hiraditya, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53967
llvm-svn: 345905
Summary:
This change cuts across compiler-rt and llvm, to increment the FDR log
version number to 4, and include the CPU ID in the custom event records.
This is a step towards allowing us to change the `llvm::xray::Trace`
object to start representing both custom and typed events in the stream
of records. Follow-on changes will allow us to change the kinds of
records we're presenting in the stream of traces, to incorporate the
data in custom/typed events.
A follow-on change will handle the typed event case, where it may not
fit within the 15-byte buffer for metadata records.
This work is part of the larger effort to enable writing analysis and
processing tools using a common in-memory representation of the events
found in traces. The work will focus on porting existing tools in LLVM
to use the common representation and informing the design of a
library/framework for expressing trace event analysis as C++ programs.
Reviewers: mboerger, eizan
Subscribers: hiraditya, mgrang, llvm-commits
Differential Revision: https://reviews.llvm.org/D53920
llvm-svn: 345798
Summary:
Windows has always been LLP64, not LP64, so the macros were incorrect.
Check for _WIN64, since AArch64 sets that too.
The tests have to be fixed up in two main ways:
1. Use 'ULL' suffixes to avoid sign extension when passing hex literals
with the sign bit set to signed 128 arguments. Clang
-fms-compatibility makes such hex literals signed, not unsigned.
2. Disable various tests for 80-bit long double interoperation with i128
values.
Reviewers: joerg, mstorsjo
Subscribers: javed.absar, kristof.beyls, hiraditya, aheejin, llvm-commits
Differential Revision: https://reviews.llvm.org/D53918
llvm-svn: 345796
libc++ now supports customizing the ABI namespace directly from the
CMake build so we no longer need to rely on custom CFLAGS.
Differential Revision: https://reviews.llvm.org/D53939
llvm-svn: 345765
Eliminate dropped operations, add new operations.
Update included headers for newer need.
Add a fallback definition of nvlist_ref_t, becaue this type
is internal to libnpf and the kernel, not exported into public
headers.
llvm-svn: 345726
Summary:
In r81552, the HAS_80_BIT_LONG_DOUBLE macro was added to the unit test
only version of int_lib.h. One month later in r85260 the duplicate
int_lib.h was removed, but the tests still passed because we don't build
with -Werror.
This is the minimal change to bring it back, and I decided to put the
configuration macro next to our 128-bit integer support macro.
Reviewers: joerg, compnerd, mstorsjo
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53838
llvm-svn: 345645
Summary:
Add a new helper macro TSAN_INTERCEPTOR_NETBSD_ALIAS_THR2
that handles pthread(3)/libc aliases in scenarios when a name in both
libraries differs not just in prefix namespace.
Handle TSan pthread_sigmask mangling accordingly into __libc_thr_sigsetmask.
Reviewers: vitalybuka, joerg
Reviewed By: vitalybuka
Subscribers: kubamracek, jfb, llvm-commits, #sanitizers, mgorny
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D53863
llvm-svn: 345627
Sort the headers more correctly according to NetBSD style.
Prevent in this code part clang-format, as shuffling the order
will cause build failures.
llvm-svn: 345586
Register new syscall getsockopt2.
Drop removed syscalls pmc_get_info and pmc_control.
While there address compiler warnings about potentially
unused variables.
llvm-svn: 345582
Summary:
This change completes the refactoring of the FDR runtime to support the
following:
- Generational buffer management.
- Centralised and well-tested controller implementation.
In this change we've had to:
- Greatly simplify the code in xray_fdr_logging.cc to only implement the
glue code for calling into the controller.
- Implement the custom and typed event logging functions in the
FDRLogWriter.
- Imbue the `XRAY_NEVER_INSTRUMENT` attribute onto all functions in the
controller implementation.
Reviewers: mboerger, eizan, jfb
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53858
llvm-svn: 345568
Summary:
Prepend minimal when UBsan is mentioned and delete a dead comment
Patch by Igor Sugak
Reviewers: eugenis, pcc
Reviewed By: eugenis
Subscribers: mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D53826
llvm-svn: 345557
This breaks some configurations, so just forward declare everything that we need.
rdar://35943793
Differential revision: https://reviews.llvm.org/D50269
llvm-svn: 345551
Summary:
Some cases where `postCurrentThreadFCT()` are not guarded by our
recursion guard. We've observed that sometimes these can lead to
deadlocks when some functions (like memcpy()) gets outlined and the
version of memcpy is XRay-instrumented, which can be materialised by the
compiler in the implementation of lower-level components used by the
profiling runtime.
This change ensures that all calls to `postCurrentThreadFCT` are guarded
by our thread-recursion guard, to prevent deadlocks.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53805
llvm-svn: 345489
Summary:
In D53560, we assumed a specific layout for memory without using an
explicit structure. This follow-up change uses more portable layout
control by using unions in a struct, and consolidating the memory
management code in the buffer queue.
We also take the opportunity to improve the documentation on the types
and operations, along with simplifying some of the logic in the buffer
queue implementation.
Reviewers: mboerger, eizan
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53802
llvm-svn: 345485
Summary:
This change implements the ref-counting for backing stores associated
with generational buffer management. We do this as an implementation
detail of the buffer queue, instead of exposing this to the interface.
This change allows us to keep the buffer queue interface and usage model
the same.
Depends on D53551.
Reviewers: mboerger, eizan
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D53560
llvm-svn: 345471
Summary:
This is an intermediary step in the full support for generational buffer
management in the FDR runtime. This change makes the FDR controller
aware of the new generation number in the buffers handed out by the
BufferQueue type.
In the process of making this change, we've realised that the cleanest
way of ensuring that the backing store per generation is live while all
the threads that need access to it will need reference counting to tie
the backing store to the lifetime of all threads that have a handle on
buffers associated with the memory.
We also learn that we're missing the edge-case in the function exit
handler's implementation where the first record being written into the
buffer is a function exit, which is caught/fixed by the test for
generational buffer management.
We still haven't wired the controller into the FDR mode runtime, which
will need the reference counting on the backing store implemented to
ensure that we're being conservatively thread-safe with this approach.
Depends on D52974.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53551
llvm-svn: 345445
There's a TSan startup crash on Linux when used in Swift programs, where MonotonicNanoTime will try to call real_clock_gettime and then jump to NULL because interceptors are not yet initialized. This is on Ubuntu 18.04. Looks like TSan's main Initialize() function is called at a point where __progname is already set, but interceptors aren't yet set up. Let's fix this by checking whether interceptors are initialized in MonotonicNanoTime.
Differential Revision: https://reviews.llvm.org/D53528
llvm-svn: 345174
Summary:
At compile-time, create an array of {PC,HumanReadableStackFrameDescription}
for every function that has an instrumented frame, and pass this array
to the run-time at the module-init time.
Similar to how we handle pc-table in SanitizerCoverage.
The run-time is dummy, will add the actual logic in later commits.
Reviewers: morehouse, eugenis
Reviewed By: eugenis
Subscribers: srhines, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D53227
llvm-svn: 344985
Some Darwin functions have pairs like dispatch_apply and dispatch_apply_f so the added _f to interceptor types causes a clash. Let's add _type suffix instead.
Differential Revision: https://reviews.llvm.org/D53167
llvm-svn: 344954
Change the assumption when releasing memory to a buffer queue that new
generations might not be able to re-use the memory mapped addresses.
llvm-svn: 344882
Summary:
This change updates the buffer queue implementation to support using a
generation number to identify the lifetime of buffers. This first part
introduces the notion of the generation number, without changing the way
we handle the buffers yet.
What's missing here is the cleanup of the buffers. Ideally we'll keep
the two most recent generations. We need to ensure that before we do any
writes to the buffers, that we check the generation number(s) first.
Those changes will follow-on from this change.
Depends on D52588.
Reviewers: mboerger, eizan
Subscribers: llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D52974
llvm-svn: 344881
Summary:
This change allows us to handle allocator exhaustion properly in the
segmented array implementation. Before this change, we relied on the
caller of the `trim` function to provide a valid number of elements to
trim. This change allows us to do the right thing in case the elements
to trim is greater than the size of the container.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53484
llvm-svn: 344880
Summary: This patch adds additional features and cpus from libgcc. Unfortunately we've overflowed the existing 32-bits of features so we had to add a new __cpu_features2 variable to hold the additional bits. This matches libgcc as far as I can tell.
Reviewers: echristo
Reviewed By: echristo
Subscribers: dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D53461
llvm-svn: 344830
Summary:
This change updates the buffer queue implementation to support using a
generation number to identify the lifetime of buffers. This first part
introduces the notion of the generation number, without changing the way
we handle the buffers yet.
What's missing here is the cleanup of the buffers. Ideally we'll keep
the two most recent generations. We need to ensure that before we do any
writes to the buffers, that we check the generation number(s) first.
Those changes will follow-on from this change.
Depends on D52588.
Reviewers: mboerger, eizan
Subscribers: llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D52974
llvm-svn: 344670
This abstracts away the file descriptor related logic which makes it
easier to port XRay to platform that don't use file descriptors or
file system for writing the log data, such as Fuchsia.
Differential Revision: https://reviews.llvm.org/D52161
llvm-svn: 344578
Summary:
This change implements a controller for abstracting away the details of
what happens when tracing with FDR mode. This controller type allows us
to test in isolation the various cases where we're encountering function
entry, exit, and other kinds of events we are handling when FDR mode is
enabled.
This change introduces a number of testing facilities we've needed to
better support expressing the conditions we need for the unit tests. We
leave some TODOs for moving those utilities into the LLVM project,
sitting in the `Testing` library, to make matching conditions on XRay
`Trace` instances through googlemock more manageable and declarative.
We don't wire in the controller right away, to allow us to incrementally
update the implementation(s) as we increase testing coverage of the
controller type. There's a need to re-think the way we're managing
buffers in a multi-threaded environment, which is more invasive than
this implementation.
This step in the process allows us to encode our assumptions in the
implementation of the controller, and then evolve the buffer queue
implementation to support generational buffer management to ensure we
can continue to support the cases we're already supporting with the
controller.
Reviewers: mboerger, eizan
Subscribers: mgorny, llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D52588
llvm-svn: 344488
Summary:
Silence warning when linking unittest binary by not passing
-lstdc++ to the linker since it is ignored.
Reviewers: morehouse
Reviewed By: morehouse
Subscribers: mgorny
Differential Revision: https://reviews.llvm.org/D53225
llvm-svn: 344480
This is a follow-up patch to r342541. After further investigations, only
48bits VMA size can be supported. As this is enforced in function
InitializePlatformEarly from lib/rt1/tsan_platform_linux.cc, the access
to the global variable vmaSize variable + switch can be removed. This
also addresses a comment from https://reviews.llvm.org/D52167.
vmaSize of 39 or 42bits are not compatible with a Go program memory
layout as the Go heap will not fit in the shadow memory area.
Patch by: Fangming Fang <Fangming.Fang@arm.com>
llvm-svn: 344329
This allows users of static libraries (such as ubsan) to link without
knowing about this transitive dependency, if using the psapi functions
that require linking to a separate psapi library. Since Windows 7,
these functions (EnumProcessModules, GetModuleInformation,
GetProcessMemoryInfo) are remapped to K32- prefixed ones, available in
kernel32.dll.
Differential Revision: https://reviews.llvm.org/D53012
llvm-svn: 344126
Without this change, tests in coverage.test and dump_coverage.test are
failing on non-x86_64 platforms.
The diff is copied from sanitizer_common library, an alternative would
be to link it together with libFuzzer.
Differential Revision: https://reviews.llvm.org/D53040
llvm-svn: 344104
Summary: This is just a minor cleanup to the allocateOneNode interface. The formals are no-longer used, so I just removed them.
Reviewers: davidxl, void
Reviewed By: davidxl
Subscribers: dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D52509
llvm-svn: 344073
Summary:
The NetBSD version of internal routines no longer call
the indirect syscall interfaces, as these functions were
switched to lib calls.
Remove the remnant code complication that is no
longer needed after this change. Remove the variations
of internal_syscall, as they were NetBSD specific.
No functional change intended.
Reviewers: vitalybuka, joerg, javed.absar
Reviewed By: vitalybuka
Subscribers: kubamracek, fedor.sergeev, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D52955
llvm-svn: 343988
Data involving struct accesses accounting work (plan to support only efficiency-cache-frag flag in the frontend side).
Reviewers: krytarowski, vitalybuka, jfb
Reviewed By : vitalybuka
Differential Revision: https://reviews.llvm.org/D52608
llvm-svn: 343812
Summary:
GetStackTrace treats top PC as a return address from an error reporting
function, and adjusts it down by 1 instruction. This is not necessary in
a signal handler, so adjust PC up to compensate.
Reviewers: kcc, vitalybuka, jfb
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D52802
llvm-svn: 343638
This change fixes two aspects of logging on Fuchsia:
1. When logging to stdout, the output file descriptor should not be
closed in ExecuteCommand, as it prevent fdio_spawn_etc from succeeding
in subsequent calls.
2. When logging to a file in anything other than standalone Zircon, the
log file needs to be created in mutable storage. The best indicator
where this is relative to the Fuchsia component will be given by
'-artifact_prefix=...', so save the log file relative to that path.
Patch by: aarongreen
Differential Revision: https://reviews.llvm.org/D52562
llvm-svn: 343607
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
Differential Revision: https://reviews.llvm.org/D52091
llvm-svn: 343606
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.
New clients can read until they get empty string.
Reviewers: eugenis, morehouse
Subscribers: kubamracek, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D52743
llvm-svn: 343605
This patch removes the instruction set check to make the msr APSR_nzcvq,
ip instruction only execute if Thumb2 is used.
The APSR is a subset of the bits of the CPSR
(B.1.3.3 of the Arm v7 A and R ARM [1]) and is only available for A and
R profiles.
However in section B.9.3.11 of the same document we see that:
"In the A and R profiles, APSR_nzcvq is the same as CPSR_f"
"ARM recommends the APSR forms when only the N, Z, C, V, Q, and GE[3:0]
bits are being written."
This patch also make those files assemble for Armv8-M Mainline
architecture profile.
The builtins were cross-compiled for Arm, Aarch64 and Armv6-M, Armv7-M
and Armv7E-M targets.
Cross-compiled tests were executed for Arm target.
[1]: https://developer.arm.com/docs/ddi0406/latest/arm-architecture-reference-manual-armv7-a-and-armv7-r-edition
Patch by hug-dev (Hugues de Valon).
Differential Revision: https://reviews.llvm.org/D51854
llvm-svn: 343601
Summary:
This essentially reverts r337010 since it breaks UBSan, which is used
for a few platform libraries. The "-z global" flag is now added for
Scudo as well. The only other sanitizer shared libraries are for asan
and hwasan, which have also been reinstated to use the global flag.
Reviewers: cryptoad, eugenis
Reviewed By: cryptoad
Subscribers: kubamracek, mgorny, delcypher, #sanitizers, nickdesaulniers, chh, kongyi, pirama, llvm-commits
Differential Revision: https://reviews.llvm.org/D52770
llvm-svn: 343599
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.
New clients can read until they get empty string.
Reviewers: eugenis, morehouse
Subscribers: kubamracek, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D52743
llvm-svn: 343554
Summary:
When possible, switch syscall(2)/__syscall(2) calls
to direct calls of internal libc symbols.
Add a new function to detect address of a libc
symbol of a function that could be intercepted.
With the address detector in GetRealLibcAddress(),
an optional interceptor of libc call will be bypassed.
The original approach with syscall(2)/__syscall(2)
wasn't portable across supported ABIs and CPU
architectures. The indirect syscall interface is
also a candidate for removal in future revisions
of NetBSD, as the C language ABI is not a good
domain for serialization of arbitrary functions
arguments.
Switch the following functions to libc calls:
- internal_mmap()
- internal_munmap()
- internal_mprotect()
- internal_close()
- internal_open()
- internal_read()
- internal_write()
- internal_ftruncate()
- internal_stat()
- internal_lstat()
- internal_fstat()
- internal_dup2()
- internal_readlink()
- internal_unlink()
- internal_rename()
- internal_sched_yield()
- internal__exit()
- internal_sleep()
- internal_execve()
- NanoTime()
- internal_clock_gettime()
- internal_waitpid()
- internal_getpid()
- internal_getppid()
- internal_getdents()
- internal_lseek()
- internal_sigaltstack()
- internal_fork()
- internal_sigprocmask()
- internal_sysctl()
- internal_sigemptyset()
- internal_sigfillset()
- GetTid()
- TgKill()
This revision leaves room for refactoring in subsequent commits.
Reviewers: vitalybuka, kcc, joerg
Reviewed By: vitalybuka
Subscribers: mgorny, fedor.sergeev, jfb, loverszhaokai, devnexen, kubamracek, llvm-commits, ro, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D51419
llvm-svn: 343523
Summary:
Stop using directly sysctl(3) routines in sanitizer_procmaps_bsd
and replace it with internal_sysctl().
This will allow to install interceptors for sysctl(3).
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D52646
llvm-svn: 343370
This seems to cause the thread's exit code to be clobbered, breaking
Chromium tests.
Also revert follow-up r342654.
> In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
>
> Differential Revision: https://reviews.llvm.org/D52091
llvm-svn: 343322
Summary:
This is for coding standard conformance, and for fixing an ODR violation
issue: __xray::ThreadLocalData is defined twice and differently in
xray_fdr_logging.cc and xray_basic_logging.cc
Reviewers: dberris, mboerger, eizan
Reviewed By: dberris
Subscribers: delcypher, jfb, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D52639
llvm-svn: 343289
Summary:
This change allows us to use the library path from which the LLVM
libraries are installed, in case the LLVM installation generates shared
libraries.
This should address llvm.org/PR39070.
Reviewers: mboerger, eizan
Subscribers: mgorny, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52597
llvm-svn: 343280
Summary:
The default values used for Space/Size for the new SizeClassMap do not work
with Android. The Compact map appears to be in the same boat.
Disable the test on Android for now to turn the bots green, but there is no
reason Compact & Dense should not have an Android test.
Added a FIXME, I will revisit this soon.
Reviewers: eugenis
Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52623
llvm-svn: 343252
Summary:
_Note_: I am not attached to the name `DenseSizeClassMap`, so if someone has a
better idea, feel free to suggest it.
The current pre-defined `SizeClassMap` hold a decent amount of cached entries,
either in cheer number of, or in amount of memory cached.
Empirical testing shows that more compact per-class arrays (whose sizes are
directly correlated to the number of cached entries) are beneficial to
performances, particularly in highly threaded environments.
The new proposed `SizeClassMap` has the following properties:
```
c00 => s: 0 diff: +0 00% l 0 cached: 0 0; id 0
c01 => s: 16 diff: +16 00% l 4 cached: 8 128; id 1
c02 => s: 32 diff: +16 100% l 5 cached: 8 256; id 2
c03 => s: 48 diff: +16 50% l 5 cached: 8 384; id 3
c04 => s: 64 diff: +16 33% l 6 cached: 8 512; id 4
c05 => s: 80 diff: +16 25% l 6 cached: 8 640; id 5
c06 => s: 96 diff: +16 20% l 6 cached: 8 768; id 6
c07 => s: 112 diff: +16 16% l 6 cached: 8 896; id 7
c08 => s: 128 diff: +16 14% l 7 cached: 8 1024; id 8
c09 => s: 144 diff: +16 12% l 7 cached: 7 1008; id 9
c10 => s: 160 diff: +16 11% l 7 cached: 6 960; id 10
c11 => s: 176 diff: +16 10% l 7 cached: 5 880; id 11
c12 => s: 192 diff: +16 09% l 7 cached: 5 960; id 12
c13 => s: 208 diff: +16 08% l 7 cached: 4 832; id 13
c14 => s: 224 diff: +16 07% l 7 cached: 4 896; id 14
c15 => s: 240 diff: +16 07% l 7 cached: 4 960; id 15
c16 => s: 256 diff: +16 06% l 8 cached: 4 1024; id 16
c17 => s: 320 diff: +64 25% l 8 cached: 3 960; id 49
c18 => s: 384 diff: +64 20% l 8 cached: 2 768; id 50
c19 => s: 448 diff: +64 16% l 8 cached: 2 896; id 51
c20 => s: 512 diff: +64 14% l 9 cached: 2 1024; id 48
c21 => s: 640 diff: +128 25% l 9 cached: 1 640; id 49
c22 => s: 768 diff: +128 20% l 9 cached: 1 768; id 50
c23 => s: 896 diff: +128 16% l 9 cached: 1 896; id 51
c24 => s: 1024 diff: +128 14% l 10 cached: 1 1024; id 48
c25 => s: 1280 diff: +256 25% l 10 cached: 1 1280; id 49
c26 => s: 1536 diff: +256 20% l 10 cached: 1 1536; id 50
c27 => s: 1792 diff: +256 16% l 10 cached: 1 1792; id 51
c28 => s: 2048 diff: +256 14% l 11 cached: 1 2048; id 48
c29 => s: 2560 diff: +512 25% l 11 cached: 1 2560; id 49
c30 => s: 3072 diff: +512 20% l 11 cached: 1 3072; id 50
c31 => s: 3584 diff: +512 16% l 11 cached: 1 3584; id 51
c32 => s: 4096 diff: +512 14% l 12 cached: 1 4096; id 48
c33 => s: 5120 diff: +1024 25% l 12 cached: 1 5120; id 49
c34 => s: 6144 diff: +1024 20% l 12 cached: 1 6144; id 50
c35 => s: 7168 diff: +1024 16% l 12 cached: 1 7168; id 51
c36 => s: 8192 diff: +1024 14% l 13 cached: 1 8192; id 48
c37 => s: 10240 diff: +2048 25% l 13 cached: 1 10240; id 49
c38 => s: 12288 diff: +2048 20% l 13 cached: 1 12288; id 50
c39 => s: 14336 diff: +2048 16% l 13 cached: 1 14336; id 51
c40 => s: 16384 diff: +2048 14% l 14 cached: 1 16384; id 48
c41 => s: 20480 diff: +4096 25% l 14 cached: 1 20480; id 49
c42 => s: 24576 diff: +4096 20% l 14 cached: 1 24576; id 50
c43 => s: 28672 diff: +4096 16% l 14 cached: 1 28672; id 51
c44 => s: 32768 diff: +4096 14% l 15 cached: 1 32768; id 48
c45 => s: 40960 diff: +8192 25% l 15 cached: 1 40960; id 49
c46 => s: 49152 diff: +8192 20% l 15 cached: 1 49152; id 50
c47 => s: 57344 diff: +8192 16% l 15 cached: 1 57344; id 51
c48 => s: 65536 diff: +8192 14% l 16 cached: 1 65536; id 48
c49 => s: 81920 diff: +16384 25% l 16 cached: 1 81920; id 49
c50 => s: 98304 diff: +16384 20% l 16 cached: 1 98304; id 50
c51 => s: 114688 diff: +16384 16% l 16 cached: 1 114688; id 51
c52 => s: 131072 diff: +16384 14% l 17 cached: 1 131072; id 48
c53 => s: 64 diff: +0 00% l 0 cached: 8 512; id 4
Total cached: 864928 (152/432)
```
It holds a bit less of 1MB of cached entries at most, and the cache fits in a
page.
The plan is to use this map by default for Scudo once we make sure that there
is no unforeseen impact for any of current use case.
Benchmarks give the most increase in performance (with Scudo) when looking at
highly threaded/contentious environments. For example, rcp2-benchmark
experiences a 10K QPS increase (~3%), and a decrease of 50MB for the max RSS
(~10%). On platforms like Android where we only have a couple of caches,
performance remain similar.
Reviewers: eugenis, kcc
Reviewed By: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52371
llvm-svn: 343246
A version of AndroidGetApiLevel for static executables that is
completely compile-time.
The generic version uses dl_iterate_phdr which, even though it works
in static executables, crashes if called before libc is initialized.
llvm-svn: 343153
In MinGW mode, the /export: directives in object files are interpreted
differently; the symbols are not assumed to be decorated.
Since the GNU linker doesn't support the /alternatename and /include
directives, there's no such distinction for them in lld. This assumes
that the resulting sanitizer libraries will be linked by lld.
Differential Revision: https://reviews.llvm.org/D51882
llvm-svn: 343015
Instead provide manual declarations of the used types, to avoid
pulling in conflicting declarations of some of the functions that
are to be overridden.
Differential Revision: https://reviews.llvm.org/D51914
llvm-svn: 343014
The MinGW headers do define _M_IX86 (contrary to clang-cl and
MSVC where it is a compiler predefined macro), but the headers that
define it aren't included here.
Also check these defines for setting the symbol prefix, instead of
inconsistently using _WIN64 for that.
Differential Revision: https://reviews.llvm.org/D51883
llvm-svn: 343013
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.
The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.
Developed in collaboration with Kostya Serebryany.
Reviewers: kcc
Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52249
llvm-svn: 342923
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.
The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.
Developed in collaboration with Kostya Serebryany.
Reviewers: kcc
Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52249
llvm-svn: 342921
Summary:
The complex division builtins (div?c3) use logb methods from libm to scale numbers during division and avoid rounding issues. However, these come from libm, meaning anyone that uses --rtlib=compiler-rt also has to include -lm. Implement logb* methods for standard ieee 754 floats so we can avoid -lm on those platforms, falling back to the old behavior (using either logb() or `__builtin_logb()`) when not supported.
These new methods are defined internally as `__compiler_rt_logb` so as not to conflict with the libm definitions in any way.
This fixes just the libm methods mentioned in PR32279 and PR28652. libc is still required, although that seems to not be an issue.
Note: this is proposed as an alternative to just adding -lm: D49330.
Reviewers: efriedma, compnerd, scanon, echristo
Reviewed By: echristo
Subscribers: jsji, echristo, nemanjai, dberris, mgorny, kbarton, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49514
llvm-svn: 342917
changing the value of `SANITIZER_MMAP_RANGE_SIZE` to something more
sensible. The available VMA is at most 64GiB and not 256TiB that
was previously being used.
This change gives us several wins:
* Drastically improves LeakSanitizer performance on
Darwin ARM64 devices. On a simple synthentic benchmark
this took leak detection time from ~30 seconds to 0.5 seconds
due to the `ForEachChunk(...)` method enumerating a much smaller
number of regions. Previously we would pointlessly iterate
over a large portion of the SizeClassAllocator32's ByteMap
that would could never be set due it being configured for a much
larger VM space than is actually availble.
* Decreases the memory required for the Primary allocator.
Previously the ByteMap inside the the allocator used
an array of pointers that took 512KiB of space. Now the required
space for the array is 128 bytes.
rdar://problem/43509428
Differential Revision: https://reviews.llvm.org/D51173
llvm-svn: 342868
Summary:
This change spans both LLVM and compiler-rt, where we do the following:
- Add XRay to the LLVMBuild system, to allow for distributing the XRay
trace loading library along with the LLVM distributions.
- Use `llvm-config` better in the compiler-rt XRay implementation, to
depend on the potentially already-distributed LLVM XRay library.
While this is tested with the standalone compiler-rt build, it does
require that the LLVMXRay library (and LLVMSupport as well) are
available during the build. In case the static libraries are available,
the unit tests will build and work fine. We're still having issues with
attempting to use a shared library version of the LLVMXRay library since
the shared library might not be accessible from the standard shared
library lookup paths.
The larger change here is the inclusion of the LLVMXRay library in the
distribution, which allows for building tools around the XRay traces and
profiles that the XRay runtime already generates.
Reviewers: echristo, beanz
Subscribers: mgorny, hiraditya, mboerger, llvm-commits
Differential Revision: https://reviews.llvm.org/D52349
llvm-svn: 342859
Summary:
`__sanitizer_set_death_callback` is a public interface function wrapping
`SetUserDieCallback`. Said function is defined in `sanitizer_termination.cc`,
which is not included in all the RT. Moving the interface function to that
file allows to not have a spurious public fuinction in RT that don't use it.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52363
llvm-svn: 342747
Summary:
The implementation of `internal_mmap(...)` deviates from the contract of
`mmap(...)` -- i.e. error returns are actually the equivalent of `errno`
results. We update how XRay uses `internal_mmap(...)` to better handle
these error conditions.
In the process, we change the default pointers we're using from `char*`
to `uint8_t*` to prevent potential usage of the pointers in the string
library functions that expect to operate on `char*`.
We also take the chance to "promote" sizes of individual `internal_mmap`
requests to at least page size bytes, consistent with the expectations
of calls to `mmap`.
Reviewers: cryptoad, mboerger
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D52361
llvm-svn: 342745
Summary:
Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
to get externally defined functions (eg: LLVMFuzzerInitialize,
LLVMFuzzerCustomMutator, etc) working again.
Also enable tests that depended on these functions (on windows)
Reviewers: rnk, morehouse
Reviewed By: rnk, morehouse
Subscribers: rnk, morehouse, mgorny
Differential Revision: https://reviews.llvm.org/D51700
llvm-svn: 342698
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
Differential Revision: https://reviews.llvm.org/D52091
llvm-svn: 342652
Summary:
As far as I can tell, there is no reason why `__sanitizer::CheckFailed` should
be exported. Looking back in time, it was added with the FIXME with the
following by @timurrrr:
```
[*San/RTL] Fix minor breakage
Grumbling: this hasn't been caught by running 'make check-{a,l,t}san check-sanitizer'
```
I can't find any detail about the breakage, all tests seem to work for me, so
maybe Windows (@rnk?) or something I have no setup for.
The reason to make it private (past the FIXME) is that Scudo defines its own
(without callback) and I am trying to make the .so be loadable with the UBsan
one (that has its own public `CheckFailed`) with as little drama as possible.
Reviewers: eugenis, rnk
Reviewed By: eugenis, rnk
Subscribers: kubamracek, delcypher, #sanitizers, timurrrr, rnk, llvm-commits
Differential Revision: https://reviews.llvm.org/D52279
llvm-svn: 342651
Same idea as r310419: The 8 byte nop is a suffix of the 9 byte nop, and we need at most 6 bytes.
Differential Revision: https://reviews.llvm.org/D51788
llvm-svn: 342649
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.
The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.
In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.
We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module. We also add
the terminfo library detection along with inclusion of the appropriate
compiler flags for header include lookup.
Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.
Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.
Reviewers: mboerger, eizan
Subscribers: mgorny, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52220
llvm-svn: 342617
This is called by Bionic on dlclose to delete the emutls pthread key.
The return value of pthread_key_delete is unchecked and behaviour of
multiple calls to the method is dependent on the implementation of
pthread_key_delete.
Differential Revision: https://reviews.llvm.org/D52251
llvm-svn: 342608
Summary:
Destroy and close a range's vmar if all its memory was unmapped.
This addresses some performance regression due to the proliferation of vmars
when Secondary backed allocations are concerned with Scudo on Fuchsia.
When a Secondary backed allocation was freed, the associated
`ReservedAddressRange` was going away after unmapping the entirety of the
mapping, but without getting rid of the associated vmar properly (which
was created specifically for that mapping). This resulted in an increase of
defunct vmars, that in turn slowed down further new vmar allocations.
This appears to solve ZX-2560/ZX-2642, at least on QEMU.
Reviewers: flowerhack, mcgrathr, phosek, mseaborn
Reviewed By: mcgrathr
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52242
llvm-svn: 342584
Summary:
This patch adds TSan runtime support for Go on linux-aarch64
platforms. This enables people working on golang to implement their
platform/language part of the TSan support.
Basic testing is done with lib/tsan/go/buildgo.sh. Additional testing will be
done as part of the work done in the Go project.
It is intended to support other VMA sizes, except 39 which does not
have enough bits to support the Go heap requirements.
Patch by Fangming Fang <Fangming.Fang@arm.com>.
Reviewers: kubamracek, dvyukov, javed.absar
Subscribers: mcrosier, dberris, mgorny, kristof.beyls, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52167
llvm-svn: 342541
Instead of assuming `-ltinfo` works, check whether there's terminfo
support on the host where LLVMSupport is compiled.
Follow-up to D52220.
llvm-svn: 342523
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.
The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.
In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.
We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module.
Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.
Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.
Reviewers: mboerger, eizan
Subscribers: mgorny, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52220
llvm-svn: 342518
On sparc64/Linux, sparc64 isn't defined; the canonical way of
checking for sparc64 is sparc && arch64, which also works on the
BSDs and Solaris. Since this problem does not occur on 32-bit
architectures, riscv32 can be ignored. This fixes and refines rL324593.
Patch by jrtc27 (James Clarke)
Differential Revision: https://reviews.llvm.org/D43146
llvm-svn: 342504
Support for .preinit_array has been implemented in Fuchsia's libc,
add Fuchsia to the list of platforms that support this feature.
Differential Revision: https://reviews.llvm.org/D52155
llvm-svn: 342357
Summary:
This change makes XRay FDR mode use a single backing store for the
buffer queue, and have indexes into that backing store instead. We also
remove the reliance on the internal allocator implementation in the FDR
mode logging implementation.
In the process of making this change we found an inconsistency with the
way we're returning buffers to the queue, and how we're setting the
extents. We take the chance to simplify the way we're managing the
extents of each buffer. It turns out we do not need the indirection for
the extents, so we co-host the atomic 64-bit int with the buffer object.
It also seems that we've not been returning the buffers for the thread
running the flush functionality when writing out the files, so we can
run into a situation where we could be missing data.
We consolidate all the allocation routines now into xray_allocator.h,
where we used to have routines defined in xray_buffer_queue.cc.
Reviewers: mboerger, eizan
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52077
llvm-svn: 342356
This API has been deprecated three months ago and shouldn't be used
anymore, all clients should migrate to the new string based API.
Differential Revision: https://reviews.llvm.org/D51606
llvm-svn: 342318
Summary:
This change has a number of fixes for FDR mode in compiler-rt along with
changes to the tooling handling the traces in llvm.
In the runtime, we do the following:
- Advance the "last record" pointer appropriately when writing the
custom event data in the log.
- Add XRAY_NEVER_INSTRUMENT in the rewinding routine.
- When collecting the argument of functions appropriately marked, we
should not attempt to rewind them (and reset the counts of functions
that can be re-wound).
In the tooling, we do the following:
- Remove the state logic in BlockIndexer and instead rely on the
presence/absence of records to indicate blocks.
- Move the verifier into a loop associated with each block.
Reviewers: mboerger, eizan
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D51965
llvm-svn: 342122
Similarly to before, D51985 again reduced the number of registers
required for the read/write routines causing this test to fail on
sanitizer-x86_64-linux-autoconf.
llvm-svn: 342092
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase, and matches how these
headers are included elsewhere in compiler-rt.
Also include these headers with angle brackets, as they are system
headers.
Differential Revision: https://reviews.llvm.org/D51913
llvm-svn: 341983
This function isn't declared with a const parameter anywhere; neither
in MSVC (neither in ucrt or in older msvcrt versions) nor in mingw-w64.
Differential Revision: https://reviews.llvm.org/D51876
llvm-svn: 341903
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase. This header actually also
is named with a lowercase name in the Windows SDK as well.
Differential Revision: https://reviews.llvm.org/D51877
llvm-svn: 341857
Summary:
In this change we apply `XRAY_NEVER_INSTRUMENT` to more functions in the
profiling implementation to ensure that these never get instrumented if
the compiler used to build the library is capable of doing XRay
instrumentation.
We also consolidate all the allocators into a single header
(xray_allocator.h) which sidestep the use of the internal allocator
implementation in sanitizer_common.
This addresses more cases mentioned in llvm.org/PR38577.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D51776
llvm-svn: 341647
Summary:
When building without COMPILER_RT_HWASAN_WITH_INTERCEPTORS, skip
interceptors for malloc/free/etc and only export their versions with
__sanitizer_ prefix.
Also remove a hack in mallinfo() interceptor that does not apply to
hwasan.
Reviewers: kcc
Subscribers: kubamracek, krytarowski, llvm-commits
Differential Revision: https://reviews.llvm.org/D51711
llvm-svn: 341598
Add the __msan_va_arg_origin_tls TLS array to keep the origins for variadic function parameters.
Change the instrumentation pass to store parameter origins in this array.
This is a reland of r341528.
test/msan/vararg.cc doesn't work on Mips, PPC and AArch64 (because this
patch doesn't touch them), XFAIL these arches.
Also turned out Clang crashed on i80 vararg arguments because of
incorrect origin type returned by getOriginPtrForVAArgument() - fixed it
and added a test.
llvm-svn: 341554
MachO symbols are prefixed with an extra '_' (that's 3 in total for this
function), so assembly calls have to go through a wrapper to insert any prefix
needed.
llvm-svn: 341540
Add the __msan_va_arg_origin_tls TLS array to keep the origins for
variadic function parameters.
Change the instrumentation pass to store parameter origins in this array.
llvm-svn: 341528
Sigtrap is used for error reporting, but all other signals are better
left for the platform.
In particular, sanitizer signal handlers do not dump registers or
memory which makes debugging harder for no good reason.
llvm-svn: 341500
Summary:
Enable tests on Windows and make check-fuzzer pass on it. Make
check-fuzzer pass on Windows by fixing libFuzzer, fixing tests, and by
disabling tests on Windows. Most of these are disabled temporarily as
support for the tests and the features they test will be added
incrementally. Other tests will not be enabled since they require
things that are not on Windows (eg: afl_driver tests). Every test
that was explicitly disabled on Windows has a comment explaining why
(unless obvious like merge-posix.test).
The lit.cfg file was modified to support running tests on windows.
fuzzer-dirs.test was fixed by making the Windows implementation print
the same error message as the posix version.
merge-control-file.test was fixed by making the test binary end with
the ".exe" extension (on all platforms).
Patch By: metzman
Reviewers: morehouse
Reviewed By: morehouse
Subscribers: srhines, mgorny
Differential Revision: https://reviews.llvm.org/D51549
llvm-svn: 341385
sysctl has a different signature under OpenBSD
Reviewers: krytarowski
Reviewed By: krytarowski
Differential Revision: https://reviews.llvm.org/D51572
llvm-svn: 341285
Summary:
Switch local sysctl(2) calls to internal_sysctl().
This is a preparation for introduction of interceptors for
the sysctl*() family of functions and switching `internal_sysctl*()`
to libc calls bypassing interceptors.
No functional change intended with this revision.
Reviewers: vitalybuka, joerg, kcc
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D51425
llvm-svn: 341181
Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.
Patch By: metzman
Reviewers: morehouse, rnk
Reviewed By: morehouse, rnk
Subscribers: #sanitizers, delcypher, morehouse, kcc, eraman
Differential Revision: https://reviews.llvm.org/D51022
llvm-svn: 341082
using sysctl to get the tic frequency data.
still linkage issue for X-ray_init not resolved.
Reviewers: dberris, kubamracek
Reviewed By: dberris
Differential Revision: https://reviews.llvm.org/D51399
llvm-svn: 341019
Now that all _zx_vmar_... calls have been updated, we can undo the
change made in r337801 and switch over to the new calls.
Differential Revision: https://reviews.llvm.org/D51468
llvm-svn: 341011
Summary:
This was an experimental feature. After evaluating it with:
1) https://github.com/google/fuzzer-test-suite/tree/master/engine-comparison
2) enabling on real world fuzz targets running at ClusterFuzz and OSS-Fuzz
The following conclusions were made:
1) With fuzz targets that have reached a code coverage plateau, the feature does
not improve libFuzzer's ability to discover new coverage and may actually
negatively impact it.
2) With fuzz targets that have not yet reached a code coverage plateau, the
feature might speed up new units discovery in some cases, but it is quite
rare and hard to confirm with a high level on confidence.
Revert of https://reviews.llvm.org/D48054 and https://reviews.llvm.org/D49621.
Reviewers: metzman, morehouse
Reviewed By: metzman, morehouse
Subscribers: delcypher, #sanitizers, llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D51455
llvm-svn: 340976
Summary:
We need this in order to properly report heap-use-after-free,
since we don't have a quarantine.
This is a first part of the code, more like a proof of concept.
But I'd like to commit at as is and proceed with refactoring,
adding a ThreadRegistry, and extending the functionality.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D51394
llvm-svn: 340971
Summary:
The previous version of the patch makes some code unable to distinguish
failure to map address 0 and error.
Revert to turn the bots back to green while figuring out a new approach.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D51451
llvm-svn: 340957
Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.
Reviewers: morehouse, rnk
Reviewed By: morehouse, rnk
Subscribers: #sanitizers, delcypher, morehouse, kcc, eraman
Differential Revision: https://reviews.llvm.org/D51022
llvm-svn: 340949
The test doesn't pass on Windows, where sizeof(long) == 4 also
on 64-bit, and so it isn't a multiple of sizeof(void*).
This also reverts the follow-up r340886.
> Summary: a constrained RingBuffer optimized for fast push
>
> Reviewers: eugenis
>
> Reviewed By: eugenis
>
> Subscribers: kubamracek, mgorny, delcypher, #sanitizers, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D51196
llvm-svn: 340924
Summary:
The UINTMAX_T type will be used in new interceptors.
While there, correct the type of strtoumax(3) from INTMAX_T to UINTMAX_T.
Original patch from Yang Zheng.
Reviewers: vitalybuka, kcc, joerg
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, tomsun.0.7, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D51106
llvm-svn: 340907
Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.
Patch By: metzman
Reviewers: morehouse, rnk
Reviewed By: morehouse, rnk
Subscribers: morehouse, kcc, eraman
Differential Revision: https://reviews.llvm.org/D51022
llvm-svn: 340860
Summary:
This change removes further cases where the profiling mode
implementation relied on dynamic memory allocation. We're using
thread-local aligned (uninitialized) memory instead, which we initialize
appropriately with placement new.
Addresses llvm.org/PR38577.
Reviewers: eizan, kpw
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D51278
llvm-svn: 340814
Summary:
This change saves and restores the full flags register in x86_64 mode.
This makes running instrumented signal handlers safer, and avoids flags
set during the execution of the event handlers from polluting the
instrumented call's flags state.
Reviewers: kpw, eizan, jfb
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D51277
llvm-svn: 340812
Now that all Zircon calls have been transitioned to take time as signed
value, we can transition back to ZX_TIME_INFINITE, undoing the change
made in r337802.
Differential Revision: https://reviews.llvm.org/D51266
llvm-svn: 340764
Summary:
The reason for the existence of `eraseHeader` was that it was deemed faster
to null-out a chunk header, effectively making it invalid, rather than marking
it as available, which incurred a checksum computation and a cmpxchg.
A previous use of `eraseHeader` was removed with D50655 due to a race.
Now we remove the second use of it in the Quarantine deallocation path and
replace is with a `compareExchangeHeader`.
The reason for this is that greatly helps debugging some heap bugs as the chunk
header is now valid and the chunk marked available, as opposed to the header
being invalid. Eg: we get an invalid state error, instead of an invalid header
error, which reduces the possibilities. The computational penalty is negligible.
Reviewers: alekseyshl, flowerhack, eugenis
Reviewed By: eugenis
Subscribers: delcypher, jfb, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D51224
llvm-svn: 340633
- GetRandom and GetnumberOfCPUs using sys call for the former.
- enabling unit tests for the other oses.
Reviewers: kubamracek
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D50937
llvm-svn: 340621
Calling task_for_pid with mach_task_self is just returning mach_task_self anyway, but it also triggers system warnings (task_for_pid is only supposed to be used by high-privileged executables). NFC.
rdar://problem/39198248
Differential Revision: https://reviews.llvm.org/D51119
llvm-svn: 340587
Summary:
`MmapNoAccess` & `MmapFixedNoAccess` return directly the result of
`internal_mmap`, as opposed to other Mmap functions that return nullptr.
This inconsistency leads to some confusion for the callers, as some check for
`~(uptr)0` (`MAP_FAILED`) for failure (while it can fail with `-ENOMEM` for
example).
Two potential solutions: change the callers, or make the functions return
`nullptr` on failure to follow the precedent set by the other functions.
The second option looked more appropriate to me.
Correct the callers that were wrongly checking for `~(uptr)0` or
`MAP_FAILED`.
TODO for follow up CLs:
- There are a couple of `internal_mmap` calls in XRay that check for
MMAP_FAILED as a result as well (cc: @dberris); they should use
`internal_iserror`;
Reviewers: eugenis, alekseyshl, dberris, kubamracek
Reviewed By: alekseyshl
Subscribers: kristina, kubamracek, delcypher, #sanitizers, dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D50940
llvm-svn: 340576
On macOS Mojave, the OS started using the XOR-by-a-secret-key scheme (same as glibc is alread doing) for storing the SP value in setjmp environment. We need to adjust for that to keep supporting setjmp/longjmp on latest Darwin. The patch is basically doing the same what we're already doing for glibc.
rdar://problem/43542596
Differential Revision: https://reviews.llvm.org/D51064
llvm-svn: 340350
The synchronous dispatch functions in GCD (dispatch_sync, dispatch_barrier_sync), don't make a copy of the passed block. To maintain binary compatibility, we should avoid doing that as well in TSan, as there's no reason to do that. The synchronous dispatch functions will not return before the block is actually executed.
rdar://problem/42242579
Differential Revision: https://reviews.llvm.org/D50920
llvm-svn: 340342
Summary:
When deallocating thread stacks, we use one thread's unsafe stack size
to deallocate another thread's unsafe stack; however, the two sizes may
differ. Record an unsafe stack's size in the thread stack linked list.
Reviewers: pcc, eugenis
Reviewed By: eugenis
Subscribers: delcypher, llvm-commits, #sanitizers, kcc
Differential Revision: https://reviews.llvm.org/D51016
llvm-svn: 340308
Summary:
The idea behind this change is to allow sanitization of libc. We are prototyping on Bionic,
but the tool interface will be general enough (or at least generalizable) to support any other libc.
When libc depends on libclang_rt.hwasan, the latter can not interpose libc functions.
In fact, majority of interceptors become unnecessary when libc code is instrumented.
This change gets rid of most hwasan interceptors and provides interface for libc to notify
hwasan about thread creation and destruction events. Some interceptors (pthread_create)
are kept under #ifdef to enable testing with uninstrumented libc. They are expressed in
terms of the new libc interface.
The new cmake switch, COMPILER_RT_HWASAN_WITH_INTERCEPTORS, ON by default, builds testing
version of the library with the aforementioned pthread_create interceptor.
With the OFF setting, the library becomes more of a libc plugin.
Reviewers: vitalybuka, kcc, jfb
Subscribers: srhines, kubamracek, mgorny, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D50922
llvm-svn: 340216
Summary:
Use `FUTEX_PRIVATE_FLAG` in conjunction with the wait & wake operations
employed by `BlockingMutex`. As far as I can tell, the mutexes are
process-private, and there is an actual performance benefit at employing the
private operations. There should be no downside to switching to it.
Reviewers: eugenis, alekseyshl, dvyukov
Reviewed By: dvyukov
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D50910
llvm-svn: 340178
task_vm_info is a "revisioned" structure, new OS versions add fields to the end, and compatibility is based on the reported size. On older OS versions, min_address/max_address is not filled back. Let's handle that case. Unfortunately, we can't really write a test (as the failure only happens when on a specific OS version).
Differential Revision: https://reviews.llvm.org/D50275
llvm-svn: 340058
Summary:
We avoid using dynamic memory allocated with the internal allocator in
the profile collection service used by profiling mode. We use aligned
storage for globals and in-struct storage of objects we dynamically
initialize.
We also remove the dependency on `Vector<...>` which also internally
uses the dynamic allocator in sanitizer_common (InternalAlloc) in favour
of the XRay allocator and segmented array implementation.
This change addresses llvm.org/PR38577.
Reviewers: eizan
Reviewed By: eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D50782
llvm-svn: 339978
Summary:
Since we're casting from double to size_t during input selection, we
really want a discrete distribution over size_t rather than a piecewise
distribution over doubles.
Reviewers: kcc
Reviewed By: kcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D50356
llvm-svn: 339973
This reapplies commit r339935 with the following changes:
* make longjmp test C, not C++, to avoid dependency on libc++/libstdc++
* untag pointer in memset interceptor
x86_64 does not have TBI, so hwasan barely works there. Tests must be carefully
written in a way that does not leak tagged pointer to system libraries.
llvm-svn: 339963
Summary:
A callback to annotate longjmp-like code.
Unlike __asan_handle_no_return, in hwasan we can not conservatively
"unpoison" the entire thread stack, because there is no such thing as
unpoisoned memory. Pointer and memory tags must always match.
Reviewers: vitalybuka, kcc
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D50752
llvm-svn: 339935
- The alternative syscall exists only on FreeBSD.
- Adding OpenBSD's equivalent and while at it other
remaining oses ones.
Reviewers: vitalybuka, krytarowsky, hiraditya
Reviewed By: hiraditya
Differential Revision: https://reviews.llvm.org/D50760
llvm-svn: 339886
Summary:
This reverses an earlier decision to allow seg-faulting from the
XRay-allocated memory if it turns out that the system cannot provide
physical memory backing that cannot be swapped in/out on Linux.
This addresses http://llvm.org/PR38588.
Reviewers: eizan
Reviewed By: eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D50831
llvm-svn: 339869
This relands commit r339405 (reverted in commit r339408.) The original
revert was due to tests failing on a darwin buildbot; however, after
looking at the affected code more I realized that the Darwin safestack
support has always been broken and disabled it in r339719. This relands
the original commit.
llvm-svn: 339723
Summary:
Darwin support does not appear to be used as evidenced by the fact that
the pthread interceptors have never worked and there is no support for
other common threading mechanisms like GCD.
Reviewers: pcc, eugenis, kubamracek
Reviewed By: pcc, kubamracek
Subscribers: kubamracek, mgorny, delcypher, llvm-commits, #sanitizers, kcc
Differential Revision: https://reviews.llvm.org/D50718
llvm-svn: 339719
Summary:
There is a race window in the deallocation path when the Quarantine is bypassed.
Initially we would just erase the header of a chunk if we were not to use the
Quarantine, as opposed to using a compare-exchange primitive, to make things
faster.
It turned out to be a poor decision, as 2 threads (or more) could simultaneously
deallocate the same pointer, and if the checks were to done before the header
got erased, this would result in the pointer being added twice (or more) to
distinct thread caches, and eventually be reused.
Winning the race is not trivial but can happen with enough control over the
allocation primitives. The repro added attempts to trigger the bug, with a
moderate success rate, but it should be enough to notice if the bug ever make
its way back into the code.
Since I am changing things in this file, there are 2 smaller changes tagging
along, marking a variable `const`, and improving the Quarantine bypass test at
runtime.
Reviewers: alekseyshl, eugenis, kcc, vitalybuka
Reviewed By: eugenis, vitalybuka
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D50655
llvm-svn: 339705
Summary:
Export __sanitizer_malloc, etc as aliases to malloc, etc.
This way users can wrap sanitizer malloc, even in fully static binaries.
Both jemalloc and tcmalloc provide similar aliases (je_* and tc_*).
Reviewers: vitalybuka, kcc
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D50570
llvm-svn: 339614
Summary:
Don't crash when /proc/self/maps is inaccessible from main thread.
It's not a big deal, really.
Reviewers: vitalybuka, kcc
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D50574
llvm-svn: 339607
Summary:
Provide __hwasan_shadow_init that can be used to initialize shadow w/o touching libc.
It can be used to bootstrap an unusual case of fully-static executable with
hwasan-instrumented libc, which needs to run hwasan code before it is ready to serve
user calls like madvise().
Reviewers: vitalybuka, kcc
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D50581
llvm-svn: 339606
Summary:
When compiling with `WERROR=ON` & a recent clang, having the `st(?)` registers
in the clobber list produces a fatal error (except `st(7)` for some reason):
```
.../sanitizer_common/sanitizer_atomic_clang_x86.h:98:9: error: inline asm clobber list contains reserved registers: ST0, ST1, ST2, ST3, ST4, ST5, ST6 [-Werror,-Winline-asm]
"movq %1, %%mm0;" // Use mmx reg for 64-bit atomic moves
^
<inline asm>:1:1: note: instantiated into assembly here
movq 8(%esp), %mm0;movq %mm0, (%esi);emms;
^
.../sanitizer_common/sanitizer_atomic_clang_x86.h:98:9: note: Reserved registers on the clobber list may not be preserved across the asm statement, and clobbering them may lead to undefined behaviour.
"movq %1, %%mm0;" // Use mmx reg for 64-bit atomic moves
^
<inline asm>:1:1: note: instantiated into assembly here
movq 8(%esp), %mm0;movq %mm0, (%esi);emms;
^
```
As far as I can tell, they were in there due to the use of the `emms`
instruction, but removing the clobber doesn't appear to have a functional
impact. I am unsure if there is a better way to address this.
Reviewers: eugenis, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, delcypher, jfb, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D50562
llvm-svn: 339575
HWASan will not run on older Android releases where we use
__android_log_write for logging.
This dependency is also harmful in the case when libc itself depends
on hwasan, because it creates a loop of
libc -> hwasan -> liblog -> libc
which makes liblog vs libc initialization order undetermined.
Without liblog the loop is just
libc -> hwasan -> libc
and any init order issues can be solved in hwasan.
llvm-svn: 339449
This reverts commit r339405, it's failing on Darwin buildbots because
it doesn't seem to have a tgkill/thr_kill2 interface. It has a
__pthread_kill() syscall, but that relies on having a handle to the
thread's port which is not equivalent to it's tid.
llvm-svn: 339408
Summary:
glibc can call SafeStack instrumented code even after the last pthread
data destructor has run. Delay cleaning-up unsafe stacks for threads
until the thread is dead by having future threads clean-up prior threads
stacks.
Reviewers: pcc, eugenis
Reviewed By: eugenis
Subscribers: cryptoad, eugenis, kubamracek, delcypher, llvm-commits, #sanitizers, kcc
Differential Revision: https://reviews.llvm.org/D50406
llvm-svn: 339405
Summary:
When compiling with WERROR=ON, a new fatal warning started popping up recently
(due to -Werror,-Winline-asm):
```
.../lib/sanitizer_common/sanitizer_linux.cc:1214:24: error: inline asm clobber list contains reserved registers: RSP [-Werror,-Winline-asm]
"syscall\n"
^
<inline asm>:1:1: note: instantiated into assembly here
syscall
^
.../lib/sanitizer_common/sanitizer_linux.cc:1214:24: note: Reserved registers on the clobber list may not be preserved across the asm statement, and clobbering them may lead to undefined behaviour.
"syscall\n"
^
<inline asm>:1:1: note: instantiated into assembly here
syscall
^
```
Removing `rsp` from the clobber list makes the warning go away, and does not
appear to have a functional impact. If there is another way to solve this, let
me know.
Reviewers: eugenis, vitalybuka
Reviewed By: eugenis
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D50519
llvm-svn: 339370
As for Linux with its getrandom's syscall, giving the possibility to fill buffer with native call for good quality but falling back to /dev/urandom in worst case similarly.
Reviewers: vitalybuka, krytarowski
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D48804
llvm-svn: 339318
Summary:
We only run the 3rd check if 2nd check finds unstable edges.
3rd UpdateUnstableCounters is now merged with ApplyUnstableCounters to only run 1 iteration.
Patch by Kyungtak Woo (@kevinwkt).
Reviewers: Dor1s, metzman, morehouse
Reviewed By: Dor1s, morehouse
Subscribers: delcypher, #sanitizers, llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D50411
llvm-svn: 339249
Summary:
There may be cases in which a user wants to know which part of their code is unstable.
We use ObservedFuncs and UnstableCounters to print at exit which of the ObservedFunctions
are unstable under the -print_unstable_stats flag.
Patch by Kyungtak Woo (@kevinwkt).
Reviewers: Dor1s, metzman, morehouse
Reviewed By: Dor1s, metzman, morehouse
Subscribers: delcypher, #sanitizers, llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D50264
llvm-svn: 339081
Summary:
Added functions that calculate stats while fuzz targets are running and give
mutations weight based on how much new coverage they provide, and choose better
performing mutations more often.
Patch by Kodé Williams (@kodewilliams).
Reviewers: Dor1s, metzman, morehouse
Reviewed By: Dor1s, morehouse
Subscribers: delcypher, kcc, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49621
llvm-svn: 338776
Summary:
This patch introduces `llvm_profile_set_dir_mode` and `llvm_profile_get_dir_mode` to
the compiler-rt profile API.
Originally, profile data was placed into a directory that was created with a hard-coded
mode value of 0755 (for non-win32 builds). In certain cases, it can be helpful to create
directories with a different mode other than 0755. This patch introduces set/get
routines to allow users to specify a desired mode. The default remains at 0755.
Reviewers: void, probinson
Reviewed By: probinson
Subscribers: probinson, dberris, cfe-commits
Differential Revision: https://reviews.llvm.org/D49953
llvm-svn: 338456
Summary:
This change provides access to the file header even in the in-memory
buffer processing. This allows in-memory processing of the buffers to
also check the version, and the format, of the profile data.
Reviewers: eizan, kpw
Reviewed By: eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D50037
llvm-svn: 338347
Summary:
Before my change, BlockingMutex used Windows critial sections. Critical
sections can only be initialized by calling InitializeCriticalSection,
dynamically.
The primary sanitizer allocator expects to be able to reinterpret zero
initialized memory as a BlockingMutex and immediately lock it.
RegionInfo contains a mutex, and it placement new is never called for
it. These objects are accessed via:
RegionInfo *GetRegionInfo(uptr class_id) const {
DCHECK_LT(class_id, kNumClasses);
RegionInfo *regions = reinterpret_cast<RegionInfo *>(SpaceEnd());
return ®ions[class_id];
}
The memory comes from the OS without any other initialization.
For various reasons described in the comments, BlockingMutex::Lock would
check if the object appeared to be zero-initialized, and it would lazily
call the LinkerInitialized constructor to initialize the critical
section. This pattern is obviously racy, and the code had a bunch of
FIXMEs about it.
The best fix here is to use slim reader writer locks, which can start
out zero-initialized. They are available starting in Windows Vista. I
think it's safe to go ahead and use them today.
Reviewers: kcc, vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D49893
llvm-svn: 338331
Summary:
rL325492 disables FPU features when using soft floating point
(-mfloat-abi=soft), which is used internally when building for arm. This causes
errors with builtins that utililize VFP instructions.
With this change we check if VFP is enabled (by checking if the preprocessor
macro __VFP_FP__ is defined), and exclude such builtins if it is not enabled.
Reviewers: rengolin, samsonov, compnerd, smeenai, javed.absar, peter.smith
Reviewed By: peter.smith
Subscribers: delcypher, peter.smith, mgorny, kristof.beyls, chrib, llvm-commits
Differential Revision: https://reviews.llvm.org/D47217
llvm-svn: 338284
Summary:
This change moves FDR mode to use `internal_mmap(...)` from
sanitizer_common instead of the internal allocator interface. We're
doing this to sidestep the alignment issues we encounter with the
`InternalAlloc(...)` functions returning pointers that have some magic
bytes at the beginning.
XRay copies bytes into the buffer memory, and does not require the magic
bytes tracking the other sanitizers use when allocating/deallocating
buffers.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D49972
llvm-svn: 338228
0x22000000 happens to be on the left of a heap allocation and the error
message is different (heap-buffer-overflow).
FreeBSD NetBSD have larger SHADOW_OFFSET (0x40000000) but let's try not
using #ifdef here.
llvm-svn: 338208
This ports the profiling runtime on Fuchsia and enables the
instrumentation. Unlike on other platforms, Fuchsia doesn't use
files to dump the instrumentation data since on Fuchsia, filesystem
may not be accessible to the instrumented process. We instead use
the data sink to pass the profiling data to the system the same
sanitizer runtimes do.
Differential Revision: https://reviews.llvm.org/D47208
llvm-svn: 337881
Summary:
Added a new mode within flag -handle_unstable for new unstable handling algorithm that does the following:
When an edge is shown as unstable, copy to UnstableCounters the value 0.
During ApplyUnstableCounters we copy back the value 0 to ModuleInline8bitCounters if the edge was unstable.
This way we would be ignoring completely features that were collected through non-determinism.
Unstable hits would be counted as if it never hit.
Reviewers: metzman, Dor1s, kcc, morehouse
Reviewed By: metzman, morehouse
Subscribers: delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49684
llvm-svn: 337853
Objective-C tagged pointers (either bottom-most or top-most bit is 1) are valid Obj-C objects but are not valid pointers. Make sure we don't crash on them when used in objc_sync_enter/objc_sync_exit. Instead, let's synchronize on a global object.
Differential Revision: https://reviews.llvm.org/D49707
llvm-svn: 337837
Until now, our code preferred backslashes to slashes, whereas Windows
allows using both types of directory separators in one path string.
Differential Revision: https://reviews.llvm.org/D49664
llvm-svn: 337826
This is a preparation for breaking change when all Zircon calls that
take time as an argument will start using signed valued. We will
transition back to ZX_TIME_INFITINE after all the changes to these
symbols are done and become part of the Fuchsia SDK.
Differential Revision: https://reviews.llvm.org/D49694
llvm-svn: 337802
This is a preparation for breaking changes to _zx_vmar_... calls.
We will transition back to _zx_vmar_... after all the changes to
these symbols are done and become part of the Fuchsia SDK.
Differential Revision: https://reviews.llvm.org/D49697
llvm-svn: 337801
The path to headers which are installed into libc++ build directory
has changed in r337727 which broke the libFuzzer build.
Differential Revision: https://reviews.llvm.org/D49705
llvm-svn: 337775
Summary:
Created unstable_handle flag that takes 1 or 2, depending on the handling type.
Modified RunOne to accommodate the following heuristic:
Use the first CollectFeatures to count how many features there are.
If no new features, CollectFeatures like before.
If there is new feature, we run CB 2 more times,
Check which edges are unstable per input and we store the least amount of hit counts for each edge.
Apply these hit counts back to inline8bitcounters so that CollectFeatures can work as intended.
Modified UnstableCounters to 8int_t and created a bitset UnstableSet to tell which edges are unstable.
Patch by Kyungtak Woo (@kevinwkt).
Reviewers: Dor1s, metzman, morehouse
Reviewed By: Dor1s, morehouse
Subscribers: delcypher, #sanitizers, llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D49525
llvm-svn: 337696
When shadow stack from Intel CET is enabled, the first instruction of all
indirect branch targets must be a special instruction, ENDBR.
lib/asan/asan_interceptors.cc has
...
int res = REAL(swapcontext)(oucp, ucp);
...
REAL(swapcontext) is a function pointer to swapcontext in libc. Since
swapcontext may return via indirect branch on x86 when shadow stack is
enabled, as in this case,
int res = REAL(swapcontext)(oucp, ucp);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This function may be
returned via an indirect branch.
Here compiler must insert ENDBR after call, like
call *bar(%rip)
endbr64
I opened an LLVM bug:
https://bugs.llvm.org/show_bug.cgi?id=38207
to add the indirect_return attribute so that it can be used to inform
compiler to insert ENDBR after REAL(swapcontext) call. We mark
REAL(swapcontext) with the indirect_return attribute if it is available.
This fixed:
https://bugs.llvm.org/show_bug.cgi?id=38249
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D49608
llvm-svn: 337603
`VM_MEMORY_SANITIZER`.
It turns out that `VM_MEMORY_ANALYSIS_TOOL` is already reserved for
use by other tools so switch to a tag reserved for use by the Sanitizers.
rdar://problem/41969783
Differential Revision: https://reviews.llvm.org/D49603
llvm-svn: 337579
Summary:
There is currently too much redundancy in the class/variable/* names in Scudo:
- we are in the namespace `__scudo`, so there is no point in having something
named `ScudoX` to end up with a final name of `__scudo::ScudoX`;
- there are a lot of types/* that have `Allocator` in the name, given that
Scudo is an allocator I figure this doubles up as well.
So change a bunch of the Scudo names to make them shorter, less redundant, and
overall simpler. They should still be pretty self explaining (or at least it
looks so to me).
The TSD part will be done in another CL (eg `__scudo::ScudoTSD`).
Reviewers: alekseyshl, eugenis
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D49505
llvm-svn: 337557
r337531 changed return type of MmapFixedNoReserve, but esan wasn't updated.
As the result esan shadow setup always fails.
We probably need to make MmapFixedNoAccess signature consistent
with MmapFixedNoReserve. But this is just to unbreak tests.
llvm-svn: 337550
MmapFixedNoReserve does not terminate process on failure.
Failure to check its result and die will always lead to harder
to debug crashes later in execution. This was observed in Go
processes due to some address space conflicts.
Consistently check result of MmapFixedNoReserve.
While we are here also add warn_unused_result attribute
to prevent such bugs in future and change return type to bool
as that's what all callers want.
Reviewed in https://reviews.llvm.org/D49367
llvm-svn: 337531
Summary:
Add __llvm_profile_get_filename interface to get the profile filename,
which can be used for identifying which profile file belongs to an app
when multiple binaries are instrumented and dumping profiles into the
same directory. The filename includes the path.
Reviewers: davidxl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D49529
llvm-svn: 337482
This change makes it so that the profiling mode implementation will only
write files when there are buffers to write. Before this change, we'd
always open a file even if there were no profiles collected when
flushing.
llvm-svn: 337443
When providing raw access to the FDR mode buffers, we used to not
include the extents metadata record. This oversight means that
processing the buffers in-memory will lose important information that
would have been written in files.
This change exposes the metadata record by serializing the data
similarly to how we would do it when flushing to files.
llvm-svn: 337441
MAP_NORESERVE is not supported or a no-op on BSD.
Reviewers: dberris
Reviewed By: dberris
Differential Revision: https://reviews.llvm.org/D49494
llvm-svn: 337440
Fuchsia doesn't have signals; instead it expects processes to have a
dedicated exception thread that binds to the process' exception port and
waits for exception packets to be delivered. On the other hand,
libFuzzer and sanitizer_common use expect to collect crash information
via libunwind from the same thread that caused the exception.
The long term fix is to improve support for remote unwinding in
libunbwind, plumb this through sanitizer_common and libFuzzer, and
handle the exception exclusively on the exception thread. In the
meantime, this revision has the exception thread "resurrect" the
crashing thread by:
* saving its general purpose register state onto the crashing thread's
stack,
* setting the crashing thread's program counter to an assembly trampoline
with the CFI information needed by libunwind, and
* resuming the crashed thread.
Patch By: aarongreen
Differential Revision: https://reviews.llvm.org/D48509
llvm-svn: 337418
Summary:
Created IterateInline8bitCounters, a single template for visiting Inline8bitCounters (nested for loop)
Made InitializeUnstableCounters and UpdateUnstableCounters both send a lambda to IterateInline8bitCounters.
Patch by Kyungtak Woo (@kevinwkt).
Reviewers: Dor1s, metzman, kcc, morehouse
Reviewed By: metzman, morehouse
Subscribers: delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49453
llvm-svn: 337403
Summary:
This is a follow-on to D49217 which simplifies and optimises the
implementation of the segmented array. In this patch we co-locate the
book-keeping for segments in the `__xray::Array<T>` with the data it's
managing. We take the chance in this patch to actually rename `Chunk` to
`Segment` to better align with the high-level description of the
segmented array.
With measurements using benchmarks landed in D48879, we've identified
that calls to `pthread_getspecific` started dominating the cycles, which
led us to revert the change made in D49217 to use C++ thread_local
initialisation instead (it reduces the cost by a huge margin, since we
save one PLT-based call to pthread functions in the hot path). In
particular, this is in `__xray::getThreadLocalData()`.
We also took the opportunity to remove the least-common-multiple based
calculation and instead pack as much data into segments of the array.
This greatly simplifies the API of the container which hides as much of
the implementation details as possible. For instance, we calculate the
number of elements we need for the each segment internally in the Array
instead of making it part of the type.
With the changes here, we're able to get a measurable improvement on the
performance of profiling mode on top of what D48879 already provides.
Depends on D48879.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D49363
llvm-svn: 337343
Summary:
This change simplifies the XRay Allocator implementation to self-manage
an mmap'ed memory segment instead of using the internal allocator
implementation in sanitizer_common.
We've found through benchmarks and profiling these benchmarks in D48879
that using the internal allocator in sanitizer_common introduces a
bottleneck on allocating memory through a central spinlock. This change
allows thread-local allocators to eliminate contention on the
centralized allocator.
To get the most benefit from this approach, we also use a managed
allocator for the chunk elements used by the segmented array
implementation. This gives us the chance to amortize the cost of
allocating memory when creating these internal segmented array data
structures.
We also took the opportunity to remove the preallocation argument from
the allocator API, simplifying the usage of the allocator throughout the
profiling implementation.
In this change we also tweak some of the flag values to reduce the
amount of maximum memory we use/need for each thread, when requesting
memory through mmap.
Depends on D48956.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D49217
llvm-svn: 337342
Summary:
Fix a bug in FDR mode which didn't allow for re-initialising the logging
in the same process. This change ensures that:
- When we flush the FDR mode logging, that the state of the logging
implementation is `XRAY_LOG_UNINITIALIZED`.
- Fix up the thread-local initialisation to use aligned storage and
`pthread_getspecific` as well as `pthread_setspecific` for the
thread-specific data.
- Actually use the pointer provided to the thread-exit cleanup handling,
instead of assuming that the thread has thread-local data associated
with it, and reaching at thread-exit time.
In this change we also have an explicit test for two consecutive
sessions for FDR mode tracing, and ensuring both sessions succeed.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D49359
llvm-svn: 337341
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.
Patch by Kodé Williams (@kodewilliams).
Reviewers: metzman, Dor1s, morehouse, kcc
Reviewed By: Dor1s, morehouse, kcc
Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny
Differential Revision: https://reviews.llvm.org/D48054
llvm-svn: 337324
This function is available for linking in from kernel32.dll, but
it's not allowed to link that function from there in Windows Store
apps.
Differential Revision: https://reviews.llvm.org/D49055
llvm-svn: 337313
This is a fix for bug 37047.
https://bugs.llvm.org/show_bug.cgi?id=37047
Implemented by basically reversing the logic. Previously all strings
were considered, with some operations excluded. Now strings are excluded
by default, and only strings during the CB considered.
Patch By: pdknsk
Differential Revision: https://reviews.llvm.org/D48800
llvm-svn: 337296
We no longer pass CLANG_DEFAULT_CXX_STDLIB to the runtimes build
as it was causing issues so we can no longer use this variable. We
instead use cxx-headers as a dependency whenever this is available
since both XRay and libFuzzer are built as static libraries so this
is sufficient.
Differential Revision: https://reviews.llvm.org/D49346
llvm-svn: 337199
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.
Patch by Kodé Williams (@kodewilliams).
Reviewers: metzman, Dor1s, morehouse, kcc
Reviewed By: Dor1s, morehouse, kcc
Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny
Differential Revision: https://reviews.llvm.org/D48054
llvm-svn: 337194
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.
Patch by Kyungtak Woo (@kevinwkt).
Reviewers: metzman, Dor1s, kcc, morehouse
Reviewed By: metzman, Dor1s, morehouse
Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s
Differential Revision: https://reviews.llvm.org/D49212
llvm-svn: 337187
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.
Patch by Kyungtak Woo (@kevinwkt).
Reviewers: metzman, Dor1s, kcc, morehouse
Reviewed By: metzman, Dor1s, morehouse
Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s
Differential Revision: https://reviews.llvm.org/D49212
llvm-svn: 337175
Summary:
Fix a TODO in CMake config for XRay tests to use the detected C++ ABI
library in the tests.
Also make the tests depend on the llvm-xray target when built in-tree.
Reviewers: kpw, eizan
Reviewed By: eizan
Subscribers: mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D49358
llvm-svn: 337142
Summary:
That flag has been introduced in https://reviews.llvm.org/D48660 for
suppressing UIO error messages in an efficient way. The main motivation is to
be able to use UIO checks in builds used for fuzzing as it might provide an
interesting signal to a fuzzing engine such as libFuzzer.
See https://github.com/google/oss-fuzz/issues/910 for more information.
Reviewers: morehouse, kcc
Reviewed By: morehouse
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D49324
llvm-svn: 337068
Summary:
Introduce built-ins to read the unsafe stack top and bottom. The unsafe
stack top is required to implement garbage collection scanning for
Oilpan. Currently there is already a built-in 'get_unsafe_stack_start'
to read the bottom of the unsafe stack, but I chose to duplicate this
API because 'start' is ambiguous (e.g. Oilpan uses WTF::GetStackStart to
read the safe stack top.)
Reviewers: pcc
Reviewed By: pcc
Subscribers: llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D49152
llvm-svn: 337037
Summary:
Use `-Wl,-z,global` for all Sanitizer shared libraries on
Android. We want them to be in the global group
(https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md#changes-to-library-search-order)
to avoid any alloc/dealloc mismatch between the libc allocator & said library.
`audioserver` was one of the binary that exhibited the problem with Scudo,
this seems to fix it.
[edited for accuracy]
Reviewers: eugenis, alekseyshl
Reviewed By: eugenis
Subscribers: kubamracek, srhines, mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D49198
llvm-svn: 337010
Summary:
llvm-xray changes:
- account-mode - process-id {...} shows after thread-id
- convert-mode - process {...} shows after thread
- parses FDR and basic mode pid entries
- Checks version number for FDR log parsing.
Basic logging changes:
- Update header version from 2 -> 3
FDR logging changes:
- Update header version from 2 -> 3
- in writeBufferPreamble, there is an additional PID Metadata record (after thread id record and tsc record)
Test cases changes:
- fdr-mode.cc, fdr-single-thread.cc, fdr-thread-order.cc modified to catch process id output in the log.
Reviewers: dberris
Reviewed By: dberris
Subscribers: hiraditya, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49153
llvm-svn: 336974
Summary:
This change adds support for writing out profiles at program exit.
Depends on D48653.
Reviewers: kpw, eizan
Reviewed By: kpw
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D48956
llvm-svn: 336969
The list duplicates information already available in the parent
directory so use that instead. It is unclear to me why we need
to spell out the dependencies explicitly but fixing that should
be done in a separate patch.
Differential Revision: https://reviews.llvm.org/D49177
llvm-svn: 336905
The variable name `SANITIZER_HEADERS` is already used for the list of
public headers in `include/CMakeLists.txt`. Although the previous
implementation worked it's probably best to avoid shadowing global
variables to avoid confusion.
Differential Revision: https://reviews.llvm.org/D49176
llvm-svn: 336904
Summary:
The write buffer contains signed chars, which means the shift operations caused values such as the arc tag value (0x01a10000) to be read incorrectly (0xffa10000).
This fixes a regression from https://reviews.llvm.org/D49132.
Reviewers: uweigand, davidxl
Reviewed By: uweigand
Subscribers: llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49161
llvm-svn: 336775
Summary: XRayRecords now includes a PID field. Basic handlers fetch pid and tid each time they are called instead of caching the value. Added a testcase that calls fork and checks if the child TID is different from the parent TID to verify that the processes' TID are different in the trace.
Reviewers: dberris, Maknee
Reviewed By: dberris, Maknee
Subscribers: kpw, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49025
llvm-svn: 336769
This reverts rL334510 due to breakage of afl_driver's command line
interface.
Patch By: Jonathan Metzman
Differential Revision: https://reviews.llvm.org/D49141
llvm-svn: 336719
It turns out that the `${XRAY_HEADERS}` CMake variable was already
in use and was used for public headers. It seems that
`lib/xray/tests/CMakeLists.txt` was depending on this.
To fix rename the new `${XRAY_HEADERS}` to `${XRAY_IMPL_HEADERS}`.
llvm-svn: 336699
Two fixes required to handle big-endian systems:
- 64-bit counter values are stored in a mixed-endian format in the
gcov files: a 32-bit low-part followed by a 32-bit high part. Note that
this is already implemented correctly on the LLVM side, see
GCOVBuffer::readInt64.
- The tag values (e.g. arcs tag, object summary tag, ...) are aways
written as the same sequence of bytes independent of byte order. But
when *reading* them back in, the code reads them as 32-bit values in
host byte order. For the comparisons to work correctly, this should
instead always read them as little-endian values.
Fixes PR 38121.
Reviewed By: marco-c
Differential Revision: https://reviews.llvm.org/D49132
llvm-svn: 336693
when building with an IDE so that header files show up in the UI.
This massively improves the development workflow in IDEs.
To implement this a new function `compiler_rt_process_sources(...)` has
been added that adds header files to the list of sources when the
generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no
changes are made to the list of source files.
The function can be passed a list of headers via the
`ADDITIONAL_HEADERS` argument. For each runtime library a list of
explicit header files has been added and passed via
`ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of
headers was already present but it was stale and has been updated
to reflect the current state of the source tree.
The original version of this patch used file globbing (`*.{h,inc,def}`)
to find the headers but the approach was changed due to this being a
CMake anti-pattern (if the list of headers changes CMake won't
automatically re-generate if globbing is used).
The LLVM repo contains a similar function named `llvm_process_sources()`
but we don't use it here for several reasons:
* It depends on the `LLVM_ENABLE_OPTION` cache variable which is
not set in standalone compiler-rt builds.
* We would have to `include(LLVMProcessSources)` which I'd like to
avoid because it would include a bunch of stuff we don't need.
Differential Revision: https://reviews.llvm.org/D48422
llvm-svn: 336663
Changes:
- Remove static assertion on size of a structure, fails on systems where
pointers aren't 8 bytes.
- Use size_t instead of deducing type of arguments to
`nearest_boundary`.
Follow-up to D48653.
llvm-svn: 336648
Summary:
We found a bug while working on a benchmark for the profiling mode which
manifests as a segmentation fault in the profiling handler's
implementation. This change adds unit tests which replicate the
issues in isolation.
We've tracked this down as a bug in the implementation of the Freelist
in the `xray::Array` type. This happens when we trim the array by a
number of elements, where we've been incorrectly assigning pointers for
the links in the freelist of chunk nodes. We've taken the chance to add
more debug-only assertions to the code path and allow us to verify these
assumptions in debug builds.
In the process, we also took the opportunity to use iterators to
implement both `front()` and `back()` which exposes a bug in the
iterator decrement operation. In particular, when we decrement past a
chunk size boundary, we end up moving too far back and reaching the
`SentinelChunk` prematurely.
This change unblocks us to allow for contributing the non-crashing
version of the benchmarks in the test-suite as well.
Reviewers: kpw
Subscribers: mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D48653
llvm-svn: 336644
Code now exists to track number of mutations that are used in fuzzing in
total and ones that produce new coverage. The stats are currently being
dumped to the command line.
Patch By: Kode Williams
Differntial Revision: https://reviews.llvm.org/D48054
llvm-svn: 336597
Summary:
- use proper Error() decorator for error messages
- refactor ASan thread id and name reporting
Reviewers: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D49044
llvm-svn: 336573
Summary: This is a minor cosmetic change. When function/path exceed ~1000 characters, the output is truncated before the line-break. I noticed this for NEW_FUNC.
Reviewers: kcc
Reviewed By: kcc
Subscribers: llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48799
llvm-svn: 336461
Summary:
Namely, set the abort message, and allow to write the message to syslog if the
option is enabled.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48902
llvm-svn: 336445
Summary:
This will make the behavior of __gcov_flush match the GCC behavior.
I would like to rename __gcov_flush to __llvm_gcov_flush (in case of programs linking to libraries built with different compilers), but I guess we can't for compatibility reasons.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: samsonov, vitalybuka, pcc, kcc, junbuml, glider, fhahn, eugenis, dvyukov, davidxl, srhines, chh, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48538
llvm-svn: 336365
Summary:
The C interceptors were using `SIZE_T` defined in the interception library as
a `__sanitizer::uptr`. On some 32-bit platforms, this lead to the following
warning:
```
warning: declaration of ‘void* malloc(SIZE_T)’ conflicts with built-in declaration ‘void* malloc(unsigned int)’ [-Wbuiltin-declaration-mismatch]
INTERCEPTOR_ATTRIBUTE void *malloc(SIZE_T size) {
```
`__sanitizer::uptr` is indeed defined as an `unsigned long` on those.
So just include `stddef.h` and use `size_t` instead.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48885
llvm-svn: 336221
Summary:
It would be useful for Flutter apps, especially, to be able to use
malloc hooks to debug memory leaks on Fuchsia. They're not able to do
this right now, so it'd be a nice bonus to throw in with the Scudo
switchover.
Reviewers: cryptoad, alekseyshl
Reviewed By: cryptoad
Differential Revision: https://reviews.llvm.org/D48618
llvm-svn: 336139
Summary:
Without this patch,
clang -fsanitize=address -xc =(printf 'int main(){}') -o a; ./a => deadlock in __asan_init>AsanInitInternal>AsanTSDInit>...>__getcontextx_size>_rtld_bind>rlock_acquire(rtld_bind_lock, &lockstate)
libexec/rtld-elf/rtld.c
wlock_acquire(rtld_bind_lock, &lockstate);
if (obj_main->crt_no_init)
preinit_main(); // unresolved PLT functions cannot be called here
lib/libthr/thread/thr_rtld.c
uc_len = __getcontextx_size(); // unresolved PLT function in libthr.so.3
check-xray tests currently rely on .preinit_array so we special case in
xray_init.cc
Subscribers: srhines, kubamracek, krytarowski, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48806
llvm-svn: 336067
Currently in FreeBSD 12.0-CURRENT with trunk clang+compiler-rt, faulty -fsanitize=address executable hangs at 'urdlck' state.
Ka Ho Ng has verified that by backporting this to llvm 6.0.1, with use_madv_dontdump=1, shadow memory is not dumped.
ASAN_OPTIONS=abort_on_error=1:disable_coredump=0:use_madv_dontdump=1 ./a
Reviewers: dimitry, kcc, dvyukov, emaste, khng300
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48257
llvm-svn: 336046
__gcov_flush is hidden.
For applications to dump profiling data of selected .so files,
they can use dlsym to find and call llvm_gcov_flush in each .so file.
Differential Revision: https://reviews.llvm.org/D45454
llvm-svn: 336019
Summary:
Report errno value on failed shadow memory madvise attempt and add a
hint message with the possible workaround.
Reviewers: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48668
llvm-svn: 335928
Summary:
Setting UBSAN_OPTIONS=silence_unsigned_overflow=1 will silence all UIO
reports. This feature, combined with
-fsanitize-recover=unsigned-integer-overflow, is useful for providing
fuzzing signal without the excessive log output.
Helps with https://github.com/google/oss-fuzz/issues/910.
Reviewers: kcc, vsk
Reviewed By: vsk
Subscribers: vsk, kubamracek, Dor1s, llvm-commits
Differential Revision: https://reviews.llvm.org/D48660
llvm-svn: 335762
build with an IDE (e.g. Xcode) as the generator.
Previously the global `USE_FOLDERS` property wasn't set in standalone
builds leading to existing settings of FOLDER not being respected.
In addition to this there were several targets that appeared at the top
level that were not interesting and clustered up the view. These have
been changed to be displayed in "Compiler-RT Misc".
Now when an Xcode project is generated from a standalone compiler-rt
build the project navigator is much less cluttered. The interesting
libraries should appear in "Compiler-RT Libraries" in the IDE.
Differential Revision: https://reviews.llvm.org/D48378
llvm-svn: 335728
Some warnings originating from googletest were causing bots to fail
while bulding unit tests. The sanitizers address this issue by not
using -Werror. We adopt this approach for libFuzzer.
llvm-svn: 335640
Similarly to CFI on virtual and indirect calls, this implementation
tries to use program type information to make the checks as precise
as possible. The basic way that it works is as follows, where `C`
is the name of the class being defined or the target of a call and
the function type is assumed to be `void()`.
For virtual calls:
- Attach type metadata to the addresses of function pointers in vtables
(not the functions themselves) of type `void (B::*)()` for each `B`
that is a recursive dynamic base class of `C`, including `C` itself.
This type metadata has an annotation that the type is for virtual
calls (to distinguish it from the non-virtual case).
- At the call site, check that the computed address of the function
pointer in the vtable has type `void (C::*)()`.
For non-virtual calls:
- Attach type metadata to each non-virtual member function whose address
can be taken with a member function pointer. The type of a function
in class `C` of type `void()` is each of the types `void (B::*)()`
where `B` is a most-base class of `C`. A most-base class of `C`
is defined as a recursive base class of `C`, including `C` itself,
that does not have any bases.
- At the call site, check that the function pointer has one of the types
`void (B::*)()` where `B` is a most-base class of `C`.
Differential Revision: https://reviews.llvm.org/D47567
llvm-svn: 335569
Speculative fix for the interface definition of __ubsan_on_report for
the Windows bots:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/30528
lib\ubsan\ubsan_interface.inc(55): error C2065: '__ubsan_on_report':
undeclared identifier
INTERCEPT_SANITIZER_WEAK_FUNCTION was the wrong macro to use to begin
with because __ubsan_on_report isn't weak. Reading through that macro,
it's still not clear to me why there is an undefined reference, though,
because it appears to define a dummy __ubsan_on_report shim.
llvm-svn: 335383
Add support to the ubsan runtime for reporting diagnostics to a monitor
process (e.g a debugger).
The Xcode IDE uses this by setting a breakpoint on __ubsan_on_report and
collecting diagnostic information via __ubsan_get_current_report_data,
which it then surfaces to users in the editor UI.
Testing for this functionality already exists in upstream lldb, here:
lldb/packages/Python/lldbsuite/test/functionalities/ubsan
Apart from that, this is `ninja check-{a,ub}san` clean.
Differential Revision: https://reviews.llvm.org/D48446
llvm-svn: 335371
MutexUnlock uses ReleaseStore on s->clock, which is the right thing to do.
However MutexReadOrWriteUnlock for writers uses Release on s->clock.
Make MutexReadOrWriteUnlock also use ReleaseStore for consistency and performance.
Unfortunately, I don't think any test can detect this as this only potentially
affects performance.
llvm-svn: 335322
Summary:
This patch follows D48373.
The point is to be able to use Scudo with `-fsanitize-minimal-runtime`. For that
we need a runtime that doesn't embed the UBSan one. This results in binaries
that can be compiled with `-fsanitize=scudo,integer -fsanitize-minimal-runtime`.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: mgorny, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48377
llvm-svn: 335296
Kamil Rytarowski