Summary:
This change implements support for the custom event logging sleds and
intrinsics at runtime. For now it only supports handling the sleds in
x86_64, with the implementations for other architectures stubbed out to
do nothing.
NOTE: Work in progress, uploaded for exposition/exploration purposes.
Depends on D27503, D30018, and D33032.
Reviewers: echristo, javed.absar, timshen
Subscribers: mehdi_amini, nemanjai, llvm-commits
Differential Revision: https://reviews.llvm.org/D30630
llvm-svn: 302857
Summary:
This change allows us to provide users and implementers of XRay handlers
a means of converting XRay function id's to addresses. This, in
combination with the facilities provided in D32695, allows users to find
out:
- How many function id's there are defined in the current binary.
- Get the address of the function associated with this function id.
- Patch only specific functions according to their requirements.
While we don't directly provide symbolization support in XRay, having
the function's address lets users determine this information easily
either during runtime, or offline with tools like 'addr2line'.
Reviewers: dblaikie, echristo, pelikan
Subscribers: kpw, llvm-commits
Differential Revision: https://reviews.llvm.org/D32846
llvm-svn: 302210
Summary:
This change allows us to patch/unpatch specific functions using the
function ID. This is useful in cases where implementations might want to
do coverage-style, or more fine-grained control of which functions to
patch or un-patch at runtime.
Depends on D32693.
Reviewers: dblaikie, echristo, kpw
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D32695
llvm-svn: 302112
This patch allows the Swift compiler to emit calls to `__tsan_external_write` before starting any modifying access, which will cause TSan to detect races on arrays, dictionaries and other classes defined in non-instrumented modules. Races on collections from the Swift standard library and user-defined structs and a frequent cause of subtle bugs and it's important that TSan detects those on top of existing LLVM IR instrumentation, which already detects races in direct memory accesses.
Differential Revision: https://reviews.llvm.org/D31630
llvm-svn: 302050
For a linker init mutex with lazy flag setup
(no __tsan_mutex_create call), it is possible that
no lock/unlock happened before the destroy call.
Then when destroy runs we still don't know that
it is a linker init mutex and will emulate a memory write.
This in turn can lead to false positives as the mutex
is in fact linker initialized.
Support linker init flag in destroy annotation to resolve this.
llvm-svn: 301795
Summary:
In this patch we document the requirements for implementations that want
to install handlers for the dynamically-controlled XRay "framework".
This clarifies what the expectations are for implementations that
want to install their handlers using this API (similar to how the FDR
logging implementation does so). It also gives users some guarantees on
semantics for the APIs.
If all goes well, users can decide to use the XRay APIs to control the
tracing/logging at the application level, without having to depend on
implementation details of the installed logging implementation. This
lets users choose the implementation that comes with compiler-rt, or
potentially multiple other implementations that use the same APIs.
We also add one convenience function (__xray_remove_log_impl()) for
explicitly removing the currently installed log implementation.
Reviewers: kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D32579
llvm-svn: 301784
Summary:
Currently the FDR log writer, upon flushing, dumps a sequence of buffers from
its freelist to disk. A reader can read the first buffer up to an EOB record,
but then it is unclear how far ahead to scan to find the next threads traces.
There are a few ways to handle this problem.
1. The reader has externalized knowledge of the buffer size.
2. The size of buffers is in the file header or otherwise encoded in the log.
3. Only write out the portion of the buffer with records. When released, the
buffers are marked with a size.
4. The reader looks for memory that matches a pattern and synchronizes on it.
2 and 3 seem the most flexible and 2 does not rule 3 out.
This is an implementation of 2.
In addition, the function handler for fdr more aggressively checks for
finalization and makes an attempt to release its buffer.
Reviewers: pelikan, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31384
llvm-svn: 298982
Summary:
This change exercises the end-to-end functionality defined in the FDR
logging implementation. We also prepare for being able to run traces
generated by the FDR logging implementation from being analysed with the
llvm-xray command that comes with the LLVM distribution.
This also unblocks D31385, D31384, and D31345.
Reviewers: kpw, pelikan
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D31452
llvm-svn: 298977
There are several problems with the current annotations (AnnotateRWLockCreate and friends):
- they don't fully support deadlock detection (we need a hook _before_ mutex lock)
- they don't support insertion of random artificial delays to perturb execution (again we need a hook _before_ mutex lock)
- they don't support setting extended mutex attributes like read/write reentrancy (only "linker init" was bolted on)
- they don't support setting mutex attributes if a mutex don't have a "constructor" (e.g. static, Java, Go mutexes)
- they don't ignore synchronization inside of lock/unlock operations which leads to slowdown and false negatives
The new annotations solve of the above problems. See tsan_interface.h for the interface specification and comments.
Reviewed in https://reviews.llvm.org/D31093
llvm-svn: 298809
Summary:
Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled
in a way to (optionally) pass their first call argument to your logging handler.
For practical and performance reasons, only the first argument is supported, and
only up to 64 bits.
Reviewers: javed.absar, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29703
llvm-svn: 297000
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
llvm-svn: 293015
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
llvm-svn: 290852
Summary: The function computes full module name and coverts pc into offset.
Reviewers: kcc
Subscribers: kubabrecka
Differential Revision: https://reviews.llvm.org/D26820
llvm-svn: 288711
Summary:
This change depends on D23986 which adds tail call-specific sleds. For
now we treat them first as normal exits, and in the future leave room
for implementing this as a different kind of log entry.
The reason for deferring the change is so that we can keep the naive
logging implementation more accurate without additional complexity for
reading the log. The accuracy is gained in effectively interpreting call
stacks like:
A()
B()
C()
Which when tail-call merged will end up not having any exit entries for
A() nor B(), but effectively in turn can be reasoned about as:
A()
B()
C()
Although we lose the fact that A() had called B() then had called C()
with the naive approach, a later iteration that adds the explicit tail
call entries would be a change in the log format and thus necessitate a
version change for the header. We can do this later to have a chance at
releasing some tools (in D21987) that are able to handle the naive log
format, then support higher version numbers of the log format too.
Reviewers: echristo, kcc, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, dberris
Differential Revision: https://reviews.llvm.org/D23988
llvm-svn: 284178
This patch extends __sanitizer_finish_switch_fiber method to optionally return previous stack base and size.
This solves the problem of coroutines/fibers library not knowing the original stack context from which the library is used. It's incorrect to assume that such context is always the default stack of current thread (e.g. one such library may be used from a fiber/coroutine created by another library). Bulding a separate stack tracking mechanism would not only duplicate AsanThread, but also require each coroutines/fibers library to integrate with it.
Author: Andrii Grynenko (andriigrynenko)
Reviewed in: https://reviews.llvm.org/D24628
llvm-svn: 282582
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
This addresses some comments from D21612, which contains the following changes:
- Update __xray_patch() and __xray_unpatch() API documentation to not imply asynchrony.
- Introduce a scope cleanup mechanism to make sure we can roll-back changes to the XRayPatching global atomic.
- Introduce a few more comments for potential extension points for other platforms (for the implementation details of patching and un-patching).
Reviewers: eugenis, rnk, kcc, echristo, majnemer
Subscribers: llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D22911
llvm-svn: 277124
Summary:
This is a fixed-up version of D21612, to address failure identified post-commit.
Original commit description:
This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting).
Fixes include:
- Gating XRay build to only Linux x86_64 and with the right dependencies in case it is the only library being built
- Including <cstddef> to fix std::size_t issue
Reviewers: kcc, rnk, echristo
Subscribers: mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D22611
llvm-svn: 276251
and also the follow-up "[xray] Only build xray on Linux for now"
Two build errors were reported on the llvm-commits list:
[ 88%] Building CXX object lib/xray/CMakeFiles/clang_rt.xray-x86_64.dir/xray_flags.cc.o
/mnt/b/sanitizer-buildbot1/sanitizer-x86_64-linux/build/llvm/projects/compiler-rt/lib/xray/xray_init.cc:23:10: fatal error: 'llvm/Support/ELF.h' file not found
#include "llvm/Support/ELF.h"
^
and
In file included from /w/src/llvm.org/projects/compiler-rt/lib/xray/xray_interface.cc:16:
/w/src/llvm.org/projects/compiler-rt/lib/xray/xray_interface_internal.h:36:8: error:
no type named 'size_t' in namespace 'std'
std::size_t Entries;
~~~~~^
llvm-svn: 276186
Summary:
This patch implements the initialisation and patching routines for the XRay runtime, along with the necessary trampolines for function entry/exit handling. For now we only define the basic hooks for allowing an implementation to define a handler that gets run on function entry/exit. We expose a minimal API for controlling the behaviour of the runtime (patching, cleanup, and setting the handler to invoke when instrumenting).
Depends on D19904
Reviewers: echristo, kcc, rnk
Subscribers: rnk, mehdi_amini, llvm-commits
Differential Revision: https://reviews.llvm.org/D21612
llvm-svn: 276117
Adds a new esan public interface routine __esan_get_sample_count() and uses
it to ensure that tests of sampling receive the minimum number of samples.
llvm-svn: 275948
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.
Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.
This patch doesn't contain behavior changes.
Reviewers: kubabrecka, rnk
Subscribers: wang0109, llvm-commits, kubabrecka, chrisha
Differential Revision: http://reviews.llvm.org/D21952
llvm-svn: 275111
Summary:
Adds a new public interface routine __esan_report() which can be used to
request profiling results prior to abnormal termination (e.g., for a server
process killed by its parent where the normal exit does not allow for
normal result reporting).
Implements this for the working-set tool. The cache frag tool is left
unimplemented as it requires missing iteration capabilities.
Adds a new test.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D22098
llvm-svn: 274964
This patch adds the __sanitizer_start_switch_fiber and
__sanitizer_finish_switch_fiber methods inspired from what can be found here
2ea64dd249 .
These methods are needed when the compiled software needs to implement
coroutines, fibers or the like. Without a way to annotate them, when the program
jumps to a stack that is not the thread stack, __asan_handle_no_return shows a
warning about that, and the fake stack mechanism may free fake frames that are
still in use.
Author: blastrock (Philippe Daouadi)
Reviewed in http://reviews.llvm.org/D20913
llvm-svn: 273260
Kostya Serebryany