Summary:
- Enabling the build.
- Using assembly for the cpuid parts.
- Using thr_self FreeBSD call to get the thread id
Patch by: David CARLIER
Reviewers: dberris, rnk, krytarowski
Reviewed By: dberris, krytarowski
Subscribers: emaste, stevecheckoway, nglevin, srhines, kubamracek, dberris, mgorny, krytarowski, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D43278
llvm-svn: 325240
Summary:
Before this change, the FDR mode implementation relied on at thread-exit
handling to return buffers back to the (global) buffer queue. This
introduces issues with the initialisation of the thread_local objects
which, even through the use of pthread_setspecific(...) may eventually
call into an allocation function. Similar to previous changes in this
line, we're finding that there is a huge potential for deadlocks when
initialising these thread-locals when the memory allocation
implementation is also xray-instrumented.
In this change, we limit the call to pthread_setspecific(...) to provide
a non-null value to associate to the key created with
pthread_key_create(...). While this doesn't completely eliminate the
potential for the deadlock(s), it does allow us to still clean up at
thread exit when we need to. The change is that we don't need to do more
work when starting and ending a thread's lifetime. We also have a test
to make sure that we actually can safely recycle the buffers in case we
end up re-using the buffer(s) available from the queue on multiple
thread entry/exits.
This change cuts across both LLVM and compiler-rt to allow us to update
both the XRay runtime implementation as well as the library support for
loading these new versions of the FDR mode logging. Version 2 of the FDR
logging implementation makes the following changes:
* Introduction of a new 'BufferExtents' metadata record that's outside
of the buffer's contents but are written before the actual buffer.
This data is associated to the Buffer handed out by the BufferQueue
rather than a record that occupies bytes in the actual buffer.
* Removal of the "end of buffer" records. This is in-line with the
changes we described above, to allow for optimistic logging without
explicit record writing at thread exit.
The optimistic logging model operates under the following assumptions:
* Threads writing to the buffers will potentially race with the thread
attempting to flush the log. To avoid this situation from occuring,
we make sure that when we've finalized the logging implementation,
that threads will see this finalization state on the next write, and
either choose to not write records the thread would have written or
write the record(s) in two phases -- first write the record(s), then
update the extents metadata.
* We change the buffer queue implementation so that once it's handed
out a buffer to a thread, that we assume that buffer is marked
"used" to be able to capture partial writes. None of this will be
safe to handle if threads are racing to write the extents records
and the reader thread is attempting to flush the log. The optimism
comes from the finalization routine being required to complete
before we attempt to flush the log.
This is a fairly significant semantics change for the FDR
implementation. This is why we've decided to update the version number
for FDR mode logs. The tools, however, still need to be able to support
older versions of the log until we finally deprecate those earlier
versions.
Reviewers: dblaikie, pelikan, kpw
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D39526
llvm-svn: 318733
Summary:
This change removes dependencies on STL types:
- std::aligned_storage -- we're using manually-aligned character
buffers instead for metadata and function records.
- std::tuple -- use a plain old struct instead.
This is an incremental step in removing all STL references from the
compiler-rt implementation of XRay (llvm.org/PR32274).
Reviewers: dblaikie, pelikan, kpw
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D39277
llvm-svn: 316816
Summary:
In FDR Mode, when we set up a new buffer for a thread that's just
overflowed, we must place the CPU identifier with the TSC record as the
first record. This is so that we can reconstruct all the function
entry/exit with deltas rooted on a TSC record for the CPU at the
beginning of the buffer.
Without doing this, the tools are rejecting the log for cases when we've
overflown and have different buffers that don't have the CPU and TSC
records as the first entry in the buffers.
Reviewers: pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38995
llvm-svn: 315987
Summary:
We avoid using C++11's thread_local keyword on non-trivially
destructible objects because it may introduce deadlocks when the C++
runtime registers destructors calling std::malloc(...). The deadlock may
happen when the allocator implementation is itself XRay instrumented.
To avoid having to call malloc(...) and free(...) in particular, we use
pthread_once, pthread_create_key, and pthread_setspecific to instead
manually register the cleanup implementation we want.
The code this replaces used an RAII type that implements the cleanup
functionality in the destructor, that was then initialized as a
function-local thread_local object. While it works in usual situations,
unfortunately it breaks when using a malloc implementation that itself
is XRay-instrumented.
Reviewers: dblaikie, kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38073
llvm-svn: 314764
The function was introduced as a convenience that used to be called in
multiple places. Recent refactorings have removed the need to call this
function in multiple places, so inlined the implementation in the single
place it's defined.
Broken out from D38119.
llvm-svn: 314489
Summary:
Write out records about logged function call first arguments. D32840
implements the reading of this in llvm-xray.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D32844
llvm-svn: 314378
Summary:
This change starts differentiating tail exits from normal exits. We also
increase the version number of the "naive" log to version 2, which will
be the starting version where these records start appearing. In FDR mode
we treat the tail exits as normal exits, and are thus subject to the
same treatment with regard to record unwriting.
Updating the version number is important to signal older builds of the
llvm-xray tool that do not deal with the tail exit records must fail
early (and that users should only use the llvm-xray tool built after
the support for tail exits to get accurate handling of these records).
Depends on D37964.
Reviewers: kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D37965
llvm-svn: 313515
Summary:
Before this change, the recursion guard for the flight data recorder
(FDR) mode handlers were independent. This change makes it so that when
a handler is already in the process of running and somehow the same or
another handler starts running -- say in a signal handler, while the
XRay handler is executing -- then we can use the same thread-local
recursion guard to stop the second handler from running.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D37612
llvm-svn: 312992
Summary:
This change hides all the initialization of thread_local variables used
by the XRay FDR mode implementation behind a function call. This makes
initialization of thread-local data to be done lazily, instead of
eagerly when they're done as globals. It also gives us an isolation
mechanism if/when we want to change the TLS implementation from using
the C++ thread_local keyword, for something more ad-hoc (potentialy
using pthread directly) on some platforms or set-ups where we cannot use
the C++ thread_local variables.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D37248
llvm-svn: 311997
Summary:
This change attempts to remove all the dependencies we have on
std::mutex and any std::shared_ptr construction in global variables. We
instead use raw pointers to these objects, and construct them on the
heap. In cases where it's possible, we lazily initialize these pointers.
While we do not have a replacement for std::shared_ptr yet in
compiler-rt, we use this work-around to avoid having to statically
initialize the objects as globals. Subsequent changes should allow us to
completely remove our dependency on std::shared_ptr and instead have our
own implementation of the std::shared_ptr and std::weak_ptr semantics
(or completely rewrite the implementaton to not need these
standard-library provided abstractions).
Reviewers: dblaikie, kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36078
llvm-svn: 309792
Summary:
This change implements support for the custom event logging sleds and
intrinsics at runtime. For now it only supports handling the sleds in
x86_64, with the implementations for other architectures stubbed out to
do nothing.
NOTE: Work in progress, uploaded for exposition/exploration purposes.
Depends on D27503, D30018, and D33032.
Reviewers: echristo, javed.absar, timshen
Subscribers: mehdi_amini, nemanjai, llvm-commits
Differential Revision: https://reviews.llvm.org/D30630
llvm-svn: 302857
Summary:
While there, make the threshold in ticks for the rewind computed only
once and not per function, unify the two versions we had and slightly
reformat bits according to coding standards.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31971
llvm-svn: 300028
Summary:
Not repeating screamy failure paths makes the 300+ line function a bit shorter.
There's no need to overload the variable name "Buffer" if it only works on the
thread local buffer. Fix some comments while there.
I plan to move the rewinding logic into a separate function too, but in this
diff it would be too much of a mess to comprehend. This is trivially NFC.
Reviewers: kpw, dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31930
llvm-svn: 300018
This patch applies a work-around to the XRay FDR tests when TSC emulation is
needed because the processor frequency cannot be determined.
This fixes PR32620 using the suggestion given by Dean in comment 1.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31967
llvm-svn: 300017
Summary:
"short" is defined as an xray flag, and buffer rewinding happens for both exits
and tail exits.
I've made the choice to seek backwards finding pairs of FunctionEntry, TailExit
record pairs and erasing them if the FunctionEntry occurred before exit from the
currently exiting function. This is a compromise so that we don't skip logging
tail calls if the function that they call into takes longer our duration.
This works by counting the consecutive function and function entry, tail exit
pairs that proceed the current point in the buffer. The buffer is rewound to
check whether these entry points happened recently enough to be erased.
It is still possible we will omit them if they call into a child function that
is not instrumented which calls a fast grandchild that is instrumented before
doing other processing.
Reviewers: pelikan, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31345
llvm-svn: 299629
Summary:
Currently the FDR log writer, upon flushing, dumps a sequence of buffers from
its freelist to disk. A reader can read the first buffer up to an EOB record,
but then it is unclear how far ahead to scan to find the next threads traces.
There are a few ways to handle this problem.
1. The reader has externalized knowledge of the buffer size.
2. The size of buffers is in the file header or otherwise encoded in the log.
3. Only write out the portion of the buffer with records. When released, the
buffers are marked with a size.
4. The reader looks for memory that matches a pattern and synchronizes on it.
2 and 3 seem the most flexible and 2 does not rule 3 out.
This is an implementation of 2.
In addition, the function handler for fdr more aggressively checks for
finalization and makes an attempt to release its buffer.
Reviewers: pelikan, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31384
llvm-svn: 298982
In file included from /home/abuild/rpmbuild/BUILD/llvm/projects/compiler-rt/lib/xray/tests/unit/xray_fdr_log_printer_tool.cc:15:
../projects/compiler-rt/lib/xray/tests/../xray_fdr_logging_impl.h:221:21: error: use of undeclared identifier 'CLOCK_MONOTONIC'
wall_clock_reader(CLOCK_MONOTONIC, &TS);
^
1 error generated.
llvm-svn: 298837
Instead of std::atomic APIs for atomic operations, we instead use APIs
include with sanitizer_common. This allows us to, at runtime, not have
to depend on potentially dynamically provided implementations of these
atomic operations.
Fixes http://llvm.org/PR32274.
llvm-svn: 298833
Summary:
Depending on C++11 <system_error> introduces a link-time requirement to
C++11 symbols. Removing it allows us to depend on header-only C++11 and
up libraries.
Partially fixes http://llvm.org/PR32274 -- we know there's more invasive work
to be done, but we're doing it incrementally.
Reviewers: dblaikie, kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31233
llvm-svn: 298480
Summary:
Separated the IO and the thread local storage state machine of logging
from the writing of log records once the contents are deterministic.
Finer granularity functions are provided as inline functions in the same
header such that stack does not grow due to the functions being separated.
An executable utility xray_fdr_log_printer is also implemented to use the
finest granularity functions to produce binary test data in the FDR format
with a relatively convenient text input.
For example, one can take a file with textual contents layed out in rows
and feed it to the binary to generate data that llvm-xray convert can then
read. This is a convenient way to build a test suite for llvm-xray convert
to ensure it's robust to the fdr format.
Example:
$cat myFile.txt
NewBuffer : { time = 2 , Tid=5}
NewCPU : { CPU =1 , TSC = 123}
Function : { FuncId = 5, TSCDelta = 3, EntryType = Entry }
Function : { FuncId = 5, TSCDelta = 5, EntryType = Exit}
TSCWrap : { TSC = 678 }
Function : { FuncId = 6, TSCDelta = 0, EntryType = Entry }
Function : { FuncId = 6, TSCDelta = 50, EntryType = Exit }
EOB : { }
$cat myFile.txt | ./bin/xray_fdr_log_printer > /tmp/binarydata.bin
$./bin/llvm-xray convert -output-format=yaml -output=- /tmp/binarydata.bin
yaml format comes out as expected.
Reviewers: dberris, pelikan
Reviewed By: dberris
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D30850
llvm-svn: 297801
Kamil Rytarowski