[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
//===-- xray_records.h ------------------------------------------*- C++ -*-===//
|
|
|
|
//
|
2019-01-19 16:50:56 +08:00
|
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
//
|
|
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
//
|
|
|
|
// This file is a part of XRay, a dynamic runtime instrumentation system.
|
|
|
|
//
|
|
|
|
// This header exposes some record types useful for the XRay in-memory logging
|
|
|
|
// implementation.
|
|
|
|
//
|
|
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
|
|
|
|
#ifndef XRAY_XRAY_RECORDS_H
|
|
|
|
#define XRAY_XRAY_RECORDS_H
|
|
|
|
|
2017-08-02 12:51:40 +08:00
|
|
|
#include <cstdint>
|
|
|
|
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
namespace __xray {
|
|
|
|
|
|
|
|
enum FileTypes {
|
|
|
|
NAIVE_LOG = 0,
|
[XRay][compiler-rt] XRay Flight Data Recorder Mode
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
llvm-svn: 293015
2017-01-25 11:50:46 +08:00
|
|
|
FDR_LOG = 1,
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
};
|
|
|
|
|
2017-03-29 13:56:37 +08:00
|
|
|
// FDR mode use of the union field in the XRayFileHeader.
|
|
|
|
struct alignas(16) FdrAdditionalHeaderData {
|
|
|
|
uint64_t ThreadBufferSize;
|
|
|
|
};
|
|
|
|
|
|
|
|
static_assert(sizeof(FdrAdditionalHeaderData) == 16,
|
|
|
|
"FdrAdditionalHeaderData != 16 bytes");
|
|
|
|
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
// This data structure is used to describe the contents of the file. We use this
|
|
|
|
// for versioning the supported XRay file formats.
|
|
|
|
struct alignas(32) XRayFileHeader {
|
|
|
|
uint16_t Version = 0;
|
|
|
|
|
|
|
|
// The type of file we're writing out. See the FileTypes enum for more
|
|
|
|
// information. This allows different implementations of the XRay logging to
|
|
|
|
// have different files for different information being stored.
|
|
|
|
uint16_t Type = 0;
|
|
|
|
|
|
|
|
// What follows are a set of flags that indicate useful things for when
|
|
|
|
// reading the data in the file.
|
|
|
|
bool ConstantTSC : 1;
|
|
|
|
bool NonstopTSC : 1;
|
|
|
|
|
|
|
|
// The frequency by which TSC increases per-second.
|
|
|
|
alignas(8) uint64_t CycleFrequency = 0;
|
[XRay][compiler-rt] XRay Flight Data Recorder Mode
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
llvm-svn: 293015
2017-01-25 11:50:46 +08:00
|
|
|
|
2017-03-29 13:56:37 +08:00
|
|
|
union {
|
|
|
|
char FreeForm[16];
|
2018-05-26 10:28:43 +08:00
|
|
|
// The current civiltime timestamp, as retrieved from 'clock_gettime'. This
|
2017-03-29 13:56:37 +08:00
|
|
|
// allows readers of the file to determine when the file was created or
|
|
|
|
// written down.
|
|
|
|
struct timespec TS;
|
|
|
|
|
|
|
|
struct FdrAdditionalHeaderData FdrData;
|
|
|
|
};
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
} __attribute__((packed));
|
|
|
|
|
|
|
|
static_assert(sizeof(XRayFileHeader) == 32, "XRayFileHeader != 32 bytes");
|
|
|
|
|
|
|
|
enum RecordTypes {
|
|
|
|
NORMAL = 0,
|
2017-10-05 13:45:51 +08:00
|
|
|
ARG_PAYLOAD = 1,
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
};
|
|
|
|
|
|
|
|
struct alignas(32) XRayRecord {
|
|
|
|
// This is the type of the record being written. We use 16 bits to allow us to
|
|
|
|
// treat this as a discriminant, and so that the first 4 bytes get packed
|
|
|
|
// properly. See RecordTypes for more supported types.
|
2017-10-05 13:45:51 +08:00
|
|
|
uint16_t RecordType = RecordTypes::NORMAL;
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
|
|
|
|
// The CPU where the thread is running. We assume number of CPUs <= 256.
|
|
|
|
uint8_t CPU = 0;
|
|
|
|
|
2017-09-18 14:18:03 +08:00
|
|
|
// The type of the event. One of the following:
|
|
|
|
// ENTER = 0
|
|
|
|
// EXIT = 1
|
|
|
|
// TAIL_EXIT = 2
|
2017-10-05 13:45:51 +08:00
|
|
|
// ENTER_ARG = 3
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
uint8_t Type = 0;
|
|
|
|
|
|
|
|
// The function ID for the record.
|
|
|
|
int32_t FuncId = 0;
|
|
|
|
|
|
|
|
// Get the full 8 bytes of the TSC when we get the log record.
|
|
|
|
uint64_t TSC = 0;
|
|
|
|
|
|
|
|
// The thread ID for the currently running thread.
|
|
|
|
uint32_t TId = 0;
|
|
|
|
|
2018-07-11 15:14:27 +08:00
|
|
|
// The ID of process that is currently running
|
|
|
|
uint32_t PId = 0;
|
|
|
|
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
// Use some bytes in the end of the record for buffers.
|
2018-07-11 15:14:27 +08:00
|
|
|
char Buffer[8] = {};
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
} __attribute__((packed));
|
|
|
|
|
|
|
|
static_assert(sizeof(XRayRecord) == 32, "XRayRecord != 32 bytes");
|
|
|
|
|
2017-10-05 13:45:51 +08:00
|
|
|
struct alignas(32) XRayArgPayload {
|
|
|
|
// We use the same 16 bits as a discriminant for the records in the log here
|
|
|
|
// too, and so that the first 4 bytes are packed properly.
|
|
|
|
uint16_t RecordType = RecordTypes::ARG_PAYLOAD;
|
|
|
|
|
|
|
|
// Add a few bytes to pad.
|
|
|
|
uint8_t Padding[2] = {};
|
|
|
|
|
|
|
|
// The function ID for the record.
|
|
|
|
int32_t FuncId = 0;
|
|
|
|
|
|
|
|
// The thread ID for the currently running thread.
|
|
|
|
uint32_t TId = 0;
|
|
|
|
|
2018-07-11 15:14:27 +08:00
|
|
|
// The ID of process that is currently running
|
|
|
|
uint32_t PId = 0;
|
2017-10-05 13:45:51 +08:00
|
|
|
|
|
|
|
// The argument payload.
|
|
|
|
uint64_t Arg = 0;
|
|
|
|
|
|
|
|
// The rest of this record ought to be left as padding.
|
|
|
|
uint8_t TailPadding[8] = {};
|
|
|
|
} __attribute__((packed));
|
|
|
|
|
|
|
|
static_assert(sizeof(XRayArgPayload) == 32, "XRayArgPayload != 32 bytes");
|
|
|
|
|
[compiler-rt][XRay] Initial per-thread inmemory logging implementation
Depends on D21612 which implements the building blocks for the compiler-rt
implementation of the XRay runtime. We use a naive in-memory log of fixed-size
entries that get written out to a log file when the buffers are full, and when
the thread exits.
This implementation lays some foundations on to allowing for more complex XRay
records to be written to the log in subsequent changes. It also defines the format
that the function call accounting tool in D21987 will start building upon.
Once D21987 lands, we should be able to start defining more tests using that tool
once the function call accounting tool becomes part of the llvm distribution.
Reviewers: echristo, kcc, rnk, eugenis, majnemer, rSerge
Subscribers: sdardis, rSerge, dberris, tberghammer, danalbert, srhines, majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D21982
llvm-svn: 279805
2016-08-26 14:39:33 +08:00
|
|
|
} // namespace __xray
|
|
|
|
|
|
|
|
#endif // XRAY_XRAY_RECORDS_H
|