2017-06-07 06:22:41 +08:00
|
|
|
//===- XRayInstrumentation.cpp - Adds XRay instrumentation to functions. --===//
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
//
|
2019-01-19 16:50:56 +08:00
|
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
//
|
|
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
//
|
|
|
|
// This file implements a MachineFunctionPass that inserts the appropriate
|
|
|
|
// XRay instrumentation instructions. We look for XRay-specific attributes
|
|
|
|
// on the function to determine whether we should insert the replacement
|
|
|
|
// operations.
|
|
|
|
//
|
|
|
|
//===---------------------------------------------------------------------===//
|
|
|
|
|
2017-06-07 06:22:41 +08:00
|
|
|
#include "llvm/ADT/STLExtras.h"
|
2017-09-08 09:47:56 +08:00
|
|
|
#include "llvm/ADT/SmallVector.h"
|
2017-06-07 06:22:41 +08:00
|
|
|
#include "llvm/ADT/Triple.h"
|
|
|
|
#include "llvm/CodeGen/MachineBasicBlock.h"
|
2017-06-06 19:49:48 +08:00
|
|
|
#include "llvm/CodeGen/MachineDominators.h"
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
#include "llvm/CodeGen/MachineFunction.h"
|
|
|
|
#include "llvm/CodeGen/MachineFunctionPass.h"
|
|
|
|
#include "llvm/CodeGen/MachineInstrBuilder.h"
|
2017-05-04 09:24:26 +08:00
|
|
|
#include "llvm/CodeGen/MachineLoopInfo.h"
|
2017-11-08 09:01:31 +08:00
|
|
|
#include "llvm/CodeGen/TargetInstrInfo.h"
|
2017-11-17 09:07:10 +08:00
|
|
|
#include "llvm/CodeGen/TargetSubtargetInfo.h"
|
2017-06-07 06:22:41 +08:00
|
|
|
#include "llvm/IR/Attributes.h"
|
|
|
|
#include "llvm/IR/Function.h"
|
Sink all InitializePasses.h includes
This file lists every pass in LLVM, and is included by Pass.h, which is
very popular. Every time we add, remove, or rename a pass in LLVM, it
caused lots of recompilation.
I found this fact by looking at this table, which is sorted by the
number of times a file was changed over the last 100,000 git commits
multiplied by the number of object files that depend on it in the
current checkout:
recompiles touches affected_files header
342380 95 3604 llvm/include/llvm/ADT/STLExtras.h
314730 234 1345 llvm/include/llvm/InitializePasses.h
307036 118 2602 llvm/include/llvm/ADT/APInt.h
213049 59 3611 llvm/include/llvm/Support/MathExtras.h
170422 47 3626 llvm/include/llvm/Support/Compiler.h
162225 45 3605 llvm/include/llvm/ADT/Optional.h
158319 63 2513 llvm/include/llvm/ADT/Triple.h
140322 39 3598 llvm/include/llvm/ADT/StringRef.h
137647 59 2333 llvm/include/llvm/Support/Error.h
131619 73 1803 llvm/include/llvm/Support/FileSystem.h
Before this change, touching InitializePasses.h would cause 1345 files
to recompile. After this change, touching it only causes 550 compiles in
an incremental rebuild.
Reviewers: bkramer, asbirlea, bollu, jdoerfert
Differential Revision: https://reviews.llvm.org/D70211
2019-11-14 05:15:01 +08:00
|
|
|
#include "llvm/InitializePasses.h"
|
2017-06-07 06:22:41 +08:00
|
|
|
#include "llvm/Pass.h"
|
|
|
|
#include "llvm/Target/TargetMachine.h"
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
|
|
|
|
using namespace llvm;
|
|
|
|
|
|
|
|
namespace {
|
2017-06-07 06:22:41 +08:00
|
|
|
|
2017-09-23 02:30:02 +08:00
|
|
|
struct InstrumentationOptions {
|
|
|
|
// Whether to emit PATCHABLE_TAIL_CALL.
|
|
|
|
bool HandleTailcall;
|
|
|
|
|
|
|
|
// Whether to emit PATCHABLE_RET/PATCHABLE_FUNCTION_EXIT for all forms of
|
|
|
|
// return, e.g. conditional return.
|
|
|
|
bool HandleAllReturns;
|
|
|
|
};
|
|
|
|
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
struct XRayInstrumentation : public MachineFunctionPass {
|
|
|
|
static char ID;
|
|
|
|
|
|
|
|
XRayInstrumentation() : MachineFunctionPass(ID) {
|
|
|
|
initializeXRayInstrumentationPass(*PassRegistry::getPassRegistry());
|
|
|
|
}
|
|
|
|
|
2017-05-04 09:24:26 +08:00
|
|
|
void getAnalysisUsage(AnalysisUsage &AU) const override {
|
|
|
|
AU.setPreservesCFG();
|
|
|
|
AU.addPreserved<MachineLoopInfo>();
|
|
|
|
AU.addPreserved<MachineDominatorTree>();
|
|
|
|
MachineFunctionPass::getAnalysisUsage(AU);
|
|
|
|
}
|
|
|
|
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
bool runOnMachineFunction(MachineFunction &MF) override;
|
|
|
|
|
2016-09-19 08:54:35 +08:00
|
|
|
private:
|
|
|
|
// Replace the original RET instruction with the exit sled code ("patchable
|
|
|
|
// ret" pseudo-instruction), so that at runtime XRay can replace the sled
|
|
|
|
// with a code jumping to XRay trampoline, which calls the tracing handler
|
|
|
|
// and, in the end, issues the RET instruction.
|
|
|
|
// This is the approach to go on CPUs which have a single RET instruction,
|
|
|
|
// like x86/x86_64.
|
|
|
|
void replaceRetWithPatchableRet(MachineFunction &MF,
|
2017-09-23 02:30:02 +08:00
|
|
|
const TargetInstrInfo *TII,
|
|
|
|
InstrumentationOptions);
|
2016-11-25 02:51:47 +08:00
|
|
|
|
2016-09-19 08:54:35 +08:00
|
|
|
// Prepend the original return instruction with the exit sled code ("patchable
|
|
|
|
// function exit" pseudo-instruction), preserving the original return
|
|
|
|
// instruction just after the exit sled code.
|
|
|
|
// This is the approach to go on CPUs which have multiple options for the
|
|
|
|
// return instruction, like ARM. For such CPUs we can't just jump into the
|
|
|
|
// XRay trampoline and issue a single return instruction there. We rather
|
|
|
|
// have to call the trampoline and return from it to the original return
|
|
|
|
// instruction of the function being instrumented.
|
|
|
|
void prependRetWithPatchableExit(MachineFunction &MF,
|
2017-09-23 02:30:02 +08:00
|
|
|
const TargetInstrInfo *TII,
|
|
|
|
InstrumentationOptions);
|
2016-09-19 08:54:35 +08:00
|
|
|
};
|
2017-06-07 06:22:41 +08:00
|
|
|
|
|
|
|
} // end anonymous namespace
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
|
2017-05-04 09:24:26 +08:00
|
|
|
void XRayInstrumentation::replaceRetWithPatchableRet(
|
2017-09-23 02:30:02 +08:00
|
|
|
MachineFunction &MF, const TargetInstrInfo *TII,
|
|
|
|
InstrumentationOptions op) {
|
2016-09-19 08:54:35 +08:00
|
|
|
// We look for *all* terminators and returns, then replace those with
|
2016-09-09 01:10:39 +08:00
|
|
|
// PATCHABLE_RET instructions.
|
|
|
|
SmallVector<MachineInstr *, 4> Terminators;
|
|
|
|
for (auto &MBB : MF) {
|
|
|
|
for (auto &T : MBB.terminators()) {
|
|
|
|
unsigned Opc = 0;
|
2017-09-23 02:30:02 +08:00
|
|
|
if (T.isReturn() &&
|
|
|
|
(op.HandleAllReturns || T.getOpcode() == TII->getReturnOpcode())) {
|
2016-09-09 01:10:39 +08:00
|
|
|
// Replace return instructions with:
|
|
|
|
// PATCHABLE_RET <Opcode>, <Operand>...
|
|
|
|
Opc = TargetOpcode::PATCHABLE_RET;
|
|
|
|
}
|
2017-09-23 02:30:02 +08:00
|
|
|
if (TII->isTailCall(T) && op.HandleTailcall) {
|
2016-09-09 01:10:39 +08:00
|
|
|
// Treat the tail call as a return instruction, which has a
|
|
|
|
// different-looking sled than the normal return case.
|
|
|
|
Opc = TargetOpcode::PATCHABLE_TAIL_CALL;
|
|
|
|
}
|
|
|
|
if (Opc != 0) {
|
|
|
|
auto MIB = BuildMI(MBB, T, T.getDebugLoc(), TII->get(Opc))
|
|
|
|
.addImm(T.getOpcode());
|
|
|
|
for (auto &MO : T.operands())
|
2017-01-13 17:58:52 +08:00
|
|
|
MIB.add(MO);
|
2016-09-09 01:10:39 +08:00
|
|
|
Terminators.push_back(&T);
|
2020-02-27 18:44:53 +08:00
|
|
|
if (T.shouldUpdateCallSiteInfo())
|
2019-10-08 23:43:12 +08:00
|
|
|
MF.eraseCallSiteInfo(&T);
|
2016-09-09 01:10:39 +08:00
|
|
|
}
|
|
|
|
}
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
}
|
2016-09-09 01:10:39 +08:00
|
|
|
|
|
|
|
for (auto &I : Terminators)
|
|
|
|
I->eraseFromParent();
|
2016-09-19 08:54:35 +08:00
|
|
|
}
|
|
|
|
|
2017-05-04 09:24:26 +08:00
|
|
|
void XRayInstrumentation::prependRetWithPatchableExit(
|
2017-09-23 02:30:02 +08:00
|
|
|
MachineFunction &MF, const TargetInstrInfo *TII,
|
|
|
|
InstrumentationOptions op) {
|
2017-09-08 09:47:56 +08:00
|
|
|
for (auto &MBB : MF)
|
2017-09-23 02:30:02 +08:00
|
|
|
for (auto &T : MBB.terminators()) {
|
|
|
|
unsigned Opc = 0;
|
|
|
|
if (T.isReturn() &&
|
|
|
|
(op.HandleAllReturns || T.getOpcode() == TII->getReturnOpcode())) {
|
|
|
|
Opc = TargetOpcode::PATCHABLE_FUNCTION_EXIT;
|
|
|
|
}
|
|
|
|
if (TII->isTailCall(T) && op.HandleTailcall) {
|
|
|
|
Opc = TargetOpcode::PATCHABLE_TAIL_CALL;
|
2016-09-19 08:54:35 +08:00
|
|
|
}
|
2017-09-23 02:30:02 +08:00
|
|
|
if (Opc != 0) {
|
|
|
|
// Prepend the return instruction with PATCHABLE_FUNCTION_EXIT or
|
|
|
|
// PATCHABLE_TAIL_CALL .
|
|
|
|
BuildMI(MBB, T, T.getDebugLoc(), TII->get(Opc));
|
|
|
|
}
|
|
|
|
}
|
2016-09-19 08:54:35 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
bool XRayInstrumentation::runOnMachineFunction(MachineFunction &MF) {
|
2017-12-16 06:22:58 +08:00
|
|
|
auto &F = MF.getFunction();
|
2016-09-19 08:54:35 +08:00
|
|
|
auto InstrAttr = F.getFnAttribute("function-instrument");
|
|
|
|
bool AlwaysInstrument = !InstrAttr.hasAttribute(Attribute::None) &&
|
|
|
|
InstrAttr.isStringAttribute() &&
|
|
|
|
InstrAttr.getValueAsString() == "xray-always";
|
2020-01-18 05:24:07 +08:00
|
|
|
auto ThresholdAttr = F.getFnAttribute("xray-instruction-threshold");
|
|
|
|
auto IgnoreLoopsAttr = F.getFnAttribute("xray-ignore-loops");
|
|
|
|
unsigned int XRayThreshold = 0;
|
2016-09-19 08:54:35 +08:00
|
|
|
if (!AlwaysInstrument) {
|
2020-01-18 05:24:07 +08:00
|
|
|
if (ThresholdAttr.hasAttribute(Attribute::None) ||
|
|
|
|
!ThresholdAttr.isStringAttribute())
|
2016-09-19 08:54:35 +08:00
|
|
|
return false; // XRay threshold attribute not found.
|
2020-01-18 05:24:07 +08:00
|
|
|
if (ThresholdAttr.getValueAsString().getAsInteger(10, XRayThreshold))
|
2016-09-19 08:54:35 +08:00
|
|
|
return false; // Invalid value for threshold.
|
2017-05-04 09:24:26 +08:00
|
|
|
|
2020-01-18 05:24:07 +08:00
|
|
|
bool IgnoreLoops = !IgnoreLoopsAttr.hasAttribute(Attribute::None);
|
|
|
|
|
[XRay] Fix computation of function size subject to XRay threshold
Summary:
Currently XRay compares its threshold against `Function::size()` . However, `Function::size()` returns the number of basic blocks (as I understand, such as cycle bodies, if/else bodies, switch-case bodies, etc.), rather than the number of instructions.
The name of the parameter `-fxray-instruction-threshold=N`, as well as XRay documentation at http://llvm.org/docs/XRay.html , suggests that instructions should be counted, rather than the number of basic blocks.
I see two options:
1. Count the number of MachineInstr`s in MachineFunction : this gives better estimate for the number of assembly instructions on the target. So a user can check in disassembly that the threshold works more or less correctly.
2. Count the number of Instruction`s in a Function : AFAIK, this gives correct number of IR instructions, which the user can check in IR listing. However, this number may be far (several times for small functions) from the number of assembly instructions finally emitted.
Option 1 is implemented in this patch because I think that having the closer estimate for the number of assembly instructions emitted is more important than to have a clear definition of the metric.
Reviewers: dberris, rengolin
Reviewed By: dberris
Subscribers: llvm-commits, iid_iunknown
Differential Revision: https://reviews.llvm.org/D34027
llvm-svn: 305072
2017-06-09 21:23:23 +08:00
|
|
|
// Count the number of MachineInstr`s in MachineFunction
|
2017-07-15 05:14:58 +08:00
|
|
|
int64_t MICount = 0;
|
2017-09-08 09:47:56 +08:00
|
|
|
for (const auto &MBB : MF)
|
2017-07-15 05:14:58 +08:00
|
|
|
MICount += MBB.size();
|
[XRay] Fix computation of function size subject to XRay threshold
Summary:
Currently XRay compares its threshold against `Function::size()` . However, `Function::size()` returns the number of basic blocks (as I understand, such as cycle bodies, if/else bodies, switch-case bodies, etc.), rather than the number of instructions.
The name of the parameter `-fxray-instruction-threshold=N`, as well as XRay documentation at http://llvm.org/docs/XRay.html , suggests that instructions should be counted, rather than the number of basic blocks.
I see two options:
1. Count the number of MachineInstr`s in MachineFunction : this gives better estimate for the number of assembly instructions on the target. So a user can check in disassembly that the threshold works more or less correctly.
2. Count the number of Instruction`s in a Function : AFAIK, this gives correct number of IR instructions, which the user can check in IR listing. However, this number may be far (several times for small functions) from the number of assembly instructions finally emitted.
Option 1 is implemented in this patch because I think that having the closer estimate for the number of assembly instructions emitted is more important than to have a clear definition of the metric.
Reviewers: dberris, rengolin
Reviewed By: dberris
Subscribers: llvm-commits, iid_iunknown
Differential Revision: https://reviews.llvm.org/D34027
llvm-svn: 305072
2017-06-09 21:23:23 +08:00
|
|
|
|
2020-01-18 05:24:07 +08:00
|
|
|
bool TooFewInstrs = MICount < XRayThreshold;
|
2018-03-21 01:02:29 +08:00
|
|
|
|
2020-01-18 05:24:07 +08:00
|
|
|
if (!IgnoreLoops) {
|
|
|
|
// Get MachineDominatorTree or compute it on the fly if it's unavailable
|
|
|
|
auto *MDT = getAnalysisIfAvailable<MachineDominatorTree>();
|
|
|
|
MachineDominatorTree ComputedMDT;
|
|
|
|
if (!MDT) {
|
|
|
|
ComputedMDT.getBase().recalculate(MF);
|
|
|
|
MDT = &ComputedMDT;
|
|
|
|
}
|
2018-03-21 01:02:29 +08:00
|
|
|
|
2020-01-18 05:24:07 +08:00
|
|
|
// Get MachineLoopInfo or compute it on the fly if it's unavailable
|
|
|
|
auto *MLI = getAnalysisIfAvailable<MachineLoopInfo>();
|
|
|
|
MachineLoopInfo ComputedMLI;
|
|
|
|
if (!MLI) {
|
|
|
|
ComputedMLI.getBase().analyze(MDT->getBase());
|
|
|
|
MLI = &ComputedMLI;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Check if we have a loop.
|
|
|
|
// FIXME: Maybe make this smarter, and see whether the loops are dependent
|
|
|
|
// on inputs or side-effects?
|
|
|
|
if (MLI->empty() && TooFewInstrs)
|
|
|
|
return false; // Function is too small and has no loops.
|
|
|
|
} else if (TooFewInstrs) {
|
|
|
|
// Function is too small
|
|
|
|
return false;
|
|
|
|
}
|
2016-09-19 08:54:35 +08:00
|
|
|
}
|
2016-09-09 01:10:39 +08:00
|
|
|
|
2016-12-19 17:20:38 +08:00
|
|
|
// We look for the first non-empty MachineBasicBlock, so that we can insert
|
|
|
|
// the function instrumentation in the appropriate place.
|
2017-06-07 06:22:41 +08:00
|
|
|
auto MBI = llvm::find_if(
|
|
|
|
MF, [&](const MachineBasicBlock &MBB) { return !MBB.empty(); });
|
2016-12-19 17:20:38 +08:00
|
|
|
if (MBI == MF.end())
|
|
|
|
return false; // The function is empty.
|
|
|
|
|
|
|
|
auto *TII = MF.getSubtarget().getInstrInfo();
|
|
|
|
auto &FirstMBB = *MBI;
|
2016-09-19 08:54:35 +08:00
|
|
|
auto &FirstMI = *FirstMBB.begin();
|
|
|
|
|
|
|
|
if (!MF.getSubtarget().isXRaySupported()) {
|
|
|
|
FirstMI.emitError("An attempt to perform XRay instrumentation for an"
|
2017-05-04 09:24:26 +08:00
|
|
|
" unsupported target.");
|
2016-09-19 08:54:35 +08:00
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2020-01-18 05:24:27 +08:00
|
|
|
if (!F.hasFnAttribute("xray-skip-entry")) {
|
|
|
|
// First, insert an PATCHABLE_FUNCTION_ENTER as the first instruction of the
|
|
|
|
// MachineFunction.
|
|
|
|
BuildMI(FirstMBB, FirstMI, FirstMI.getDebugLoc(),
|
|
|
|
TII->get(TargetOpcode::PATCHABLE_FUNCTION_ENTER));
|
2016-09-19 08:54:35 +08:00
|
|
|
}
|
2020-01-18 05:24:27 +08:00
|
|
|
|
|
|
|
if (!F.hasFnAttribute("xray-skip-exit")) {
|
|
|
|
switch (MF.getTarget().getTargetTriple().getArch()) {
|
|
|
|
case Triple::ArchType::arm:
|
|
|
|
case Triple::ArchType::thumb:
|
|
|
|
case Triple::ArchType::aarch64:
|
|
|
|
case Triple::ArchType::mips:
|
|
|
|
case Triple::ArchType::mipsel:
|
|
|
|
case Triple::ArchType::mips64:
|
|
|
|
case Triple::ArchType::mips64el: {
|
|
|
|
// For the architectures which don't have a single return instruction
|
|
|
|
InstrumentationOptions op;
|
|
|
|
op.HandleTailcall = false;
|
|
|
|
op.HandleAllReturns = true;
|
|
|
|
prependRetWithPatchableExit(MF, TII, op);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
case Triple::ArchType::ppc64le: {
|
|
|
|
// PPC has conditional returns. Turn them into branch and plain returns.
|
|
|
|
InstrumentationOptions op;
|
|
|
|
op.HandleTailcall = false;
|
|
|
|
op.HandleAllReturns = true;
|
|
|
|
replaceRetWithPatchableRet(MF, TII, op);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
default: {
|
|
|
|
// For the architectures that have a single return instruction (such as
|
|
|
|
// RETQ on x86_64).
|
|
|
|
InstrumentationOptions op;
|
|
|
|
op.HandleTailcall = true;
|
|
|
|
op.HandleAllReturns = false;
|
|
|
|
replaceRetWithPatchableRet(MF, TII, op);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
2017-09-23 02:30:02 +08:00
|
|
|
}
|
XRay: Add entry and exit sleds
Summary:
In this patch we implement the following parts of XRay:
- Supporting a function attribute named 'function-instrument' which currently only supports 'xray-always'. We should be able to use this attribute for other instrumentation approaches.
- Supporting a function attribute named 'xray-instruction-threshold' used to determine whether a function is instrumented with a minimum number of instructions (IR instruction counts).
- X86-specific nop sleds as described in the white paper.
- A machine function pass that adds the different instrumentation marker instructions at a very late stage.
- A way of identifying which return opcode is considered "normal" for each architecture.
There are some caveats here:
1) We don't handle PATCHABLE_RET in platforms other than x86_64 yet -- this means if IR used PATCHABLE_RET directly instead of a normal ret, instruction lowering for that platform might do the wrong thing. We think this should be handled at instruction selection time to by default be unpacked for platforms where XRay is not availble yet.
2) The generated section for X86 is different from what is described from the white paper for the sole reason that LLVM allows us to do this neatly. We're taking the opportunity to deviate from the white paper from this perspective to allow us to get richer information from the runtime library.
Reviewers: sanjoy, eugenis, kcc, pcc, echristo, rnk
Subscribers: niravd, majnemer, atrick, rnk, emaste, bmakam, mcrosier, mehdi_amini, llvm-commits
Differential Revision: http://reviews.llvm.org/D19904
llvm-svn: 275367
2016-07-14 12:06:33 +08:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
char XRayInstrumentation::ID = 0;
|
|
|
|
char &llvm::XRayInstrumentationID = XRayInstrumentation::ID;
|
2017-05-04 09:24:26 +08:00
|
|
|
INITIALIZE_PASS_BEGIN(XRayInstrumentation, "xray-instrumentation",
|
|
|
|
"Insert XRay ops", false, false)
|
|
|
|
INITIALIZE_PASS_DEPENDENCY(MachineLoopInfo)
|
|
|
|
INITIALIZE_PASS_END(XRayInstrumentation, "xray-instrumentation",
|
|
|
|
"Insert XRay ops", false, false)
|