llvm-project/llvm/lib/CodeGen/ImplicitNullChecks.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

824 lines
29 KiB
C++
Raw Normal View History

//===- ImplicitNullChecks.cpp - Fold null checks into memory accesses -----===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// This pass turns explicit null checks of the form
//
// test %r10, %r10
// je throw_npe
// movl (%r10), %esi
// ...
//
// to
//
// faulting_load_op("movl (%r10), %esi", throw_npe)
// ...
//
// With the help of a runtime that understands the .fault_maps section,
// faulting_load_op branches to throw_npe if executing movl (%r10), %esi incurs
// a page fault.
// Store and LoadStore are also supported.
//
//===----------------------------------------------------------------------===//
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/None.h"
#include "llvm/ADT/Optional.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/Statistic.h"
#include "llvm/Analysis/AliasAnalysis.h"
#include "llvm/Analysis/MemoryLocation.h"
#include "llvm/CodeGen/FaultMaps.h"
#include "llvm/CodeGen/MachineBasicBlock.h"
#include "llvm/CodeGen/MachineFunction.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineInstr.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
#include "llvm/CodeGen/MachineMemOperand.h"
#include "llvm/CodeGen/MachineOperand.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
#include "llvm/CodeGen/PseudoSourceValue.h"
#include "llvm/CodeGen/TargetInstrInfo.h"
#include "llvm/CodeGen/TargetOpcodes.h"
#include "llvm/CodeGen/TargetRegisterInfo.h"
#include "llvm/CodeGen/TargetSubtargetInfo.h"
#include "llvm/IR/BasicBlock.h"
#include "llvm/IR/DebugLoc.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/InitializePasses.h"
#include "llvm/MC/MCInstrDesc.h"
#include "llvm/MC/MCRegisterInfo.h"
#include "llvm/Pass.h"
#include "llvm/Support/CommandLine.h"
#include <cassert>
#include <cstdint>
#include <iterator>
using namespace llvm;
static cl::opt<int> PageSize("imp-null-check-page-size",
cl::desc("The page size of the target in bytes"),
cl::init(4096), cl::Hidden);
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
static cl::opt<unsigned> MaxInstsToConsider(
"imp-null-max-insts-to-consider",
cl::desc("The max number of instructions to consider hoisting loads over "
"(the algorithm is quadratic over this number)"),
cl::Hidden, cl::init(8));
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
#define DEBUG_TYPE "implicit-null-checks"
STATISTIC(NumImplicitNullChecks,
"Number of explicit null checks made implicit");
namespace {
class ImplicitNullChecks : public MachineFunctionPass {
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
/// Return true if \c computeDependence can process \p MI.
static bool canHandle(const MachineInstr *MI);
/// Helper function for \c computeDependence. Return true if \p A
/// and \p B do not have any dependences between them, and can be
/// re-ordered without changing program semantics.
bool canReorder(const MachineInstr *A, const MachineInstr *B);
/// A data type for representing the result computed by \c
/// computeDependence. States whether it is okay to reorder the
/// instruction passed to \c computeDependence with at most one
2018-09-08 10:04:20 +08:00
/// dependency.
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
struct DependenceResult {
/// Can we actually re-order \p MI with \p Insts (see \c
/// computeDependence).
bool CanReorder;
/// If non-None, then an instruction in \p Insts that also must be
/// hoisted.
Optional<ArrayRef<MachineInstr *>::iterator> PotentialDependence;
/*implicit*/ DependenceResult(
bool CanReorder,
Optional<ArrayRef<MachineInstr *>::iterator> PotentialDependence)
: CanReorder(CanReorder), PotentialDependence(PotentialDependence) {
assert((!PotentialDependence || CanReorder) &&
"!CanReorder && PotentialDependence.hasValue() not allowed!");
}
};
/// Compute a result for the following question: can \p MI be
/// re-ordered from after \p Insts to before it.
///
/// \c canHandle should return true for all instructions in \p
/// Insts.
DependenceResult computeDependence(const MachineInstr *MI,
ArrayRef<MachineInstr *> Block);
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
/// Represents one null check that can be made implicit.
class NullCheck {
// The memory operation the null check can be folded into.
MachineInstr *MemOperation;
// The instruction actually doing the null check (Ptr != 0).
MachineInstr *CheckOperation;
// The block the check resides in.
MachineBasicBlock *CheckBlock;
// The block branched to if the pointer is non-null.
MachineBasicBlock *NotNullSucc;
// The block branched to if the pointer is null.
MachineBasicBlock *NullSucc;
// If this is non-null, then MemOperation has a dependency on this
// instruction; and it needs to be hoisted to execute before MemOperation.
MachineInstr *OnlyDependency;
public:
explicit NullCheck(MachineInstr *memOperation, MachineInstr *checkOperation,
MachineBasicBlock *checkBlock,
MachineBasicBlock *notNullSucc,
MachineBasicBlock *nullSucc,
MachineInstr *onlyDependency)
: MemOperation(memOperation), CheckOperation(checkOperation),
CheckBlock(checkBlock), NotNullSucc(notNullSucc), NullSucc(nullSucc),
OnlyDependency(onlyDependency) {}
MachineInstr *getMemOperation() const { return MemOperation; }
MachineInstr *getCheckOperation() const { return CheckOperation; }
MachineBasicBlock *getCheckBlock() const { return CheckBlock; }
MachineBasicBlock *getNotNullSucc() const { return NotNullSucc; }
MachineBasicBlock *getNullSucc() const { return NullSucc; }
MachineInstr *getOnlyDependency() const { return OnlyDependency; }
};
const TargetInstrInfo *TII = nullptr;
const TargetRegisterInfo *TRI = nullptr;
AliasAnalysis *AA = nullptr;
MachineFrameInfo *MFI = nullptr;
bool analyzeBlockForNullChecks(MachineBasicBlock &MBB,
SmallVectorImpl<NullCheck> &NullCheckList);
MachineInstr *insertFaultingInstr(MachineInstr *MI, MachineBasicBlock *MBB,
MachineBasicBlock *HandlerMBB);
void rewriteNullChecks(ArrayRef<NullCheck> NullCheckList);
enum AliasResult {
AR_NoAlias,
AR_MayAlias,
AR_WillAliasEverything
};
/// Returns AR_NoAlias if \p MI memory operation does not alias with
/// \p PrevMI, AR_MayAlias if they may alias and AR_WillAliasEverything if
/// they may alias and any further memory operation may alias with \p PrevMI.
AliasResult areMemoryOpsAliased(const MachineInstr &MI,
const MachineInstr *PrevMI) const;
enum SuitabilityResult {
SR_Suitable,
SR_Unsuitable,
SR_Impossible
};
/// Return SR_Suitable if \p MI a memory operation that can be used to
/// implicitly null check the value in \p PointerReg, SR_Unsuitable if
/// \p MI cannot be used to null check and SR_Impossible if there is
/// no sense to continue lookup due to any other instruction will not be able
/// to be used. \p PrevInsts is the set of instruction seen since
/// the explicit null check on \p PointerReg.
SuitabilityResult isSuitableMemoryOp(const MachineInstr &MI,
unsigned PointerReg,
ArrayRef<MachineInstr *> PrevInsts);
/// Returns true if \p DependenceMI can clobber the liveIns in NullSucc block
/// if it was hoisted to the NullCheck block. This is used by caller
/// canHoistInst to decide if DependenceMI can be hoisted safely.
bool canDependenceHoistingClobberLiveIns(MachineInstr *DependenceMI,
MachineBasicBlock *NullSucc);
/// Return true if \p FaultingMI can be hoisted from after the
/// instructions in \p InstsSeenSoFar to before them. Set \p Dependence to a
/// non-null value if we also need to (and legally can) hoist a dependency.
bool canHoistInst(MachineInstr *FaultingMI,
ArrayRef<MachineInstr *> InstsSeenSoFar,
MachineBasicBlock *NullSucc, MachineInstr *&Dependence);
public:
static char ID;
ImplicitNullChecks() : MachineFunctionPass(ID) {
initializeImplicitNullChecksPass(*PassRegistry::getPassRegistry());
}
bool runOnMachineFunction(MachineFunction &MF) override;
void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<AAResultsWrapperPass>();
MachineFunctionPass::getAnalysisUsage(AU);
}
MachineFunctionProperties getRequiredProperties() const override {
return MachineFunctionProperties().set(
MachineFunctionProperties::Property::NoVRegs);
}
};
} // end anonymous namespace
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
bool ImplicitNullChecks::canHandle(const MachineInstr *MI) {
Allow target to handle STRICT floating-point nodes The ISD::STRICT_ nodes used to implement the constrained floating-point intrinsics are currently never passed to the target back-end, which makes it impossible to handle them correctly (e.g. mark instructions are depending on a floating-point status and control register, or mark instructions as possibly trapping). This patch allows the target to use setOperationAction to switch the action on ISD::STRICT_ nodes to Legal. If this is done, the SelectionDAG common code will stop converting the STRICT nodes to regular floating-point nodes, but instead pass the STRICT nodes to the target using normal SelectionDAG matching rules. To avoid having the back-end duplicate all the floating-point instruction patterns to handle both strict and non-strict variants, we make the MI codegen explicitly aware of the floating-point exceptions by introducing two new concepts: - A new MCID flag "mayRaiseFPException" that the target should set on any instruction that possibly can raise FP exception according to the architecture definition. - A new MI flag FPExcept that CodeGen/SelectionDAG will set on any MI instruction resulting from expansion of any constrained FP intrinsic. Any MI instruction that is *both* marked as mayRaiseFPException *and* FPExcept then needs to be considered as raising exceptions by MI-level codegen (e.g. scheduling). Setting those two new flags is straightforward. The mayRaiseFPException flag is simply set via TableGen by marking all relevant instruction patterns in the .td files. The FPExcept flag is set in SDNodeFlags when creating the STRICT_ nodes in the SelectionDAG, and gets inherited in the MachineSDNode nodes created from it during instruction selection. The flag is then transfered to an MIFlag when creating the MI from the MachineSDNode. This is handled just like fast-math flags like no-nans are handled today. This patch includes both common code changes required to implement the new features, and the SystemZ implementation. Reviewed By: andrew.w.kaylor Differential Revision: https://reviews.llvm.org/D55506 llvm-svn: 362663
2019-06-06 06:33:10 +08:00
if (MI->isCall() || MI->mayRaiseFPException() ||
MI->hasUnmodeledSideEffects())
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
return false;
auto IsRegMask = [](const MachineOperand &MO) { return MO.isRegMask(); };
(void)IsRegMask;
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
assert(!llvm::any_of(MI->operands(), IsRegMask) &&
"Calls were filtered out above!");
auto IsUnordered = [](MachineMemOperand *MMO) { return MMO->isUnordered(); };
return llvm::all_of(MI->memoperands(), IsUnordered);
}
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
ImplicitNullChecks::DependenceResult
ImplicitNullChecks::computeDependence(const MachineInstr *MI,
ArrayRef<MachineInstr *> Block) {
assert(llvm::all_of(Block, canHandle) && "Check this first!");
assert(!is_contained(Block, MI) && "Block must be exclusive of MI!");
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
Optional<ArrayRef<MachineInstr *>::iterator> Dep;
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
for (auto I = Block.begin(), E = Block.end(); I != E; ++I) {
if (canReorder(*I, MI))
continue;
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
if (Dep == None) {
// Found one possible dependency, keep track of it.
Dep = I;
} else {
// We found two dependencies, so bail out.
return {false, None};
}
}
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
return {true, Dep};
}
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
bool ImplicitNullChecks::canReorder(const MachineInstr *A,
const MachineInstr *B) {
assert(canHandle(A) && canHandle(B) && "Precondition!");
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
// canHandle makes sure that we _can_ correctly analyze the dependencies
// between A and B here -- for instance, we should not be dealing with heap
// load-store dependencies here.
for (const auto &MOA : A->operands()) {
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
if (!(MOA.isReg() && MOA.getReg()))
continue;
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
Register RegA = MOA.getReg();
for (const auto &MOB : B->operands()) {
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
if (!(MOB.isReg() && MOB.getReg()))
continue;
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
Register RegB = MOB.getReg();
if (TRI->regsOverlap(RegA, RegB) && (MOA.isDef() || MOB.isDef()))
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
return false;
}
}
return true;
}
bool ImplicitNullChecks::runOnMachineFunction(MachineFunction &MF) {
TII = MF.getSubtarget().getInstrInfo();
TRI = MF.getRegInfo().getTargetRegisterInfo();
MFI = &MF.getFrameInfo();
AA = &getAnalysis<AAResultsWrapperPass>().getAAResults();
SmallVector<NullCheck, 16> NullCheckList;
for (auto &MBB : MF)
analyzeBlockForNullChecks(MBB, NullCheckList);
if (!NullCheckList.empty())
rewriteNullChecks(NullCheckList);
return !NullCheckList.empty();
}
// Return true if any register aliasing \p Reg is live-in into \p MBB.
static bool AnyAliasLiveIn(const TargetRegisterInfo *TRI,
MachineBasicBlock *MBB, unsigned Reg) {
for (MCRegAliasIterator AR(Reg, TRI, /*IncludeSelf*/ true); AR.isValid();
++AR)
if (MBB->isLiveIn(*AR))
return true;
return false;
}
ImplicitNullChecks::AliasResult
ImplicitNullChecks::areMemoryOpsAliased(const MachineInstr &MI,
const MachineInstr *PrevMI) const {
// If it is not memory access, skip the check.
if (!(PrevMI->mayStore() || PrevMI->mayLoad()))
return AR_NoAlias;
// Load-Load may alias
if (!(MI.mayStore() || PrevMI->mayStore()))
return AR_NoAlias;
// We lost info, conservatively alias. If it was store then no sense to
// continue because we won't be able to check against it further.
if (MI.memoperands_empty())
return MI.mayStore() ? AR_WillAliasEverything : AR_MayAlias;
if (PrevMI->memoperands_empty())
return PrevMI->mayStore() ? AR_WillAliasEverything : AR_MayAlias;
for (MachineMemOperand *MMO1 : MI.memoperands()) {
// MMO1 should have a value due it comes from operation we'd like to use
// as implicit null check.
assert(MMO1->getValue() && "MMO1 should have a Value!");
for (MachineMemOperand *MMO2 : PrevMI->memoperands()) {
if (const PseudoSourceValue *PSV = MMO2->getPseudoValue()) {
if (PSV->mayAlias(MFI))
return AR_MayAlias;
continue;
}
llvm::AliasResult AAResult = AA->alias(
MemoryLocation::getAfter(MMO1->getValue(), MMO1->getAAInfo()),
MemoryLocation::getAfter(MMO2->getValue(), MMO2->getAAInfo()));
if (AAResult != NoAlias)
return AR_MayAlias;
}
}
return AR_NoAlias;
}
ImplicitNullChecks::SuitabilityResult
ImplicitNullChecks::isSuitableMemoryOp(const MachineInstr &MI,
unsigned PointerReg,
ArrayRef<MachineInstr *> PrevInsts) {
2020-09-18 06:39:50 +08:00
// Implementation restriction for faulting_op insertion
// TODO: This could be relaxed if we find a test case which warrants it.
if (MI.getDesc().getNumDefs() > 1)
return SR_Unsuitable;
if (!MI.mayLoadOrStore() || MI.isPredicable())
return SR_Unsuitable;
auto AM = TII->getAddrModeFromMemoryOp(MI, TRI);
if (!AM)
2020-09-18 06:39:50 +08:00
return SR_Unsuitable;
auto AddrMode = *AM;
const Register BaseReg = AddrMode.BaseReg, ScaledReg = AddrMode.ScaledReg;
int64_t Displacement = AddrMode.Displacement;
// We need the base of the memory instruction to be same as the register
// where the null check is performed (i.e. PointerReg).
if (BaseReg != PointerReg && ScaledReg != PointerReg)
return SR_Unsuitable;
const MachineRegisterInfo &MRI = MI.getMF()->getRegInfo();
unsigned PointerRegSizeInBits = TRI->getRegSizeInBits(PointerReg, MRI);
// Bail out of the sizes of BaseReg, ScaledReg and PointerReg are not the
// same.
if ((BaseReg &&
TRI->getRegSizeInBits(BaseReg, MRI) != PointerRegSizeInBits) ||
(ScaledReg &&
TRI->getRegSizeInBits(ScaledReg, MRI) != PointerRegSizeInBits))
return SR_Unsuitable;
// Returns true if RegUsedInAddr is used for calculating the displacement
// depending on addressing mode. Also calculates the Displacement.
auto CalculateDisplacementFromAddrMode = [&](Register RegUsedInAddr,
int64_t Multiplier) {
// The register can be NoRegister, which is defined as zero for all targets.
// Consider instruction of interest as `movq 8(,%rdi,8), %rax`. Here the
// ScaledReg is %rdi, while there is no BaseReg.
if (!RegUsedInAddr)
return false;
assert(Multiplier && "expected to be non-zero!");
MachineInstr *ModifyingMI = nullptr;
for (auto It = std::next(MachineBasicBlock::const_reverse_iterator(&MI));
It != MI.getParent()->rend(); It++) {
const MachineInstr *CurrMI = &*It;
if (CurrMI->modifiesRegister(RegUsedInAddr, TRI)) {
ModifyingMI = const_cast<MachineInstr *>(CurrMI);
break;
}
}
if (!ModifyingMI)
return false;
// Check for the const value defined in register by ModifyingMI. This means
// all other previous values for that register has been invalidated.
int64_t ImmVal;
if (!TII->getConstValDefinedInReg(*ModifyingMI, RegUsedInAddr, ImmVal))
return false;
// Calculate the reg size in bits, since this is needed for bailing out in
// case of overflow.
int32_t RegSizeInBits = TRI->getRegSizeInBits(RegUsedInAddr, MRI);
APInt ImmValC(RegSizeInBits, ImmVal, true /*IsSigned*/);
APInt MultiplierC(RegSizeInBits, Multiplier);
assert(MultiplierC.isStrictlyPositive() &&
"expected to be a positive value!");
bool IsOverflow;
// Sign of the product depends on the sign of the ImmVal, since Multiplier
// is always positive.
APInt Product = ImmValC.smul_ov(MultiplierC, IsOverflow);
if (IsOverflow)
return false;
APInt DisplacementC(64, Displacement, true /*isSigned*/);
DisplacementC = Product.sadd_ov(DisplacementC, IsOverflow);
if (IsOverflow)
return false;
// We only handle diplacements upto 64 bits wide.
if (DisplacementC.getActiveBits() > 64)
return false;
Displacement = DisplacementC.getSExtValue();
return true;
};
// If a register used in the address is constant, fold it's effect into the
// displacement for ease of analysis.
bool BaseRegIsConstVal = false, ScaledRegIsConstVal = false;
if (CalculateDisplacementFromAddrMode(BaseReg, 1))
BaseRegIsConstVal = true;
if (CalculateDisplacementFromAddrMode(ScaledReg, AddrMode.Scale))
ScaledRegIsConstVal = true;
// The register which is not null checked should be part of the Displacement
// calculation, otherwise we do not know whether the Displacement is made up
// by some symbolic values.
// This matters because we do not want to incorrectly assume that load from
// falls in the zeroth faulting page in the "sane offset check" below.
if ((BaseReg && BaseReg != PointerReg && !BaseRegIsConstVal) ||
(ScaledReg && ScaledReg != PointerReg && !ScaledRegIsConstVal))
return SR_Unsuitable;
// We want the mem access to be issued at a sane offset from PointerReg,
// so that if PointerReg is null then the access reliably page faults.
if (!(-PageSize < Displacement && Displacement < PageSize))
return SR_Unsuitable;
// Finally, check whether the current memory access aliases with previous one.
for (auto *PrevMI : PrevInsts) {
AliasResult AR = areMemoryOpsAliased(MI, PrevMI);
if (AR == AR_WillAliasEverything)
return SR_Impossible;
if (AR == AR_MayAlias)
return SR_Unsuitable;
}
return SR_Suitable;
}
bool ImplicitNullChecks::canDependenceHoistingClobberLiveIns(
MachineInstr *DependenceMI, MachineBasicBlock *NullSucc) {
for (const auto &DependenceMO : DependenceMI->operands()) {
if (!(DependenceMO.isReg() && DependenceMO.getReg()))
continue;
// Make sure that we won't clobber any live ins to the sibling block by
// hoisting Dependency. For instance, we can't hoist INST to before the
// null check (even if it safe, and does not violate any dependencies in
// the non_null_block) if %rdx is live in to _null_block.
//
// test %rcx, %rcx
// je _null_block
// _non_null_block:
[CodeGen] Use MachineOperand::print in the MIRPrinter for MO_Register. Work towards the unification of MIR and debug output by refactoring the interfaces. For MachineOperand::print, keep a simple version that can be easily called from `dump()`, and a more complex one which will be called from both the MIRPrinter and MachineInstr::print. Add extra checks inside MachineOperand for detached operands (operands with getParent() == nullptr). https://reviews.llvm.org/D40836 * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/kill: ([^ ]+) ([^ ]+)<def> ([^ ]+)/kill: \1 def \2 \3/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/kill: ([^ ]+) ([^ ]+) ([^ ]+)<def>/kill: \1 \2 def \3/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/kill: def ([^ ]+) ([^ ]+) ([^ ]+)<def>/kill: def \1 \2 def \3/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/<def>//g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<kill>/killed \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<imp-use,kill>/implicit killed \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<dead>/dead \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<def[ ]*,[ ]*dead>/dead \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<imp-def[ ]*,[ ]*dead>/implicit-def dead \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<imp-def>/implicit-def \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<imp-use>/implicit \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<internal>/internal \1/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" -o -name "*.s" \) -type f -print0 | xargs -0 sed -i '' -E 's/([^ ]+)<undef>/undef \1/g' llvm-svn: 320022
2017-12-07 18:40:31 +08:00
// %rdx = INST
// ...
//
// This restriction does not apply to the faulting load inst because in
// case the pointer loaded from is in the null page, the load will not
// semantically execute, and affect machine state. That is, if the load
// was loading into %rax and it faults, the value of %rax should stay the
// same as it would have been had the load not have executed and we'd have
// branched to NullSucc directly.
if (AnyAliasLiveIn(TRI, NullSucc, DependenceMO.getReg()))
return true;
}
// The dependence does not clobber live-ins in NullSucc block.
return false;
}
bool ImplicitNullChecks::canHoistInst(MachineInstr *FaultingMI,
ArrayRef<MachineInstr *> InstsSeenSoFar,
MachineBasicBlock *NullSucc,
MachineInstr *&Dependence) {
auto DepResult = computeDependence(FaultingMI, InstsSeenSoFar);
if (!DepResult.CanReorder)
return false;
if (!DepResult.PotentialDependence) {
Dependence = nullptr;
return true;
}
auto DependenceItr = *DepResult.PotentialDependence;
auto *DependenceMI = *DependenceItr;
// We don't want to reason about speculating loads. Note -- at this point
// we should have already filtered out all of the other non-speculatable
// things, like calls and stores.
// We also do not want to hoist stores because it might change the memory
// while the FaultingMI may result in faulting.
assert(canHandle(DependenceMI) && "Should never have reached here!");
if (DependenceMI->mayLoadOrStore())
return false;
if (canDependenceHoistingClobberLiveIns(DependenceMI, NullSucc))
return false;
auto DepDepResult =
computeDependence(DependenceMI, {InstsSeenSoFar.begin(), DependenceItr});
if (!DepDepResult.CanReorder || DepDepResult.PotentialDependence)
return false;
Dependence = DependenceMI;
return true;
}
/// Analyze MBB to check if its terminating branch can be turned into an
/// implicit null check. If yes, append a description of the said null check to
/// NullCheckList and return true, else return false.
bool ImplicitNullChecks::analyzeBlockForNullChecks(
MachineBasicBlock &MBB, SmallVectorImpl<NullCheck> &NullCheckList) {
using MachineBranchPredicate = TargetInstrInfo::MachineBranchPredicate;
MDNode *BranchMD = nullptr;
if (auto *BB = MBB.getBasicBlock())
BranchMD = BB->getTerminator()->getMetadata(LLVMContext::MD_make_implicit);
if (!BranchMD)
return false;
MachineBranchPredicate MBP;
if (TII->analyzeBranchPredicate(MBB, MBP, true))
return false;
// Is the predicate comparing an integer to zero?
if (!(MBP.LHS.isReg() && MBP.RHS.isImm() && MBP.RHS.getImm() == 0 &&
(MBP.Predicate == MachineBranchPredicate::PRED_NE ||
MBP.Predicate == MachineBranchPredicate::PRED_EQ)))
return false;
2020-09-18 06:39:50 +08:00
// If there is a separate condition generation instruction, we chose not to
// transform unless we can remove both condition and consuming branch.
if (MBP.ConditionDef && !MBP.SingleUseCondition)
return false;
MachineBasicBlock *NotNullSucc, *NullSucc;
if (MBP.Predicate == MachineBranchPredicate::PRED_NE) {
NotNullSucc = MBP.TrueDest;
NullSucc = MBP.FalseDest;
} else {
NotNullSucc = MBP.FalseDest;
NullSucc = MBP.TrueDest;
}
// We handle the simplest case for now. We can potentially do better by using
// the machine dominator tree.
if (NotNullSucc->pred_size() != 1)
return false;
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
const Register PointerReg = MBP.LHS.getReg();
2020-09-18 06:39:50 +08:00
if (MBP.ConditionDef) {
// To prevent the invalid transformation of the following code:
//
// mov %rax, %rcx
// test %rax, %rax
// %rax = ...
// je throw_npe
// mov(%rcx), %r9
// mov(%rax), %r10
//
// into:
//
// mov %rax, %rcx
// %rax = ....
// faulting_load_op("movl (%rax), %r10", throw_npe)
// mov(%rcx), %r9
//
// we must ensure that there are no instructions between the 'test' and
// conditional jump that modify %rax.
assert(MBP.ConditionDef->getParent() == &MBB &&
"Should be in basic block");
2020-09-18 06:39:50 +08:00
for (auto I = MBB.rbegin(); MBP.ConditionDef != &*I; ++I)
if (I->modifiesRegister(PointerReg, TRI))
return false;
}
// Starting with a code fragment like:
//
// test %rax, %rax
// jne LblNotNull
//
// LblNull:
// callq throw_NullPointerException
//
// LblNotNull:
// Inst0
// Inst1
// ...
// Def = Load (%rax + <offset>)
// ...
//
//
// we want to end up with
//
// Def = FaultingLoad (%rax + <offset>), LblNull
// jmp LblNotNull ;; explicit or fallthrough
//
// LblNotNull:
// Inst0
// Inst1
// ...
//
// LblNull:
// callq throw_NullPointerException
//
//
// To see why this is legal, consider the two possibilities:
//
// 1. %rax is null: since we constrain <offset> to be less than PageSize, the
// load instruction dereferences the null page, causing a segmentation
// fault.
//
// 2. %rax is not null: in this case we know that the load cannot fault, as
// otherwise the load would've faulted in the original program too and the
// original program would've been undefined.
//
// This reasoning cannot be extended to justify hoisting through arbitrary
// control flow. For instance, in the example below (in pseudo-C)
//
// if (ptr == null) { throw_npe(); unreachable; }
// if (some_cond) { return 42; }
// v = ptr->field; // LD
// ...
//
// we cannot (without code duplication) use the load marked "LD" to null check
// ptr -- clause (2) above does not apply in this case. In the above program
// the safety of ptr->field can be dependent on some_cond; and, for instance,
// ptr could be some non-null invalid reference that never gets loaded from
// because some_cond is always true.
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
SmallVector<MachineInstr *, 8> InstsSeenSoFar;
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
for (auto &MI : *NotNullSucc) {
if (!canHandle(&MI) || InstsSeenSoFar.size() >= MaxInstsToConsider)
return false;
MachineInstr *Dependence;
SuitabilityResult SR = isSuitableMemoryOp(MI, PointerReg, InstsSeenSoFar);
if (SR == SR_Impossible)
return false;
if (SR == SR_Suitable &&
canHoistInst(&MI, InstsSeenSoFar, NullSucc, Dependence)) {
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
NullCheckList.emplace_back(&MI, MBP.ConditionDef, &MBB, NotNullSucc,
NullSucc, Dependence);
return true;
}
// If MI re-defines the PointerReg in a way that changes the value of
// PointerReg if it was null, then we cannot move further.
if (!TII->preservesZeroValueInReg(&MI, PointerReg, TRI))
return false;
Reimplement depedency tracking in the ImplicitNullChecks pass Summary: This change rewrites a core component in the ImplicitNullChecks pass for greater simplicity since the original design was over-complicated for no good reason. Please review this as essentially a new pass. The change is almost NFC and I've added a test case for a scenario that this new code handles that wasn't handled earlier. The implicit null check pass, at its core, is a code hoisting transform. It differs from "normal" code transforms in that it speculates potentially faulting instructions (by design), but a lot of the usual hazard detection logic (register read-after-write etc.) still applies. We previously detected hazards by keeping track of registers defined and used by machine instructions over an instruction range, but that was unwieldy and did not actually confer any performance benefits. The intent was to have linear time complexity over the number of machine instructions considered, but it ended up being N^2 is practice. This new version is more obviously O(N^2) (with N capped to 8 by default) in hazard detection. It does not attempt to be clever in tracking register uses or defs (the previous cleverness here was a source of bugs). Once this is checked in, I'll extract out the `IsSuitableMemoryOp` and `CanHoistLoadInst` lambda into member functions (they're too complicated to be inline lambdas) and do some other related NFC cleanups. Reviewers: reames, anna, atrick Subscribers: mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D27592 llvm-svn: 290394
2016-12-23 08:41:21 +08:00
InstsSeenSoFar.push_back(&MI);
}
return false;
}
/// Wrap a machine instruction, MI, into a FAULTING machine instruction.
/// The FAULTING instruction does the same load/store as MI
/// (defining the same register), and branches to HandlerMBB if the mem access
/// faults. The FAULTING instruction is inserted at the end of MBB.
MachineInstr *ImplicitNullChecks::insertFaultingInstr(
MachineInstr *MI, MachineBasicBlock *MBB, MachineBasicBlock *HandlerMBB) {
const unsigned NoRegister = 0; // Guaranteed to be the NoRegister value for
// all targets.
DebugLoc DL;
unsigned NumDefs = MI->getDesc().getNumDefs();
assert(NumDefs <= 1 && "other cases unhandled!");
unsigned DefReg = NoRegister;
if (NumDefs != 0) {
DefReg = MI->getOperand(0).getReg();
assert(NumDefs == 1 && "expected exactly one def!");
}
FaultMaps::FaultKind FK;
if (MI->mayLoad())
FK =
MI->mayStore() ? FaultMaps::FaultingLoadStore : FaultMaps::FaultingLoad;
else
FK = FaultMaps::FaultingStore;
auto MIB = BuildMI(MBB, DL, TII->get(TargetOpcode::FAULTING_OP), DefReg)
.addImm(FK)
.addMBB(HandlerMBB)
.addImm(MI->getOpcode());
for (auto &MO : MI->uses()) {
if (MO.isReg()) {
MachineOperand NewMO = MO;
if (MO.isUse()) {
NewMO.setIsKill(false);
} else {
assert(MO.isDef() && "Expected def or use");
NewMO.setIsDead(false);
}
MIB.add(NewMO);
} else {
MIB.add(MO);
}
}
[MI] Change the array of `MachineMemOperand` pointers to be a generically extensible collection of extra info attached to a `MachineInstr`. The primary change here is cleaning up the APIs used for setting and manipulating the `MachineMemOperand` pointer arrays so chat we can change how they are allocated. Then we introduce an extra info object that using the trailing object pattern to attach some number of MMOs but also other extra info. The design of this is specifically so that this extra info has a fixed necessary cost (the header tracking what extra info is included) and everything else can be tail allocated. This pattern works especially well with a `BumpPtrAllocator` which we use here. I've also added the basic scaffolding for putting interesting pointers into this, namely pre- and post-instruction symbols. These aren't used anywhere yet, they're just there to ensure I've actually gotten the data structure types correct. I'll flesh out support for these in a subsequent patch (MIR dumping, parsing, the works). Finally, I've included an optimization where we store any single pointer inline in the `MachineInstr` to avoid the allocation overhead. This is expected to be the overwhelmingly most common case and so should avoid any memory usage growth due to slightly less clever / dense allocation when dealing with >1 MMO. This did require several ergonomic improvements to the `PointerSumType` to reasonably support the various usage models. This also has a side effect of freeing up 8 bits within the `MachineInstr` which could be repurposed for something else. The suggested direction here came largely from Hal Finkel. I hope it was worth it. ;] It does hopefully clear a path for subsequent extensions w/o nearly as much leg work. Lots of thanks to Reid and Justin for careful reviews and ideas about how to do all of this. Differential Revision: https://reviews.llvm.org/D50701 llvm-svn: 339940
2018-08-17 05:30:05 +08:00
MIB.setMemRefs(MI->memoperands());
return MIB;
}
/// Rewrite the null checks in NullCheckList into implicit null checks.
void ImplicitNullChecks::rewriteNullChecks(
ArrayRef<ImplicitNullChecks::NullCheck> NullCheckList) {
DebugLoc DL;
for (auto &NC : NullCheckList) {
// Remove the conditional branch dependent on the null check.
unsigned BranchesRemoved = TII->removeBranch(*NC.getCheckBlock());
(void)BranchesRemoved;
assert(BranchesRemoved > 0 && "expected at least one branch!");
if (auto *DepMI = NC.getOnlyDependency()) {
DepMI->removeFromParent();
NC.getCheckBlock()->insert(NC.getCheckBlock()->end(), DepMI);
}
// Insert a faulting instruction where the conditional branch was
// originally. We check earlier ensures that this bit of code motion
// is legal. We do not touch the successors list for any basic block
// since we haven't changed control flow, we've just made it implicit.
MachineInstr *FaultingInstr = insertFaultingInstr(
NC.getMemOperation(), NC.getCheckBlock(), NC.getNullSucc());
// Now the values defined by MemOperation, if any, are live-in of
// the block of MemOperation.
// The original operation may define implicit-defs alongside
// the value.
MachineBasicBlock *MBB = NC.getMemOperation()->getParent();
for (const MachineOperand &MO : FaultingInstr->operands()) {
if (!MO.isReg() || !MO.isDef())
continue;
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
Register Reg = MO.getReg();
if (!Reg || MBB->isLiveIn(Reg))
continue;
MBB->addLiveIn(Reg);
}
if (auto *DepMI = NC.getOnlyDependency()) {
for (auto &MO : DepMI->operands()) {
if (!MO.isReg() || !MO.getReg() || !MO.isDef() || MO.isDead())
continue;
if (!NC.getNotNullSucc()->isLiveIn(MO.getReg()))
NC.getNotNullSucc()->addLiveIn(MO.getReg());
}
}
NC.getMemOperation()->eraseFromParent();
2020-09-18 06:39:50 +08:00
if (auto *CheckOp = NC.getCheckOperation())
CheckOp->eraseFromParent();
2020-09-18 06:39:50 +08:00
// Insert an *unconditional* branch to not-null successor - we expect
// block placement to remove fallthroughs later.
TII->insertBranch(*NC.getCheckBlock(), NC.getNotNullSucc(), nullptr,
/*Cond=*/None, DL);
NumImplicitNullChecks++;
}
}
char ImplicitNullChecks::ID = 0;
char &llvm::ImplicitNullChecksID = ImplicitNullChecks::ID;
INITIALIZE_PASS_BEGIN(ImplicitNullChecks, DEBUG_TYPE,
"Implicit null checks", false, false)
INITIALIZE_PASS_DEPENDENCY(AAResultsWrapperPass)
INITIALIZE_PASS_END(ImplicitNullChecks, DEBUG_TYPE,
"Implicit null checks", false, false)