maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/CodeGen/IfConversion.cpp

2253 lines
84 KiB
C++
Raw Normal View History

Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
//===-- IfConversion.cpp - Machine code if conversion pass. ---------------===//
//
// The LLVM Compiler Infrastructure
//
Remove attribution from file headers, per discussion on llvmdev. llvm-svn: 45418
2007-12-30 04:36:04 +08:00
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
//
//===----------------------------------------------------------------------===//
//
[ifcnv] Add brief comment explaining what ifcnv is. llvm-svn: 265088
2016-04-01 09:09:03 +08:00
// This file implements the machine instruction level if-conversion pass, which
// tries to convert conditional branches into predicated instructions.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
//
//===----------------------------------------------------------------------===//
#include "llvm/CodeGen/Passes.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "BranchFolding.h"
#include "llvm/ADT/STLExtras.h"
IfConversion: Handle inclusive ranges more carefully. This may affect calculations for thresholds, but is not a significant change in behavior. The problem was that an inclusive range must have an additonal flag to showr that it is empty, because otherwise begin == end implies that the range has one element, and it may not be possible to move past on either side. llvm-svn: 279166
2016-08-19 06:09:23 +08:00
#include "llvm/ADT/ScopeExit.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/ADT/SmallSet.h"
#include "llvm/ADT/Statistic.h"
Re-sort all of the includes with ./utils/sort_includes.py so that subsequent changes are easier to review. About to fix some layering issues, and wanted to separate out the necessary churn. Also comment and sink the include of "Windows.h" in three .inc files to match the usage in Memory.inc. llvm-svn: 198685
2014-01-07 19:48:04 +08:00
#include "llvm/CodeGen/LivePhysRegs.h"
[Branch probability] Recompute branch weights of tail-merged basic blocks. BranchFolderPass was not correctly setting the basic block branch weights when tail-merging created or merged blocks. This patch recomutes the weights of tail-merged blocks using the following formula: branch_weight(merged block to successor j) = sum(block_frequency(bb) * branch_probability(bb -> j)) bb is a block that is in the set of merged blocks. <rdar://problem/16256423> llvm-svn: 215135
2014-08-08 03:30:13 +08:00
#include "llvm/CodeGen/MachineBlockFrequencyInfo.h"
Fix typo in #include which revealed in the case-sensitive filesystem. llvm-svn: 136828
2011-08-04 06:53:41 +08:00
#include "llvm/CodeGen/MachineBranchProbabilityInfo.h"
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
#include "llvm/CodeGen/MachineFunctionPass.h"
Use MachineInstrBuilder in a few CodeGen passes. This automatically passes a context pointer to MI->addOperand(). llvm-svn: 170711
2012-12-21 02:08:06 +08:00
#include "llvm/CodeGen/MachineInstrBuilder.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/CodeGen/MachineModuleInfo.h"
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
#include "llvm/CodeGen/MachineRegisterInfo.h"
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
#include "llvm/CodeGen/TargetSchedule.h"
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
#include "llvm/Support/CommandLine.h"
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
#include "llvm/Support/Debug.h"
Convert more assert(0)+abort() -> LLVM_UNREACHABLE, and abort()/exit() -> llvm_report_error(). llvm-svn: 75363
2009-07-11 21:10:19 +08:00
#include "llvm/Support/ErrorHandling.h"
#include "llvm/Support/raw_ostream.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/Target/TargetInstrInfo.h"
#include "llvm/Target/TargetLowering.h"
#include "llvm/Target/TargetRegisterInfo.h"
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
#include "llvm/Target/TargetSubtargetInfo.h"
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
#include <algorithm>
Apply clang-tidy's misc-move-constructor-init throughout LLVM. No functionality change intended, maybe a tiny performance improvement. llvm-svn: 270997
2016-05-27 22:27:24 +08:00
#include <utility>
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
using namespace llvm;
[Modules] Remove potential ODR violations by sinking the DEBUG_TYPE define below all header includes in the lib/CodeGen/... tree. While the current modules implementation doesn't check for this kind of ODR violation yet, it is likely to grow support for it in the future. It also removes one layer of macro pollution across all the included headers. Other sub-trees will follow. llvm-svn: 206837
2014-04-22 10:02:50 +08:00
#define DEBUG_TYPE "ifcvt"
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
// Hidden options for help debugging.
static cl::opt<int> IfCvtFnStart("ifcvt-fn-start", cl::init(-1), cl::Hidden);
static cl::opt<int> IfCvtFnStop("ifcvt-fn-stop", cl::init(-1), cl::Hidden);
static cl::opt<int> IfCvtLimit("ifcvt-limit", cl::init(-1), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableSimple("disable-ifcvt-simple",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableSimpleF("disable-ifcvt-simple-false",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangle("disable-ifcvt-triangle",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangleR("disable-ifcvt-triangle-rev",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangleF("disable-ifcvt-triangle-false",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangleFR("disable-ifcvt-triangle-false-rev",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableDiamond("disable-ifcvt-diamond",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
static cl::opt<bool> DisableForkedDiamond("disable-ifcvt-forked-diamond",
cl::init(false), cl::Hidden);
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
static cl::opt<bool> IfCvtBranchFold("ifcvt-branch-fold",
cl::init(true), cl::Hidden);
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
STATISTIC(NumSimple, "Number of simple if-conversions performed");
STATISTIC(NumSimpleFalse, "Number of simple (F) if-conversions performed");
STATISTIC(NumTriangle, "Number of triangle if-conversions performed");
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
STATISTIC(NumTriangleRev, "Number of triangle (R) if-conversions performed");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
STATISTIC(NumTriangleFalse,"Number of triangle (F) if-conversions performed");
STATISTIC(NumTriangleFRev, "Number of triangle (F/R) if-conversions performed");
STATISTIC(NumDiamonds, "Number of diamond if-conversions performed");
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
STATISTIC(NumForkedDiamonds, "Number of forked-diamond if-conversions performed");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
STATISTIC(NumIfConvBBs, "Number of if-converted blocks");
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
STATISTIC(NumDupBBs, "Number of duplicated blocks");
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
STATISTIC(NumUnpred, "Number of true blocks of diamonds unpredicated");
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
namespace {
Remove VISIBILITY_HIDDEN from class/struct found inside anonymous namespaces. Chris claims we should never have visibility_hidden inside any .cpp file but that's still not true even after this commit. llvm-svn: 85042
2009-10-25 14:33:48 +08:00
class IfConverter : public MachineFunctionPass {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
enum IfcvtKind {
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
ICNotClassfied, // BB data valid, but not classified.
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
ICSimpleFalse, // Same as ICSimple, but on the false path.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
ICSimple, // BB is entry of an one split, no rejoin sub-CFG.
ICTriangleFRev, // Same as ICTriangleFalse, but false path rev condition.
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
ICTriangleRev, // Same as ICTriangle, but true path rev condition.
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
ICTriangleFalse, // Same as ICTriangle, but on the false path.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
ICTriangle, // BB is entry of a triangle sub-CFG.
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
ICDiamond, // BB is entry of a diamond sub-CFG.
ICForkedDiamond // BB is entry of an almost diamond sub-CFG, with a
// common tail that can be shared.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
};
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// One per MachineBasicBlock, this is used to cache the result
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
/// if-conversion feasibility analysis. This includes results from
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
/// TargetInstrInfo::analyzeBranch() (i.e. TBB, FBB, and Cond), and its
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
/// classification, and common tail block of its successors (if it's a
Some restructuring in preparation for most aggressive if-conversion. llvm-svn: 37231
2007-05-19 02:14:37 +08:00
/// diamond shape), its size, whether it's predicable, and whether any
/// instruction can clobber the 'would-be' predicate.
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
///
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
/// IsDone - True if BB is not to be considered for ifcvt.
/// IsBeingAnalyzed - True if BB is currently being analyzed.
/// IsAnalyzed - True if BB has been analyzed (info is still valid).
/// IsEnqueued - True if BB has been enqueued to be ifcvt'ed.
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
/// IsBrAnalyzable - True if analyzeBranch() returns false.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
/// HasFallThrough - True if BB may fallthrough to the following BB.
/// IsUnpredicable - True if BB is known to be unpredicable.
Somehow this wasn't committed last time. M_CLOBBERS_PRED is gone. llvm-svn: 38495
2007-07-11 01:50:43 +08:00
/// ClobbersPred - True if BB could modify predicates (e.g. has
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
/// cmp, call, etc.)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
/// NonPredSize - Number of non-predicated instructions.
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
/// ExtraCost - Extra cost for multi-cycle instructions.
/// ExtraCost2 - Some instructions are slower when predicated
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
/// BB - Corresponding MachineBasicBlock.
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
/// TrueBB / FalseBB- See analyzeBranch().
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
/// BrCond - Conditions for end of block conditional branches.
/// Predicate - Predicate used in the BB.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
struct BBInfo {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool IsDone : 1;
bool IsBeingAnalyzed : 1;
bool IsAnalyzed : 1;
bool IsEnqueued : 1;
bool IsBrAnalyzable : 1;
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool IsBrReversible : 1;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool HasFallThrough : 1;
bool IsUnpredicable : 1;
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
bool CannotBeCopied : 1;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool ClobbersPred : 1;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
unsigned NonPredSize;
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
unsigned ExtraCost;
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
unsigned ExtraCost2;
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
MachineBasicBlock *BB;
MachineBasicBlock *TrueBB;
MachineBasicBlock *FalseBB;
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> BrCond;
SmallVector<MachineOperand, 4> Predicate;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
BBInfo() : IsDone(false), IsBeingAnalyzed(false),
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
IsAnalyzed(false), IsEnqueued(false), IsBrAnalyzable(false),
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
IsBrReversible(false), HasFallThrough(false),
IsUnpredicable(false), CannotBeCopied(false),
ClobbersPred(false), NonPredSize(0), ExtraCost(0),
ExtraCost2(0), BB(nullptr), TrueBB(nullptr),
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
FalseBB(nullptr) {}
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
};
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Record information about pending if-conversions to attempt:
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
/// BBI - Corresponding BBInfo.
/// Kind - Type of block. See IfcvtKind.
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
/// NeedSubsumption - True if the to-be-predicated BB has already been
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
/// predicated.
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
/// NumDups - Number of instructions that would be duplicated due
/// to this if-conversion. (For diamonds, the number of
/// identical instructions at the beginnings of both
/// paths).
/// NumDups2 - For diamonds, the number of identical instructions
/// at the ends of both paths.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
struct IfcvtToken {
BBInfo &BBI;
IfcvtKind Kind;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
unsigned NumDups;
unsigned NumDups2;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool NeedSubsumption : 1;
bool TClobbersPred : 1;
bool FClobbersPred : 1;
IfcvtToken(BBInfo &b, IfcvtKind k, bool s, unsigned d, unsigned d2 = 0,
bool tc = false, bool fc = false)
: BBI(b), Kind(k), NumDups(d), NumDups2(d2), NeedSubsumption(s),
TClobbersPred(tc), FClobbersPred(fc) {}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
};
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Results of if-conversion feasibility analysis indexed by basic block
/// number.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
std::vector<BBInfo> BBAnalysis;
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
TargetSchedModel SchedModel;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Split TargetLowering into a CodeGen and a SelectionDAG part. This fixes some of the cycles between libCodeGen and libSelectionDAG. It's still a complete mess but as long as the edges consist of virtual call it doesn't cause breakage. BasicTTI did static calls and thus broke some build configurations. llvm-svn: 172246
2013-01-12 04:05:37 +08:00
const TargetLoweringBase *TLI;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
const TargetInstrInfo *TII;
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
const TargetRegisterInfo *TRI;
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
const MachineBranchProbabilityInfo *MBPI;
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
MachineRegisterInfo *MRI;
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
Convert register liveness tracking to work on a sub-register level instead of just register units. Reviewed by Andy llvm-svn: 197315
2013-12-14 14:52:56 +08:00
LivePhysRegs Redefs;
LivePhysRegs DontKill;
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
bool PreRegAlloc;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
bool MadeChange;
Fewer static variables, part 3 of many. llvm-svn: 74140
2009-06-25 07:41:44 +08:00
int FnNum;
CodeGen/Passes: Pass MachineFunction as functor arg; NFC Passing a MachineFunction as argument is more natural and avoids an unnecessary round-trip through the logic determining the correct Subtarget because MachineFunction already has a reference anyway. llvm-svn: 285039
2016-10-25 07:23:02 +08:00
std::function<bool(const MachineFunction &)> PredicateFtor;
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
public:
static char ID;
CodeGen/Passes: Pass MachineFunction as functor arg; NFC Passing a MachineFunction as argument is more natural and avoids an unnecessary round-trip through the logic determining the correct Subtarget because MachineFunction already has a reference anyway. llvm-svn: 285039
2016-10-25 07:23:02 +08:00
IfConverter(std::function<bool(const MachineFunction &)> Ftor = nullptr)
Apply clang-tidy's misc-move-constructor-init throughout LLVM. No functionality change intended, maybe a tiny performance improvement. llvm-svn: 270997
2016-05-27 22:27:24 +08:00
: MachineFunctionPass(ID), FnNum(-1), PredicateFtor(std::move(Ftor)) {
Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which must be called in the pass's constructor. This function uses static dependency declarations to recursively initialize the pass's dependencies. Clients that only create passes through the createFooPass() APIs will require no changes. Clients that want to use the CommandLine options for passes will need to manually call the appropriate initialization functions in PassInitialization.h before parsing commandline arguments. I have tested this with all standard configurations of clang and llvm-gcc on Darwin. It is possible that there are problems with the static dependencies that will only be visible with non-standard options. If you encounter any crash in pass registration/creation, please send the testcase to me directly. llvm-svn: 116820
2010-10-20 01:21:58 +08:00
initializeIfConverterPass(*PassRegistry::getPassRegistry());
}
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
[C++11] Add 'override' keyword to virtual methods that override their base class. llvm-svn: 203220
2014-03-07 17:26:03 +08:00
void getAnalysisUsage(AnalysisUsage &AU) const override {
[Branch probability] Recompute branch weights of tail-merged basic blocks. BranchFolderPass was not correctly setting the basic block branch weights when tail-merging created or merged blocks. This patch recomutes the weights of tail-merged blocks using the following formula: branch_weight(merged block to successor j) = sum(block_frequency(bb) * branch_probability(bb -> j)) bb is a block that is in the set of merged blocks. <rdar://problem/16256423> llvm-svn: 215135
2014-08-08 03:30:13 +08:00
AU.addRequired<MachineBlockFrequencyInfo>();
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
AU.addRequired<MachineBranchProbabilityInfo>();
Give the if-converter access to MachineLoopInfo, and use it to generate plausible branch prediction estimates. llvm-svn: 114981
2010-09-29 04:42:15 +08:00
MachineFunctionPass::getAnalysisUsage(AU);
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
[C++11] Add 'override' keyword to virtual methods that override their base class. llvm-svn: 203220
2014-03-07 17:26:03 +08:00
bool runOnMachineFunction(MachineFunction &MF) override;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Add MachineFunctionProperty checks for AllVRegsAllocated for target passes Summary: This adds the same checks that were added in r264593 to all target-specific passes that run after register allocation. Reviewers: qcolombet Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D18525 llvm-svn: 265313
2016-04-05 01:09:25 +08:00
MachineFunctionProperties getRequiredProperties() const override {
return MachineFunctionProperties().set(
MachineFunctionProperties/MIRParser: Rename AllVRegsAllocated->NoVRegs, compute it Rename AllVRegsAllocated to NoVRegs. This avoids the connotation of running after register and simply describes that no vregs are used in a machine function. With that we can simply compute the property and do not need to dump/parse it in .mir files. Differential Revision: http://reviews.llvm.org/D23850 llvm-svn: 279698
2016-08-25 09:27:13 +08:00
MachineFunctionProperties::Property::NoVRegs);
Add MachineFunctionProperty checks for AllVRegsAllocated for target passes Summary: This adds the same checks that were added in r264593 to all target-specific passes that run after register allocation. Reviewers: qcolombet Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D18525 llvm-svn: 265313
2016-04-05 01:09:25 +08:00
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
private:
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
bool reverseBranchCondition(BBInfo &BBI) const;
Thread the determination of branch prediction hit rates back through the if-conversion heuristic APIs. For now, stick with a constant estimate of 90% (branch predictors are good!), but we might find that we want to provide more nuanced estimates in the future. llvm-svn: 115364
2010-10-02 06:45:50 +08:00
bool ValidSimple(BBInfo &TrueBBI, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const;
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
bool ValidTriangle(BBInfo &TrueBBI, BBInfo &FalseBBI,
Thread the determination of branch prediction hit rates back through the if-conversion heuristic APIs. For now, stick with a constant estimate of 90% (branch predictors are good!), but we might find that we want to provide more nuanced estimates in the future. llvm-svn: 115364
2010-10-02 06:45:50 +08:00
bool FalseBranch, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool CountDuplicatedInstructions(
MachineBasicBlock::iterator &TIB, MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE, MachineBasicBlock::iterator &FIE,
unsigned &Dups1, unsigned &Dups2,
MachineBasicBlock &TBB, MachineBasicBlock &FBB,
bool SkipUnconditionalBranches) const;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
bool ValidDiamond(BBInfo &TrueBBI, BBInfo &FalseBBI,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const;
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool ValidForkedDiamond(BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const;
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
void AnalyzeBranches(BBInfo &BBI);
void ScanInstructions(BBInfo &BBI,
MachineBasicBlock::iterator &Begin,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
MachineBasicBlock::iterator &End,
bool BranchUnpredicable = false) const;
bool RescanInstructions(
MachineBasicBlock::iterator &TIB, MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE, MachineBasicBlock::iterator &FIE,
BBInfo &TrueBBI, BBInfo &FalseBBI) const;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
void AnalyzeBlock(MachineBasicBlock &MBB,
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
std::vector<std::unique_ptr<IfcvtToken>> &Tokens);
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
bool FeasibilityAnalysis(BBInfo &BBI, SmallVectorImpl<MachineOperand> &Cond,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool isTriangle = false, bool RevBranch = false,
bool hasCommonTail = false);
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
void AnalyzeBlocks(MachineFunction &MF,
std::vector<std::unique_ptr<IfcvtToken>> &Tokens);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
void InvalidatePreds(MachineBasicBlock &MBB);
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
void RemoveExtraEdges(BBInfo &BBI);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool IfConvertSimple(BBInfo &BBI, IfcvtKind Kind);
bool IfConvertTriangle(BBInfo &BBI, IfcvtKind Kind);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool IfConvertDiamondCommon(BBInfo &BBI, BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
bool RemoveBranch, bool MergeAddEdges);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
bool IfConvertDiamond(BBInfo &BBI, IfcvtKind Kind,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
unsigned NumDups1, unsigned NumDups2,
bool TClobbers, bool FClobbers);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool IfConvertForkedDiamond(BBInfo &BBI, IfcvtKind Kind,
unsigned NumDups1, unsigned NumDups2,
bool TClobbers, bool FClobbers);
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
void PredicateBlock(BBInfo &BBI,
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
MachineBasicBlock::iterator E,
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
SmallVectorImpl<MachineOperand> &Cond,
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
SmallSet<unsigned, 4> *LaterRedefs = nullptr);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
void CopyAndPredicateBlock(BBInfo &ToBBI, BBInfo &FromBBI,
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVectorImpl<MachineOperand> &Cond,
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
bool IgnoreBr = false);
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
void MergeBlocks(BBInfo &ToBBI, BBInfo &FromBBI, bool AddEdges = true);
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
bool MeetIfcvtSizeLimit(MachineBasicBlock &BB,
unsigned Cycle, unsigned Extra,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const {
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
return Cycle > 0 && TII->isProfitableToIfCvt(BB, Cycle, Extra,
Use BranchProbability instead of floating points in IfConverter. llvm-svn: 134858
2011-07-10 10:58:07 +08:00
Prediction);
Change if-conversion block size limit checks to add some flexibility. llvm-svn: 106901
2010-06-26 06:42:03 +08:00
}
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
bool MeetIfcvtSizeLimit(MachineBasicBlock &TBB,
unsigned TCycle, unsigned TExtra,
MachineBasicBlock &FBB,
unsigned FCycle, unsigned FExtra,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const {
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
return TCycle > 0 && FCycle > 0 &&
TII->isProfitableToIfCvt(TBB, TCycle, TExtra, FBB, FCycle, FExtra,
Use BranchProbability instead of floating points in IfConverter. llvm-svn: 134858
2011-07-10 10:58:07 +08:00
Prediction);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true if Block ends without a terminator.
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
bool blockAlwaysFallThrough(BBInfo &BBI) const {
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
return BBI.IsBrAnalyzable && BBI.TrueBB == nullptr;
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Used to sort if-conversion candidates.
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
static bool IfcvtTokenCmp(const std::unique_ptr<IfcvtToken> &C1,
const std::unique_ptr<IfcvtToken> &C2) {
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
int Incr1 = (C1->Kind == ICDiamond)
? -(int)(C1->NumDups + C1->NumDups2) : (int)C1->NumDups;
int Incr2 = (C2->Kind == ICDiamond)
? -(int)(C2->NumDups + C2->NumDups2) : (int)C2->NumDups;
if (Incr1 > Incr2)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
return true;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
else if (Incr1 == Incr2) {
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Favors subsumption.
Simplify expressions involving boolean constants with clang-tidy Patch by Richard (legalize at xmission dot com). Differential Revision: http://reviews.llvm.org/D8154 llvm-svn: 231617
2015-03-09 09:57:13 +08:00
if (!C1->NeedSubsumption && C2->NeedSubsumption)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
return true;
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
else if (C1->NeedSubsumption == C2->NeedSubsumption) {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
// Favors diamond over triangle, etc.
if ((unsigned)C1->Kind < (unsigned)C2->Kind)
return true;
else if (C1->Kind == C2->Kind)
return C1->BBI.BB->getNumber() < C2->BBI.BB->getNumber();
}
}
return false;
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
};
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
char IfConverter::ID = 0;
Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC) Apparently, the style needs to be agreed upon first. llvm-svn: 240390
2015-06-23 17:49:53 +08:00
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Codegen pass definition cleanup. No functionality. Moving toward a uniform style of pass definition to allow easier target configuration. Globally declare Pass ID. Globally declare pass initializer. Use INITIALIZE_PASS consistently. Add a call to the initializer from CodeGen.cpp. Remove redundant "createPass" functions and "getPassName" methods. While cleaning up declarations, cleaned up comments (sorry for large diff). llvm-svn: 150100
2012-02-09 05:23:13 +08:00
char &llvm::IfConverterID = IfConverter::ID;
Begin adding static dependence information to passes, which will allow us to perform initialization without static constructors AND without explicit initialization by the client. For the moment, passes are required to initialize both their (potential) dependencies and any passes they preserve. I hope to be able to relax the latter requirement in the future. llvm-svn: 116334
2010-10-13 03:48:12 +08:00
INITIALIZE_PASS_BEGIN(IfConverter, "if-converter", "If Converter", false, false)
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
INITIALIZE_PASS_DEPENDENCY(MachineBranchProbabilityInfo)
Begin adding static dependence information to passes, which will allow us to perform initialization without static constructors AND without explicit initialization by the client. For the moment, passes are required to initialize both their (potential) dependencies and any passes they preserve. I hope to be able to relax the latter requirement in the future. llvm-svn: 116334
2010-10-13 03:48:12 +08:00
INITIALIZE_PASS_END(IfConverter, "if-converter", "If Converter", false, false)
Revert r85346 change to control tail merging by CodeGenOpt::Level. I'm going to redo this using the OptimizeForSize function attribute. llvm-svn: 85426
2009-10-29 04:46:46 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
bool IfConverter::runOnMachineFunction(MachineFunction &MF) {
CodeGen/Passes: Pass MachineFunction as functor arg; NFC Passing a MachineFunction as argument is more natural and avoids an unnecessary round-trip through the logic determining the correct Subtarget because MachineFunction already has a reference anyway. llvm-svn: 285039
2016-10-25 07:23:02 +08:00
if (skipFunction(*MF.getFunction()) || (PredicateFtor && !PredicateFtor(MF)))
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
return false;
The subtarget is cached on the MachineFunction. Access it directly. llvm-svn: 227173
2015-01-27 15:31:29 +08:00
const TargetSubtargetInfo &ST = MF.getSubtarget();
TLI = ST.getTargetLowering();
TII = ST.getInstrInfo();
TRI = ST.getRegisterInfo();
Reapply "[MBP] Reduce code size by running tail merging in MBP."" This reapplies commit r271930, r271915, r271923. They hit a bug in Thumb which is fixed in r272258 now. The original message: The code layout that TailMerging (inside BranchFolding) works on is not the final layout optimized based on the branch probability. Generally, after BlockPlacement, many new merging opportunities emerge. This patch calls Tail Merging after MBP and calls MBP again if Tail Merging merges anything. llvm-svn: 272267
2016-06-09 23:24:29 +08:00
BranchFolder::MBFIWrapper MBFI(getAnalysis<MachineBlockFrequencyInfo>());
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
MBPI = &getAnalysis<MachineBranchProbabilityInfo>();
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
MRI = &MF.getRegInfo();
Change MCSchedModel to be a struct of statically initialized data. This removes static initializers from the backends which generate this data, and also makes this struct match the other Tablegen generated structs in behaviour Reviewed by Andy Trick and Chandler C llvm-svn: 216919
2014-09-03 01:43:54 +08:00
SchedModel.init(ST.getSchedModel(), &ST, TII);
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
if (!TII) return false;
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
PreRegAlloc = MRI->isSSA();
bool BFChange = false;
if (!PreRegAlloc) {
// Tail merge tend to expose more if-conversion opportunities.
Reapply "[MBP] Reduce code size by running tail merging in MBP."" This reapplies commit r271930, r271915, r271923. They hit a bug in Thumb which is fixed in r272258 now. The original message: The code layout that TailMerging (inside BranchFolding) works on is not the final layout optimized based on the branch probability. Generally, after BlockPlacement, many new merging opportunities emerge. This patch calls Tail Merging after MBP and calls MBP again if Tail Merging merges anything. llvm-svn: 272267
2016-06-09 23:24:29 +08:00
BranchFolder BF(true, false, MBFI, *MBPI);
The subtarget is cached on the MachineFunction. Access it directly. llvm-svn: 227173
2015-01-27 15:31:29 +08:00
BFChange = BF.OptimizeFunction(MF, TII, ST.getRegisterInfo(),
Have MachineFunction cache a pointer to the subtarget to make lookups shorter/easier and have the DAG use that to do the same lookup. This can be used in the future for TargetMachine based caching lookups from the MachineFunction easily. Update the MIPS subtarget switching machinery to update this pointer at the same time it runs. llvm-svn: 214838
2014-08-05 10:39:49 +08:00
getAnalysisIfAvailable<MachineModuleInfo>());
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
}
Allow ARM if-converter to be run after post allocation scheduling. - This fixed a number of bugs in if-converter, tail merging, and post-allocation scheduler. If-converter now runs branch folding / tail merging first to maximize if-conversion opportunities. - Also changed the t2IT instruction slightly. It now defines the ITSTATE register which is read by instructions in the IT block. - Added Thumb2 specific hazard recognizer to ensure the scheduler doesn't change the instruction ordering in the IT block (since IT mask has been finalized). It also ensures no other instructions can be scheduled between instructions in the IT block. This is not yet enabled. llvm-svn: 106344
2010-06-19 07:09:54 +08:00
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << "\nIfcvt: function (" << ++FnNum << ") \'"
Add a getName function to MachineFunction. Use it in places that previously did getFunction()->getName(). Remove includes of Function.h that are no longer needed. llvm-svn: 162347
2012-08-22 14:07:19 +08:00
<< MF.getName() << "\'");
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
if (FnNum < IfCvtFnStart || (IfCvtFnStop != -1 && FnNum > IfCvtFnStop)) {
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << " skipped\n");
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
return false;
}
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << "\n");
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
MF.RenumberBlocks();
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
BBAnalysis.resize(MF.getNumBlockIDs());
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
std::vector<std::unique_ptr<IfcvtToken>> Tokens;
If true / false blocks fallthrough before ifcvt, add unconditional branches to ifcvt'd block. llvm-svn: 37200
2007-05-18 09:55:58 +08:00
MadeChange = false;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
unsigned NumIfCvts = NumSimple + NumSimpleFalse + NumTriangle +
NumTriangleRev + NumTriangleFalse + NumTriangleFRev + NumDiamonds;
while (IfCvtLimit == -1 || (int)NumIfCvts < IfCvtLimit) {
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Do an initial analysis for each basic block and find all the potential
// candidates to perform if-conversion.
IfConversion's AnalyzeBlocks method always returns false; clean it up. llvm-svn: 106027
2010-06-16 02:57:15 +08:00
bool Change = false;
AnalyzeBlocks(MF, Tokens);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
while (!Tokens.empty()) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
std::unique_ptr<IfcvtToken> Token = std::move(Tokens.back());
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Tokens.pop_back();
BBInfo &BBI = Token->BBI;
IfcvtKind Kind = Token->Kind;
fix leakage of IfcvtTokens llvm-svn: 58690
2008-11-04 21:02:59 +08:00
unsigned NumDups = Token->NumDups;
Fix typo. Patch by nlewycky. llvm-svn: 58709
2008-11-05 02:05:30 +08:00
unsigned NumDups2 = Token->NumDups2;
fix leakage of IfcvtTokens llvm-svn: 58690
2008-11-04 21:02:59 +08:00
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
// If the block has been evicted out of the queue or it has already been
// marked dead (due to it being predicated), then skip it.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (BBI.IsDone)
BBI.IsEnqueued = false;
if (!BBI.IsEnqueued)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
continue;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Fix some stupid bugs that have effectively disabled if-conversion. llvm-svn: 37575
2007-06-15 04:28:52 +08:00
BBI.IsEnqueued = false;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
bool RetVal = false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
switch (Kind) {
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
default: llvm_unreachable("Unexpected!");
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
case ICSimple:
Minor statistics counting bug. llvm-svn: 37451
2007-06-06 09:12:44 +08:00
case ICSimpleFalse: {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool isFalse = Kind == ICSimpleFalse;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if ((isFalse && DisableSimpleF) || (!isFalse && DisableSimple)) break;
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
DEBUG(dbgs() << "Ifcvt (Simple" << (Kind == ICSimpleFalse ?
" false" : "")
Convert DOUT to DEBUG(errs()...). llvm-svn: 79751
2009-08-23 04:11:17 +08:00
<< "): BB#" << BBI.BB->getNumber() << " ("
<< ((Kind == ICSimpleFalse)
? BBI.FalseBB->getNumber()
: BBI.TrueBB->getNumber()) << ") ");
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
RetVal = IfConvertSimple(BBI, Kind);
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
Update gcc 4.3 warnings fix patch with recent head changes llvm-svn: 47368
2008-02-20 19:10:28 +08:00
if (RetVal) {
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (isFalse) ++NumSimpleFalse;
else ++NumSimple;
Update gcc 4.3 warnings fix patch with recent head changes llvm-svn: 47368
2008-02-20 19:10:28 +08:00
}
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
break;
Minor statistics counting bug. llvm-svn: 37451
2007-06-06 09:12:44 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
case ICTriangle:
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
case ICTriangleRev:
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
case ICTriangleFalse:
Fix the build. llvm-svn: 37537
2007-06-10 08:19:17 +08:00
case ICTriangleFRev: {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool isFalse = Kind == ICTriangleFalse;
bool isRev = (Kind == ICTriangleRev || Kind == ICTriangleFRev);
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
if (DisableTriangle && !isFalse && !isRev) break;
if (DisableTriangleR && !isFalse && isRev) break;
if (DisableTriangleF && isFalse && !isRev) break;
if (DisableTriangleFR && isFalse && isRev) break;
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << "Ifcvt (Triangle");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if (isFalse)
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << " false");
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
if (isRev)
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << " rev");
DEBUG(dbgs() << "): BB#" << BBI.BB->getNumber() << " (T:"
Convert DOUT to DEBUG(errs()...). llvm-svn: 79751
2009-08-23 04:11:17 +08:00
<< BBI.TrueBB->getNumber() << ",F:"
<< BBI.FalseBB->getNumber() << ") ");
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
RetVal = IfConvertTriangle(BBI, Kind);
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if (RetVal) {
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
if (isFalse) {
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (isRev) ++NumTriangleFRev;
else ++NumTriangleFalse;
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
} else {
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (isRev) ++NumTriangleRev;
else ++NumTriangle;
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
}
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
break;
Fix the build. llvm-svn: 37537
2007-06-10 08:19:17 +08:00
}
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
case ICDiamond: {
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
if (DisableDiamond) break;
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << "Ifcvt (Diamond): BB#" << BBI.BB->getNumber() << " (T:"
Convert DOUT to DEBUG(errs()...). llvm-svn: 79751
2009-08-23 04:11:17 +08:00
<< BBI.TrueBB->getNumber() << ",F:"
<< BBI.FalseBB->getNumber() << ") ");
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
RetVal = IfConvertDiamond(BBI, Kind, NumDups, NumDups2,
Token->TClobbersPred,
Token->FClobbersPred);
Change errs() to dbgs(). llvm-svn: 92520
2010-01-05 06:02:01 +08:00
DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (RetVal) ++NumDiamonds;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
break;
}
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
case ICForkedDiamond: {
if (DisableForkedDiamond) break;
DEBUG(dbgs() << "Ifcvt (Forked Diamond): BB#"
<< BBI.BB->getNumber() << " (T:"
<< BBI.TrueBB->getNumber() << ",F:"
<< BBI.FalseBB->getNumber() << ") ");
RetVal = IfConvertForkedDiamond(BBI, Kind, NumDups, NumDups2,
Token->TClobbersPred,
Token->FClobbersPred);
DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
if (RetVal) ++NumForkedDiamonds;
break;
}
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
}
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
Change |= RetVal;
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
NumIfCvts = NumSimple + NumSimpleFalse + NumTriangle + NumTriangleRev +
NumTriangleFalse + NumTriangleFRev + NumDiamonds;
if (IfCvtLimit != -1 && (int)NumIfCvts >= IfCvtLimit)
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
break;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
if (!Change)
break;
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
MadeChange |= Change;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
If true / false blocks fallthrough before ifcvt, add unconditional branches to ifcvt'd block. llvm-svn: 37200
2007-05-18 09:55:58 +08:00
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Tokens.clear();
If true / false blocks fallthrough before ifcvt, add unconditional branches to ifcvt'd block. llvm-svn: 37200
2007-05-18 09:55:58 +08:00
BBAnalysis.clear();
Fix an inverted condition. llvm-svn: 106330
2010-06-19 06:17:13 +08:00
if (MadeChange && IfCvtBranchFold) {
Reapply "[MBP] Reduce code size by running tail merging in MBP."" This reapplies commit r271930, r271915, r271923. They hit a bug in Thumb which is fixed in r272258 now. The original message: The code layout that TailMerging (inside BranchFolding) works on is not the final layout optimized based on the branch probability. Generally, after BlockPlacement, many new merging opportunities emerge. This patch calls Tail Merging after MBP and calls MBP again if Tail Merging merges anything. llvm-svn: 272267
2016-06-09 23:24:29 +08:00
BranchFolder BF(false, false, MBFI, *MBPI);
Have MachineFunction cache a pointer to the subtarget to make lookups shorter/easier and have the DAG use that to do the same lookup. This can be used in the future for TargetMachine based caching lookups from the MachineFunction easily. Update the MIPS subtarget switching machinery to update this pointer at the same time it runs. llvm-svn: 214838
2014-08-05 10:39:49 +08:00
BF.OptimizeFunction(MF, TII, MF.getSubtarget().getRegisterInfo(),
Run branch folding if if-converter make some transformations. llvm-svn: 80994
2009-09-04 15:47:40 +08:00
getAnalysisIfAvailable<MachineModuleInfo>());
}
Allow ARM if-converter to be run after post allocation scheduling. - This fixed a number of bugs in if-converter, tail merging, and post-allocation scheduler. If-converter now runs branch folding / tail merging first to maximize if-conversion opportunities. - Also changed the t2IT instruction slightly. It now defines the ITSTATE register which is read by instructions in the IT block. - Added Thumb2 specific hazard recognizer to ensure the scheduler doesn't change the instruction ordering in the IT block (since IT mask has been finalized). It also ensures no other instructions can be scheduled between instructions in the IT block. This is not yet enabled. llvm-svn: 106344
2010-06-19 07:09:54 +08:00
MadeChange |= BFChange;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
return MadeChange;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// BB has a fallthrough. Find its 'false' successor given its 'true' successor.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
static MachineBasicBlock *findFalseBlock(MachineBasicBlock *BB,
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
MachineBasicBlock *TrueBB) {
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock *SuccBB : BB->successors()) {
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
if (SuccBB != TrueBB)
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
return SuccBB;
}
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
return nullptr;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Reverse the condition of the end of the block branch. Swap block's 'true'
/// and 'false' successors.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
bool IfConverter::reverseBranchCondition(BBInfo &BBI) const {
Add a DebugLoc parameter to TargetInstrInfo::InsertBranch(). This addresses a longstanding deficiency noted in many FIXMEs scattered across all the targets. This effectively moves the problem up one level, replacing eleven FIXMEs in the targets with eight FIXMEs in CodeGen, plus one path through FastISel where we actually supply a DebugLoc, fixing Radar 7421831. llvm-svn: 106243
2010-06-18 06:43:56 +08:00
DebugLoc dl; // FIXME: this is nowhere
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (!TII->reverseBranchCondition(BBI.BrCond)) {
TII->removeBranch(*BBI.BB);
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(*BBI.BB, BBI.FalseBB, BBI.TrueBB, BBI.BrCond, dl);
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
std::swap(BBI.TrueBB, BBI.FalseBB);
return true;
}
return false;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns the next block in the function blocks ordering. If it is the end,
/// returns NULL.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
static inline MachineBasicBlock *getNextBlock(MachineBasicBlock &MBB) {
MachineFunction::iterator I = MBB.getIterator();
MachineFunction::iterator E = MBB.getParent()->end();
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if (++I == E)
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
return nullptr;
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
return &*I;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true if the 'true' block (along with its predecessor) forms a valid
/// simple shape for ifcvt. It also returns the number of instructions that the
/// ifcvt would need to duplicate if performed in Dups.
Give the if-converter access to MachineLoopInfo, and use it to generate plausible branch prediction estimates. llvm-svn: 114981
2010-09-29 04:42:15 +08:00
bool IfConverter::ValidSimple(BBInfo &TrueBBI, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = 0;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
Avoid if-converting simple block that ends with unconditional branch or fallthrough unless it branches / falls to the 'false' block. Not profitable, may end up increasing code size. llvm-svn: 37660
2007-06-20 05:45:13 +08:00
if (TrueBBI.IsBrAnalyzable)
return false;
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
if (TrueBBI.BB->pred_size() > 1) {
if (TrueBBI.CannotBeCopied ||
Give the if-converter access to MachineLoopInfo, and use it to generate plausible branch prediction estimates. llvm-svn: 114981
2010-09-29 04:42:15 +08:00
!TII->isProfitableToDupForIfCvt(*TrueBBI.BB, TrueBBI.NonPredSize,
Use BranchProbability instead of floating points in IfConverter. llvm-svn: 134858
2011-07-10 10:58:07 +08:00
Prediction))
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = TrueBBI.NonPredSize;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Avoid if-converting simple block that ends with unconditional branch or fallthrough unless it branches / falls to the 'false' block. Not profitable, may end up increasing code size. llvm-svn: 37660
2007-06-20 05:45:13 +08:00
return true;
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true if the 'true' and 'false' blocks (along with their common
/// predecessor) forms a valid triangle shape for ifcvt. If 'FalseBranch' is
/// true, it checks if 'true' block's false branch branches to the 'false' block
/// rather than the other way around. It also returns the number of instructions
/// that the ifcvt would need to duplicate if performed in 'Dups'.
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
bool IfConverter::ValidTriangle(BBInfo &TrueBBI, BBInfo &FalseBBI,
Thread the determination of branch prediction hit rates back through the if-conversion heuristic APIs. For now, stick with a constant estimate of 90% (branch predictors are good!), but we might find that we want to provide more nuanced estimates in the future. llvm-svn: 115364
2010-10-02 06:45:50 +08:00
bool FalseBranch, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = 0;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
if (TrueBBI.BB->pred_size() > 1) {
if (TrueBBI.CannotBeCopied)
return false;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
unsigned Size = TrueBBI.NonPredSize;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (TrueBBI.IsBrAnalyzable) {
Use empty() instead of comparing size() with zero. llvm-svn: 46514
2008-01-29 21:02:09 +08:00
if (TrueBBI.TrueBB && TrueBBI.BrCond.empty())
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Ends with an unconditional branch. It will be removed.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
--Size;
else {
MachineBasicBlock *FExit = FalseBranch
? TrueBBI.TrueBB : TrueBBI.FalseBB;
if (FExit)
// Require a conditional branch
++Size;
}
}
Use BranchProbability instead of floating points in IfConverter. llvm-svn: 134858
2011-07-10 10:58:07 +08:00
if (!TII->isProfitableToDupForIfCvt(*TrueBBI.BB, Size, Prediction))
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = Size;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
MachineBasicBlock *TExit = FalseBranch ? TrueBBI.FalseBB : TrueBBI.TrueBB;
if (!TExit && blockAlwaysFallThrough(TrueBBI)) {
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
MachineFunction::iterator I = TrueBBI.BB->getIterator();
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
if (++I == TrueBBI.BB->getParent()->end())
return false;
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
TExit = &*I;
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
return TExit && TExit == FalseBBI.BB;
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
IfConversion: Document countDuplicatedInstructions. NFC llvm-svn: 277903
2016-08-06 09:52:33 +08:00
/// Count duplicated instructions and move the iterators to show where they
/// are.
/// @param TIB True Iterator Begin
/// @param FIB False Iterator Begin
/// These two iterators initially point to the first instruction of the two
/// blocks, and finally point to the first non-shared instruction.
/// @param TIE True Iterator End
/// @param FIE False Iterator End
/// These two iterators initially point to End() for the two blocks() and
/// finally point to the first shared instruction in the tail.
/// Upon return [TIB, TIE), and [FIB, FIE) mark the un-duplicated portions of
/// two blocks.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
/// @param Dups1 count of duplicated instructions at the beginning of the 2
/// blocks.
/// @param Dups2 count of duplicated instructions at the end of the 2 blocks.
/// @param SkipUnconditionalBranches if true, Don't make sure that
/// unconditional branches at the end of the blocks are the same. True is
/// passed when the blocks are analyzable to allow for fallthrough to be
/// handled.
/// @return false if the shared portion prevents if conversion.
bool IfConverter::CountDuplicatedInstructions(
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
MachineBasicBlock::iterator &TIB,
MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE,
MachineBasicBlock::iterator &FIE,
unsigned &Dups1, unsigned &Dups2,
MachineBasicBlock &TBB, MachineBasicBlock &FBB,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool SkipUnconditionalBranches) const {
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
When the "true" and "false" blocks of a diamond if-conversion are the same, do not double-count the duplicate instructions by counting once from the beginning and again from the end. Keep track of where the duplicates from the beginning ended and don't go past that point when counting duplicates at the end. Radar 8589805. This change causes one of the MC/ARM/simple-fp-encoding tests to produce different (better!) code without the vmovne instruction being tested. I changed the test to produce vmovne and vmoveq instructions but moving between register files in the opposite direction. That's not quite the same but predicated versions of those instructions weren't being tested before, so at least the test coverage is not any worse, just different. llvm-svn: 117333
2010-10-26 08:02:24 +08:00
while (TIB != TIE && FIB != FIE) {
Teach iff-converter to properly count # of dups. It was not skipping over dbg_value's which resulted in non-duplicated instructions being deleted. rdar://8104384. llvm-svn: 106323
2010-06-19 05:52:57 +08:00
// Skip dbg_value instructions. These do not count.
[codegen] Add generic functions to skip debug values. Summary: This commits moves skipDebugInstructionsForward and skipDebugInstructionsBackward from lib/CodeGen/IfConversion.cpp to include/llvm/CodeGen/MachineBasicBlock.h and updates some codgen files to use them. This refactoring was suggested in https://reviews.llvm.org/D27688 and I thought it's best to do the refactoring in a separate review, but I could also put both changes in a single review if that's preferred. Also, the names for the functions aren't the snappiest and I would be happy to rename them if anybody has suggestions. Reviewers: eli.friedman, iteratee, aprantl, MatzeB Subscribers: MatzeB, llvm-commits Differential Revision: https://reviews.llvm.org/D27782 llvm-svn: 289933
2016-12-16 19:10:26 +08:00
TIB = skipDebugInstructionsForward(TIB, TIE);
FIB = skipDebugInstructionsForward(FIB, FIE);
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (TIB == TIE || FIB == FIE)
IfConversion: factor out 2 functions to skip debug instrs. NFC Skipping debug instructions occurrs repeatedly, factor it out. llvm-svn: 277902
2016-08-06 09:52:31 +08:00
break;
CodeGen: Change MachineInstr to use MachineInstr&, NFC Change MachineInstr API to prefer MachineInstr& over MachineInstr* whenever the parameter is expected to be non-null. Slowly inching toward being able to fix PR26753. llvm-svn: 262149
2016-02-28 04:01:33 +08:00
if (!TIB->isIdenticalTo(*FIB))
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
break;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// A pred-clobbering instruction in the shared portion prevents
// if-conversion.
std::vector<MachineOperand> PredDefs;
if (TII->DefinesPredicate(*TIB, PredDefs))
return false;
IfConversion: Don't count branches in # of duplicates. If the entire blocks match, we would count the branch instructions toward the number of duplicated instructions. This doesn't match what we do elsewhere, and was causing a bug. llvm-svn: 280448
2016-09-02 09:20:06 +08:00
// If we get all the way to the branch instructions, don't count them.
if (!TIB->isBranch())
++Dups1;
When the "true" and "false" blocks of a diamond if-conversion are the same, do not double-count the duplicate instructions by counting once from the beginning and again from the end. Keep track of where the duplicates from the beginning ended and don't go past that point when counting duplicates at the end. Radar 8589805. This change causes one of the MC/ARM/simple-fp-encoding tests to produce different (better!) code without the vmovne instruction being tested. I changed the test to produce vmovne and vmoveq instructions but moving between register files in the opposite direction. That's not quite the same but predicated versions of those instructions weren't being tested before, so at least the test coverage is not any worse, just different. llvm-svn: 117333
2010-10-26 08:02:24 +08:00
++TIB;
++FIB;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// Check for already containing all of the block.
if (TIB == TIE || FIB == FIE)
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
return true;
IfConversion: Handle inclusive ranges more carefully. This may affect calculations for thresholds, but is not a significant change in behavior. The problem was that an inclusive range must have an additonal flag to showr that it is empty, because otherwise begin == end implies that the range has one element, and it may not be possible to move past on either side. llvm-svn: 279166
2016-08-19 06:09:23 +08:00
// Now, in preparation for counting duplicate instructions at the ends of the
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
// blocks, switch to reverse_iterators. Note that getReverse() returns an
// iterator that points to the same instruction, unlike std::reverse_iterator.
// We have to do our own shifting so that we get the same range.
MachineBasicBlock::reverse_iterator RTIE = std::next(TIE.getReverse());
MachineBasicBlock::reverse_iterator RFIE = std::next(FIE.getReverse());
const MachineBasicBlock::reverse_iterator RTIB = std::next(TIB.getReverse());
const MachineBasicBlock::reverse_iterator RFIB = std::next(FIB.getReverse());
IfConversion: Handle inclusive ranges more carefully. This may affect calculations for thresholds, but is not a significant change in behavior. The problem was that an inclusive range must have an additonal flag to showr that it is empty, because otherwise begin == end implies that the range has one element, and it may not be possible to move past on either side. llvm-svn: 279166
2016-08-19 06:09:23 +08:00
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
if (!TBB.succ_empty() || !FBB.succ_empty()) {
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (SkipUnconditionalBranches) {
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
while (RTIE != RTIB && RTIE->isUnconditionalBranch())
++RTIE;
while (RFIE != RFIB && RFIE->isUnconditionalBranch())
++RFIE;
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
}
When the "true" and "false" blocks of a diamond if-conversion are the same, do not double-count the duplicate instructions by counting once from the beginning and again from the end. Keep track of where the duplicates from the beginning ended and don't go past that point when counting duplicates at the end. Radar 8589805. This change causes one of the MC/ARM/simple-fp-encoding tests to produce different (better!) code without the vmovne instruction being tested. I changed the test to produce vmovne and vmoveq instructions but moving between register files in the opposite direction. That's not quite the same but predicated versions of those instructions weren't being tested before, so at least the test coverage is not any worse, just different. llvm-svn: 117333
2010-10-26 08:02:24 +08:00
}
// Count duplicate instructions at the ends of the blocks.
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
while (RTIE != RTIB && RFIE != RFIB) {
Teach iff-converter to properly count # of dups. It was not skipping over dbg_value's which resulted in non-duplicated instructions being deleted. rdar://8104384. llvm-svn: 106323
2010-06-19 05:52:57 +08:00
// Skip dbg_value instructions. These do not count.
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
// Note that these are reverse iterators going forward.
RTIE = skipDebugInstructionsForward(RTIE, RTIB);
RFIE = skipDebugInstructionsForward(RFIE, RFIB);
if (RTIE == RTIB || RFIE == RFIB)
IfConversion: factor out 2 functions to skip debug instrs. NFC Skipping debug instructions occurrs repeatedly, factor it out. llvm-svn: 277902
2016-08-06 09:52:31 +08:00
break;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (!RTIE->isIdenticalTo(*RFIE))
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
break;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// We have to verify that any branch instructions are the same, and then we
// don't count them toward the # of duplicate instructions.
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (!RTIE->isBranch())
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
++Dups2;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
++RTIE;
++RFIE;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
TIE = std::next(RTIE.getReverse());
FIE = std::next(RFIE.getReverse());
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
return true;
}
/// RescanInstructions - Run ScanInstructions on a pair of blocks.
/// @param TIB - True Iterator Begin, points to first non-shared instruction
/// @param FIB - False Iterator Begin, points to first non-shared instruction
/// @param TIE - True Iterator End, points past last non-shared instruction
/// @param FIE - False Iterator End, points past last non-shared instruction
/// @param TrueBBI - BBInfo to update for the true block.
/// @param FalseBBI - BBInfo to update for the false block.
/// @returns - false if either block cannot be predicated or if both blocks end
/// with a predicate-clobbering instruction.
bool IfConverter::RescanInstructions(
MachineBasicBlock::iterator &TIB, MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE, MachineBasicBlock::iterator &FIE,
BBInfo &TrueBBI, BBInfo &FalseBBI) const {
bool BranchUnpredicable = true;
TrueBBI.IsUnpredicable = FalseBBI.IsUnpredicable = false;
ScanInstructions(TrueBBI, TIB, TIE, BranchUnpredicable);
if (TrueBBI.IsUnpredicable)
return false;
ScanInstructions(FalseBBI, FIB, FIE, BranchUnpredicable);
if (FalseBBI.IsUnpredicable)
return false;
if (TrueBBI.ClobbersPred && FalseBBI.ClobbersPred)
return false;
return true;
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
}
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
#ifndef NDEBUG
static void verifySameBranchInstructions(
MachineBasicBlock *MBB1,
MachineBasicBlock *MBB2) {
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
const MachineBasicBlock::reverse_iterator B1 = MBB1->rend();
const MachineBasicBlock::reverse_iterator B2 = MBB2->rend();
MachineBasicBlock::reverse_iterator E1 = MBB1->rbegin();
MachineBasicBlock::reverse_iterator E2 = MBB2->rbegin();
while (E1 != B1 && E2 != B2) {
skipDebugInstructionsForward(E1, B1);
skipDebugInstructionsForward(E2, B2);
if (E1 == B1 && E2 == B2)
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
break;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (E1 == B1) {
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
assert(!E2->isBranch() && "Branch mis-match, one block is empty.");
break;
}
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (E2 == B2) {
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
assert(!E1->isBranch() && "Branch mis-match, one block is empty.");
break;
}
if (E1->isBranch() || E2->isBranch())
assert(E1->isIdenticalTo(*E2) &&
"Branch mis-match, branch instructions don't match.");
else
break;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
++E1;
++E2;
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
}
}
#endif
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/// ValidForkedDiamond - Returns true if the 'true' and 'false' blocks (along
/// with their common predecessor) form a diamond if a common tail block is
/// extracted.
/// While not strictly a diamond, this pattern would form a diamond if
/// tail-merging had merged the shared tails.
/// EBB
/// _/ \_
/// | |
/// TBB FBB
/// / \ / \
/// FalseBB TrueBB FalseBB
/// Currently only handles analyzable branches.
/// Specifically excludes actual diamonds to avoid overlap.
bool IfConverter::ValidForkedDiamond(
BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const {
Dups1 = Dups2 = 0;
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone ||
FalseBBI.IsBeingAnalyzed || FalseBBI.IsDone)
return false;
if (!TrueBBI.IsBrAnalyzable || !FalseBBI.IsBrAnalyzable)
return false;
// Don't IfConvert blocks that can't be folded into their predecessor.
if (TrueBBI.BB->pred_size() > 1 || FalseBBI.BB->pred_size() > 1)
return false;
// This function is specifically looking for conditional tails, as
// unconditional tails are already handled by the standard diamond case.
if (TrueBBI.BrCond.size() == 0 ||
FalseBBI.BrCond.size() == 0)
return false;
MachineBasicBlock *TT = TrueBBI.TrueBB;
MachineBasicBlock *TF = TrueBBI.FalseBB;
MachineBasicBlock *FT = FalseBBI.TrueBB;
MachineBasicBlock *FF = FalseBBI.FalseBB;
if (!TT)
TT = getNextBlock(*TrueBBI.BB);
if (!TF)
TF = getNextBlock(*TrueBBI.BB);
if (!FT)
FT = getNextBlock(*FalseBBI.BB);
if (!FF)
FF = getNextBlock(*FalseBBI.BB);
if (!TT || !TF)
return false;
// Check successors. If they don't match, bail.
if (!((TT == FT && TF == FF) || (TF == FT && TT == FF)))
return false;
bool FalseReversed = false;
if (TF == FT && TT == FF) {
// If the branches are opposing, but we can't reverse, don't do it.
if (!FalseBBI.IsBrReversible)
return false;
FalseReversed = true;
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
reverseBranchCondition(FalseBBI);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
}
auto UnReverseOnExit = make_scope_exit([&]() {
if (FalseReversed)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
reverseBranchCondition(FalseBBI);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
});
// Count duplicate instructions at the beginning of the true and false blocks.
MachineBasicBlock::iterator TIB = TrueBBI.BB->begin();
MachineBasicBlock::iterator FIB = FalseBBI.BB->begin();
MachineBasicBlock::iterator TIE = TrueBBI.BB->end();
MachineBasicBlock::iterator FIE = FalseBBI.BB->end();
if(!CountDuplicatedInstructions(TIB, FIB, TIE, FIE, Dups1, Dups2,
*TrueBBI.BB, *FalseBBI.BB,
/* SkipUnconditionalBranches */ true))
return false;
TrueBBICalc.BB = TrueBBI.BB;
FalseBBICalc.BB = FalseBBI.BB;
if (!RescanInstructions(TIB, FIB, TIE, FIE, TrueBBICalc, FalseBBICalc))
return false;
// The size is used to decide whether to if-convert, and the shared portions
// are subtracted off. Because of the subtraction, we just use the size that
// was calculated by the original ScanInstructions, as it is correct.
TrueBBICalc.NonPredSize = TrueBBI.NonPredSize;
FalseBBICalc.NonPredSize = FalseBBI.NonPredSize;
return true;
}
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
/// ValidDiamond - Returns true if the 'true' and 'false' blocks (along
/// with their common predecessor) forms a valid diamond shape for ifcvt.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool IfConverter::ValidDiamond(
BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const {
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
Dups1 = Dups2 = 0;
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone ||
FalseBBI.IsBeingAnalyzed || FalseBBI.IsDone)
return false;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
MachineBasicBlock *TT = TrueBBI.TrueBB;
MachineBasicBlock *FT = FalseBBI.TrueBB;
if (!TT && blockAlwaysFallThrough(TrueBBI))
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
TT = getNextBlock(*TrueBBI.BB);
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
if (!FT && blockAlwaysFallThrough(FalseBBI))
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
FT = getNextBlock(*FalseBBI.BB);
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
if (TT != FT)
return false;
if (!TT && (TrueBBI.IsBrAnalyzable || FalseBBI.IsBrAnalyzable))
return false;
if (TrueBBI.BB->pred_size() > 1 || FalseBBI.BB->pred_size() > 1)
return false;
// FIXME: Allow true block to have an early exit?
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (TrueBBI.FalseBB || FalseBBI.FalseBB)
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
return false;
// Count duplicate instructions at the beginning and end of the true and
// false blocks.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// Skip unconditional branches only if we are considering an analyzable
// diamond. Otherwise the branches must be the same.
bool SkipUnconditionalBranches =
TrueBBI.IsBrAnalyzable && FalseBBI.IsBrAnalyzable;
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
MachineBasicBlock::iterator TIB = TrueBBI.BB->begin();
MachineBasicBlock::iterator FIB = FalseBBI.BB->begin();
MachineBasicBlock::iterator TIE = TrueBBI.BB->end();
MachineBasicBlock::iterator FIE = FalseBBI.BB->end();
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if(!CountDuplicatedInstructions(TIB, FIB, TIE, FIE, Dups1, Dups2,
*TrueBBI.BB, *FalseBBI.BB,
SkipUnconditionalBranches))
return false;
TrueBBICalc.BB = TrueBBI.BB;
FalseBBICalc.BB = FalseBBI.BB;
if (!RescanInstructions(TIB, FIB, TIE, FIE, TrueBBICalc, FalseBBICalc))
return false;
// The size is used to decide whether to if-convert, and the shared portions
// are subtracted off. Because of the subtraction, we just use the size that
// was calculated by the original ScanInstructions, as it is correct.
TrueBBICalc.NonPredSize = TrueBBI.NonPredSize;
FalseBBICalc.NonPredSize = FalseBBI.NonPredSize;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
return true;
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
/// AnalyzeBranches - Look at the branches at the end of a block to determine if
/// the block is predicable.
void IfConverter::AnalyzeBranches(BBInfo &BBI) {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (BBI.IsDone)
return;
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
BBI.TrueBB = BBI.FalseBB = nullptr;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.BrCond.clear();
BBI.IsBrAnalyzable =
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
!TII->analyzeBranch(*BBI.BB, BBI.TrueBB, BBI.FalseBB, BBI.BrCond);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
SmallVector<MachineOperand, 4> RevCond(BBI.BrCond.begin(), BBI.BrCond.end());
BBI.IsBrReversible = (RevCond.size() == 0) ||
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
!TII->reverseBranchCondition(RevCond);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
BBI.HasFallThrough = BBI.IsBrAnalyzable && BBI.FalseBB == nullptr;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (BBI.BrCond.size()) {
// No false branch. This BB must end with a conditional branch and a
// fallthrough.
if (!BBI.FalseBB)
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
BBI.FalseBB = findFalseBlock(BBI.BB, BBI.TrueBB);
ifcvt should ignore cfg where true and false successors are the same. llvm-svn: 73423
2009-06-16 05:24:34 +08:00
if (!BBI.FalseBB) {
// Malformed bcc? True and false blocks are the same?
BBI.IsUnpredicable = true;
}
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
}
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
}
/// ScanInstructions - Scan all the instructions in the block to determine if
/// the block is predicable. In most cases, that means all the instructions
/// in the block are isPredicable(). Also checks if the block contains any
/// instruction which can clobber a predicate (e.g. condition code register).
/// If so, the block is not predicable unless it's the last instruction.
void IfConverter::ScanInstructions(BBInfo &BBI,
MachineBasicBlock::iterator &Begin,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
MachineBasicBlock::iterator &End,
bool BranchUnpredicable) const {
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
if (BBI.IsDone || BBI.IsUnpredicable)
return;
bool AlreadyPredicated = !BBI.Predicate.empty();
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.NonPredSize = 0;
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
BBI.ExtraCost = 0;
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
BBI.ExtraCost2 = 0;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.ClobbersPred = false;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineInstr &MI : make_range(Begin, End)) {
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (MI.isDebugValue())
Make if-conversion ignore dbg_value instructions in its analysis. rdar://7797940 llvm-svn: 105498
2010-06-05 07:01:26 +08:00
continue;
[ifcnv] Don't duplicate blocks that contain convergent instructions. It's unsafe to duplicate blocks that contain convergent instructions during ifcnv. See the patch for details. Reviewers: hfinkel Differential Revision: http://reviews.llvm.org/D17518 llvm-svn: 266404
2016-04-15 09:38:41 +08:00
// It's unsafe to duplicate convergent instructions in this context, so set
// BBI.CannotBeCopied to true if MI is convergent. To see why, consider the
// following CFG, which is subject to our "simple" transformation.
//
// BB0 // if (c1) goto BB1; else goto BB2;
// / \
// BB1 |
// | BB2 // if (c2) goto TBB; else goto FBB;
// | / |
// | / |
// TBB |
// | |
// | FBB
// |
// exit
//
// Suppose we want to move TBB's contents up into BB1 and BB2 (in BB1 they'd
// be unconditional, and in BB2, they'd be predicated upon c2), and suppose
// TBB contains a convergent instruction. This is safe iff doing so does
// not add a control-flow dependency to the convergent instruction -- i.e.,
// it's safe iff the set of control flows that leads us to the convergent
// instruction does not get smaller after the transformation.
//
// Originally we executed TBB if c1 || c2. After the transformation, there
// are two copies of TBB's instructions. We get to the first if c1, and we
// get to the second if !c1 && c2.
//
// There are clearly fewer ways to satisfy the condition "c1" than
// "c1 || c2". Since we've shrunk the set of control flows which lead to
// our convergent instruction, the transformation is unsafe.
if (MI.isNotDuplicable() || MI.isConvergent())
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
BBI.CannotBeCopied = true;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
bool isPredicated = TII->isPredicated(MI);
bool isCondBr = BBI.IsBrAnalyzable && MI.isConditionalBranch();
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (BranchUnpredicable && MI.isBranch()) {
BBI.IsUnpredicable = true;
return;
}
[ARMv8] Prevent generation of deprecated IT blocks on ARMv8 in Thumb mode. IT blocks can only be one instruction lonf, and can only contain a subset of the 16 instructions. Patch by Artyom Skrobov! llvm-svn: 190309
2013-09-09 22:21:49 +08:00
// A conditional branch is not predicable, but it may be eliminated.
if (isCondBr)
continue;
if (!isPredicated) {
BBI.NonPredSize++;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
unsigned ExtraPredCost = TII->getPredicationCost(MI);
unsigned NumCycles = SchedModel.computeInstrLatency(&MI, false);
[ARMv8] Prevent generation of deprecated IT blocks on ARMv8 in Thumb mode. IT blocks can only be one instruction lonf, and can only contain a subset of the 16 instructions. Patch by Artyom Skrobov! llvm-svn: 190309
2013-09-09 22:21:49 +08:00
if (NumCycles > 1)
BBI.ExtraCost += NumCycles-1;
BBI.ExtraCost2 += ExtraPredCost;
} else if (!AlreadyPredicated) {
// FIXME: This instruction is already predicated before the
// if-conversion pass. It's probably something like a conditional move.
// Mark this block unpredicable for now.
BBI.IsUnpredicable = true;
return;
Teach if-conversion about instructions that were already predicated, e.g. conditional move. llvm-svn: 37964
2007-07-07 07:24:39 +08:00
}
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (BBI.ClobbersPred && !isPredicated) {
// Predicate modification instruction should end the block (except for
// already predicated instructions and end of block branches).
// Predicate may have been modified, the subsequent (currently)
Teach if-conversion about instructions that were already predicated, e.g. conditional move. llvm-svn: 37964
2007-07-07 07:24:39 +08:00
// unpredicated instructions cannot be correctly predicated.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsUnpredicable = true;
return;
}
Somehow this wasn't committed last time. M_CLOBBERS_PRED is gone. llvm-svn: 38495
2007-07-11 01:50:43 +08:00
// FIXME: Make use of PredDefs? e.g. ADDC, SUBC sets predicates but are
// still potentially predicable.
std::vector<MachineOperand> PredDefs;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (TII->DefinesPredicate(MI, PredDefs))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.ClobbersPred = true;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (!TII->isPredicable(MI)) {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsUnpredicable = true;
return;
}
}
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Determine if the block is a suitable candidate to be predicated by the
/// specified predicate.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
/// @param BBI BBInfo for the block to check
/// @param Pred Predicate array for the branch that leads to BBI
/// @param isTriangle true if the Analysis is for a triangle
/// @param RevBranch true if Reverse(Pred) leads to BBI (e.g. BBI is the false
/// case
/// @param hasCommonTail true if BBI shares a tail with a sibling block that
/// contains any instruction that would make the block unpredicable.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool IfConverter::FeasibilityAnalysis(BBInfo &BBI,
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVectorImpl<MachineOperand> &Pred,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool isTriangle, bool RevBranch,
bool hasCommonTail) {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
// If the block is dead or unpredicable, then it cannot be predicated.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// Two blocks may share a common unpredicable tail, but this doesn't prevent
// them from being if-converted. The non-shared portion is assumed to have
// been checked
if (BBI.IsDone || (BBI.IsUnpredicable && !hasCommonTail))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
[CodeGen][IfCvt] Don't re-ifcvt blocks with unanalyzable terminators. If we couldn't analyze its terminator (i.e., it's an indirectbr, or some other weirdness), we can't safely re-if-convert a predicated block, because we can't tell whether the predicated terminator can fallthrough (it does). Currently, we would completely ignore the fallthrough successor. In the added testcase, this means we used to generate: ... @ %entry: cmp r5, #21 ittt ne @ %cc1f: cmpne r7, #42 @ %cc2t: strne.w r5, [r8] movne pc, r10 @ %cc1t: ... Whereas the successor of %cc1f was originally %bb1. With the fix, we get the correct: ... @ %entry: cmp r5, #21 itt eq @ %cc1t: streq.w r5, [r11] moveq pc, r0 @ %cc1f: cmp r7, #42 itt ne @ %cc2t: strne.w r5, [r8] movne pc, r10 @ %bb1: ... rdar://20192768 Differential Revision: http://reviews.llvm.org/D8509 llvm-svn: 232872
2015-03-21 09:23:15 +08:00
// If it is already predicated but we couldn't analyze its terminator, the
// latter might fallthrough, but we can't determine where to.
// Conservatively avoid if-converting again.
if (BBI.Predicate.size() && !BBI.IsBrAnalyzable)
return false;
Fix a bug in IfConverter with nested predicates. Prior to this patch, IfConverter may widen the cases where a sequence of instructions were executed because of the way it uses nested predicates. This result in incorrect execution. For instance, Let A be a basic block that flows conditionally into B and B be a predicated block. B can be predicated with A.BrToBPredicate into A iff B.Predicate is less "permissive" than A.BrToBPredicate, i.e., iff A.BrToBPredicate subsumes B.Predicate. The IfConverter was checking the opposite: B.Predicate subsumes A.BrToBPredicate. <rdar://problem/14379453> llvm-svn: 187071
2013-07-25 04:20:37 +08:00
// If it is already predicated, check if the new predicate subsumes
// its predicate.
if (BBI.Predicate.size() && !TII->SubsumesPredicate(Pred, BBI.Predicate))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (!hasCommonTail && BBI.BrCond.size()) {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (!isTriangle)
return false;
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Test predicate subsumption.
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> RevPred(Pred.begin(), Pred.end());
SmallVector<MachineOperand, 4> Cond(BBI.BrCond.begin(), BBI.BrCond.end());
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (RevBranch) {
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(Cond))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
}
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(RevPred) ||
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
!TII->SubsumesPredicate(Cond, RevPred))
return false;
}
return true;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Analyze the structure of the sub-CFG starting from the specified block.
/// Record its successors and whether it looks like an if-conversion candidate.
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
void IfConverter::AnalyzeBlock(
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &MBB, std::vector<std::unique_ptr<IfcvtToken>> &Tokens) {
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
struct BBState {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBState(MachineBasicBlock &MBB) : MBB(&MBB), SuccsAnalyzed(false) {}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
MachineBasicBlock *MBB;
/// This flag is true if MBB's successors have been analyzed.
bool SuccsAnalyzed;
};
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Push MBB to the stack.
SmallVector<BBState, 16> BBStack(1, MBB);
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
while (!BBStack.empty()) {
BBState &State = BBStack.back();
MachineBasicBlock *BB = State.MBB;
BBInfo &BBI = BBAnalysis[BB->getNumber()];
Silly boog. llvm-svn: 37328
2007-05-25 08:59:01 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (!State.SuccsAnalyzed) {
if (BBI.IsAnalyzed || BBI.IsBeingAnalyzed) {
BBStack.pop_back();
continue;
}
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
BBI.BB = BB;
BBI.IsBeingAnalyzed = true;
isBlockPredicable() always ignore terminal instructions; add comments. llvm-svn: 37126
2007-05-17 05:54:37 +08:00
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
AnalyzeBranches(BBI);
MachineBasicBlock::iterator Begin = BBI.BB->begin();
MachineBasicBlock::iterator End = BBI.BB->end();
ScanInstructions(BBI, Begin, End);
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Unanalyzable or ends with fallthrough or unconditional branch, or if is
// not considered for ifcvt anymore.
if (!BBI.IsBrAnalyzable || BBI.BrCond.empty() || BBI.IsDone) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
ifcvt should ignore cfg where true and false successors are the same. llvm-svn: 73423
2009-06-16 05:24:34 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Do not ifcvt if either path is a back edge to the entry block.
if (BBI.TrueBB == BB || BBI.FalseBB == BB) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Do not ifcvt if true and false fallthrough blocks are the same.
if (!BBI.FalseBB) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Push the False and True blocks to the stack.
State.SuccsAnalyzed = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBStack.push_back(*BBI.FalseBB);
BBStack.push_back(*BBI.TrueBB);
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
continue;
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (TrueBBI.IsDone && FalseBBI.IsDone) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
SmallVector<MachineOperand, 4>
RevCond(BBI.BrCond.begin(), BBI.BrCond.end());
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
bool CanRevCond = !TII->reverseBranchCondition(RevCond);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
unsigned Dups = 0;
unsigned Dups2 = 0;
bool TNeedSub = !TrueBBI.Predicate.empty();
bool FNeedSub = !FalseBBI.Predicate.empty();
bool Enqueued = false;
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
BranchProbability Prediction = MBPI->getEdgeProbability(BB, TrueBBI.BB);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (CanRevCond) {
BBInfo TrueBBICalc, FalseBBICalc;
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
auto feasibleDiamond = [&]() {
bool MeetsSize = MeetIfcvtSizeLimit(
*TrueBBI.BB, (TrueBBICalc.NonPredSize - (Dups + Dups2) +
TrueBBICalc.ExtraCost), TrueBBICalc.ExtraCost2,
*FalseBBI.BB, (FalseBBICalc.NonPredSize - (Dups + Dups2) +
FalseBBICalc.ExtraCost), FalseBBICalc.ExtraCost2,
Prediction);
bool TrueFeasible = FeasibilityAnalysis(TrueBBI, BBI.BrCond,
/* IsTriangle */ false, /* RevCond */ false,
/* hasCommonTail */ true);
bool FalseFeasible = FeasibilityAnalysis(FalseBBI, RevCond,
/* IsTriangle */ false, /* RevCond */ false,
/* hasCommonTail */ true);
return MeetsSize && TrueFeasible && FalseFeasible;
};
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (ValidDiamond(TrueBBI, FalseBBI, Dups, Dups2,
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
TrueBBICalc, FalseBBICalc)) {
if (feasibleDiamond()) {
// Diamond:
// EBB
// / \_
// | |
// TBB FBB
// \ /
// TailBB
// Note TailBB can be empty.
Tokens.push_back(llvm::make_unique<IfcvtToken>(
BBI, ICDiamond, TNeedSub | FNeedSub, Dups, Dups2,
(bool) TrueBBICalc.ClobbersPred, (bool) FalseBBICalc.ClobbersPred));
Enqueued = true;
}
} else if (ValidForkedDiamond(TrueBBI, FalseBBI, Dups, Dups2,
TrueBBICalc, FalseBBICalc)) {
if (feasibleDiamond()) {
// ForkedDiamond:
// if TBB and FBB have a common tail that includes their conditional
// branch instructions, then we can If Convert this pattern.
// EBB
// _/ \_
// | |
// TBB FBB
// / \ / \
// FalseBB TrueBB FalseBB
//
Tokens.push_back(llvm::make_unique<IfcvtToken>(
BBI, ICForkedDiamond, TNeedSub | FNeedSub, Dups, Dups2,
(bool) TrueBBICalc.ClobbersPred, (bool) FalseBBICalc.ClobbersPred));
Enqueued = true;
}
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
}
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (ValidTriangle(TrueBBI, FalseBBI, false, Dups, Prediction) &&
MeetIfcvtSizeLimit(*TrueBBI.BB, TrueBBI.NonPredSize + TrueBBI.ExtraCost,
TrueBBI.ExtraCost2, Prediction) &&
FeasibilityAnalysis(TrueBBI, BBI.BrCond, true)) {
// Triangle:
// EBB
// | \_
// | |
// | TBB
// | /
// FBB
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
llvm::make_unique<IfcvtToken>(BBI, ICTriangle, TNeedSub, Dups));
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Enqueued = true;
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (ValidTriangle(TrueBBI, FalseBBI, true, Dups, Prediction) &&
MeetIfcvtSizeLimit(*TrueBBI.BB, TrueBBI.NonPredSize + TrueBBI.ExtraCost,
TrueBBI.ExtraCost2, Prediction) &&
FeasibilityAnalysis(TrueBBI, BBI.BrCond, true, true)) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
llvm::make_unique<IfcvtToken>(BBI, ICTriangleRev, TNeedSub, Dups));
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Enqueued = true;
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (ValidSimple(TrueBBI, Dups, Prediction) &&
MeetIfcvtSizeLimit(*TrueBBI.BB, TrueBBI.NonPredSize + TrueBBI.ExtraCost,
TrueBBI.ExtraCost2, Prediction) &&
FeasibilityAnalysis(TrueBBI, BBI.BrCond)) {
// Simple (split, no rejoin):
// EBB
// | \_
// | |
// | TBB---> exit
// |
// FBB
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
llvm::make_unique<IfcvtToken>(BBI, ICSimple, TNeedSub, Dups));
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Enqueued = true;
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (CanRevCond) {
// Try the other path...
if (ValidTriangle(FalseBBI, TrueBBI, false, Dups,
Prediction.getCompl()) &&
MeetIfcvtSizeLimit(*FalseBBI.BB,
FalseBBI.NonPredSize + FalseBBI.ExtraCost,
FalseBBI.ExtraCost2, Prediction.getCompl()) &&
FeasibilityAnalysis(FalseBBI, RevCond, true)) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(llvm::make_unique<IfcvtToken>(BBI, ICTriangleFalse,
FNeedSub, Dups));
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
Enqueued = true;
}
if (ValidTriangle(FalseBBI, TrueBBI, true, Dups,
Prediction.getCompl()) &&
MeetIfcvtSizeLimit(*FalseBBI.BB,
FalseBBI.NonPredSize + FalseBBI.ExtraCost,
FalseBBI.ExtraCost2, Prediction.getCompl()) &&
FeasibilityAnalysis(FalseBBI, RevCond, true, true)) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
llvm::make_unique<IfcvtToken>(BBI, ICTriangleFRev, FNeedSub, Dups));
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
Enqueued = true;
}
if (ValidSimple(FalseBBI, Dups, Prediction.getCompl()) &&
MeetIfcvtSizeLimit(*FalseBBI.BB,
FalseBBI.NonPredSize + FalseBBI.ExtraCost,
FalseBBI.ExtraCost2, Prediction.getCompl()) &&
FeasibilityAnalysis(FalseBBI, RevCond)) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
llvm::make_unique<IfcvtToken>(BBI, ICSimpleFalse, FNeedSub, Dups));
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
Enqueued = true;
}
}
BBI.IsEnqueued = Enqueued;
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
}
Some restructuring in preparation for most aggressive if-conversion. llvm-svn: 37231
2007-05-19 02:14:37 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Analyze all blocks and find entries for all if-conversion candidates.
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
void IfConverter::AnalyzeBlocks(
MachineFunction &MF, std::vector<std::unique_ptr<IfcvtToken>> &Tokens) {
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock &MBB : MF)
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
AnalyzeBlock(MBB, Tokens);
Change traversal order to bottom up in preparation for more aggressive if-conversion. llvm-svn: 37365
2007-05-31 03:49:19 +08:00
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
// Sort to favor more complex ifcvt scheme.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
std::stable_sort(Tokens.begin(), Tokens.end(), IfcvtTokenCmp);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true either if ToMBB is the next block after MBB or that all the
/// intervening blocks are empty (given MBB can fall through to its next block).
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
static bool canFallThroughTo(MachineBasicBlock &MBB, MachineBasicBlock &ToMBB) {
MachineFunction::iterator PI = MBB.getIterator();
[C++11] Replace llvm::next and llvm::prior with std::next and std::prev. Remove the old functions. llvm-svn: 202636
2014-03-02 20:27:27 +08:00
MachineFunction::iterator I = std::next(PI);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineFunction::iterator TI = ToMBB.getIterator();
MachineFunction::iterator E = MBB.getParent()->end();
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
while (I != TI) {
// Check isSuccessor to avoid case where the next block is empty, but
// it's not a successor.
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
if (I == E || !I->empty() || !PI->isSuccessor(&*I))
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
return false;
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
PI = I++;
}
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
return true;
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Invalidate predecessor BB info so it would be re-analyzed to determine if it
/// can be if-converted. If predecessor is already enqueued, dequeue it!
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
void IfConverter::InvalidatePreds(MachineBasicBlock &MBB) {
for (const MachineBasicBlock *Predecessor : MBB.predecessors()) {
CodeGen: switch to a range based for loop Use a range based for loop instead of manual iteration. NFC. llvm-svn: 215287
2014-08-10 01:21:29 +08:00
BBInfo &PBBI = BBAnalysis[Predecessor->getNumber()];
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (PBBI.IsDone || PBBI.BB == &MBB)
If BB is predicated, invalidate its predecessor(s) which would if-convert it. It needs to be re-analyzed. llvm-svn: 37580
2007-06-15 07:13:19 +08:00
continue;
No really, clear predcessors states. llvm-svn: 37581
2007-06-15 07:34:09 +08:00
PBBI.IsAnalyzed = false;
PBBI.IsEnqueued = false;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
}
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Inserts an unconditional branch from \p MBB to \p ToMBB.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
static void InsertUncondBranch(MachineBasicBlock &MBB, MachineBasicBlock &ToMBB,
If there is an empty block between a source and its successor block, it still requires a unconditional branch. llvm-svn: 37344
2007-05-30 06:31:16 +08:00
const TargetInstrInfo *TII) {
Add a DebugLoc parameter to TargetInstrInfo::InsertBranch(). This addresses a longstanding deficiency noted in many FIXMEs scattered across all the targets. This effectively moves the problem up one level, replacing eleven FIXMEs in the targets with eight FIXMEs in CodeGen, plus one path through FastISel where we actually supply a DebugLoc, fixing Radar 7421831. llvm-svn: 106243
2010-06-18 06:43:56 +08:00
DebugLoc dl; // FIXME: this is nowhere
Fix SmallVector's size calculation so that a size of 0 is handled correctly, and change a few SmallVector uses to use size 0 to more clearly reflect their intent. llvm-svn: 55181
2008-08-23 00:07:55 +08:00
SmallVector<MachineOperand, 0> NoCond;
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(MBB, &ToMBB, nullptr, NoCond, dl);
If there is an empty block between a source and its successor block, it still requires a unconditional branch. llvm-svn: 37344
2007-05-30 06:31:16 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Remove true / false edges if either / both are no longer successors.
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
void IfConverter::RemoveExtraEdges(BBInfo &BBI) {
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
MachineBasicBlock *TBB = nullptr, *FBB = nullptr;
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> Cond;
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
if (!TII->analyzeBranch(*BBI.BB, TBB, FBB, Cond))
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
BBI.BB->CorrectExtraCFGEdges(TBB, FBB, !Cond.empty());
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
}
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
/// Behaves like LiveRegUnits::StepForward() but also adds implicit uses to all
[IfConversion] Bugfix: Don't use undef flag while adding use operands. IfConversion used to always add the undef flag when adding a use operand on a newly predicated instruction. This would be an operand for the register being conditionally redefined. Due to the undef flag, the liveness of this register prior to the predicated instruction would get lost. This patch changes this so that such use operands are added only when the register is live, without the undef flag. This was reverted but pushed again now, for details follow link below. Reviewed by Quentin Colombet. http://reviews.llvm.org/D209077 llvm-svn: 277571
2016-08-03 13:46:35 +08:00
/// values defined in MI which are also live/used by MI.
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
static void UpdatePredRedefs(MachineInstr &MI, LivePhysRegs &Redefs) {
[IfConversion] Bugfix: Don't use undef flag while adding use operands. IfConversion used to always add the undef flag when adding a use operand on a newly predicated instruction. This would be an operand for the register being conditionally redefined. Due to the undef flag, the liveness of this register prior to the predicated instruction would get lost. This patch changes this so that such use operands are added only when the register is live, without the undef flag. This was reverted but pushed again now, for details follow link below. Reviewed by Quentin Colombet. http://reviews.llvm.org/D209077 llvm-svn: 277571
2016-08-03 13:46:35 +08:00
const TargetRegisterInfo *TRI = MI.getParent()->getParent()
->getSubtarget().getRegisterInfo();
// Before stepping forward past MI, remember which regs were live
// before MI. This is needed to set the Undef flag only when reg is
// dead.
SparseSet<unsigned> LiveBeforeMI;
LiveBeforeMI.setUniverse(TRI->getNumRegs());
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (unsigned Reg : Redefs)
[IfConversion] Bugfix: Don't use undef flag while adding use operands. IfConversion used to always add the undef flag when adding a use operand on a newly predicated instruction. This would be an operand for the register being conditionally redefined. Due to the undef flag, the liveness of this register prior to the predicated instruction would get lost. This patch changes this so that such use operands are added only when the register is live, without the undef flag. This was reverted but pushed again now, for details follow link below. Reviewed by Quentin Colombet. http://reviews.llvm.org/D209077 llvm-svn: 277571
2016-08-03 13:46:35 +08:00
LiveBeforeMI.insert(Reg);
Refactor UpdatePredRedefs and StepForward to avoid duplication. NFC Note, this is a reapplication of r236515 with a fix to not assert on non-register operands, but instead only handle them until the subsequent commit. Original commit message follows. The code was basically the same here already. Just added an out parameter for a vector of seen defs so that UpdatePredRedefs can call StepForward first, then do its own post processing on the seen defs. Will be used in the next commit to also handle regmasks. llvm-svn: 236538
2015-05-06 04:14:22 +08:00
SmallVector<std::pair<unsigned, const MachineOperand*>, 4> Clobbers;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
Redefs.stepForward(MI, Clobbers);
Refactor UpdatePredRedefs and StepForward to avoid duplication. NFC Note, this is a reapplication of r236515 with a fix to not assert on non-register operands, but instead only handle them until the subsequent commit. Original commit message follows. The code was basically the same here already. Just added an out parameter for a vector of seen defs so that UpdatePredRedefs can call StepForward first, then do its own post processing on the seen defs. Will be used in the next commit to also handle regmasks. llvm-svn: 236538
2015-05-06 04:14:22 +08:00
// Now add the implicit uses for each of the clobbered values.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
for (auto Clobber : Clobbers) {
Refactor UpdatePredRedefs and StepForward to avoid duplication. NFC Note, this is a reapplication of r236515 with a fix to not assert on non-register operands, but instead only handle them until the subsequent commit. Original commit message follows. The code was basically the same here already. Just added an out parameter for a vector of seen defs so that UpdatePredRedefs can call StepForward first, then do its own post processing on the seen defs. Will be used in the next commit to also handle regmasks. llvm-svn: 236538
2015-05-06 04:14:22 +08:00
// FIXME: Const cast here is nasty, but better than making StepForward
// take a mutable instruction instead of const.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
unsigned Reg = Clobber.first;
MachineOperand &Op = const_cast<MachineOperand&>(*Clobber.second);
Handle dead defs in the if converter. We had code such as this: r2 = ... t2Bcc label1: ldr ... r2 label2; return r2<dead, def> The if converter was transforming this to r2<def> = ... return [pred] r2<dead,def> ldr <r2, kill> return which fails the machine verifier because the ldr now reads from a dead def. The fix here detects dead defs in stepForward and passes them back to the caller in the clobbers list. The caller then clears the dead flag from the def is the value is live. llvm-svn: 236660
2015-05-07 06:51:04 +08:00
MachineInstr *OpMI = Op.getParent();
Refactor UpdatePredRedefs and StepForward to avoid duplication. NFC Note, this is a reapplication of r236515 with a fix to not assert on non-register operands, but instead only handle them until the subsequent commit. Original commit message follows. The code was basically the same here already. Just added an out parameter for a vector of seen defs so that UpdatePredRedefs can call StepForward first, then do its own post processing on the seen defs. Will be used in the next commit to also handle regmasks. llvm-svn: 236538
2015-05-06 04:14:22 +08:00
MachineInstrBuilder MIB(*OpMI->getParent()->getParent(), OpMI);
Fix IfConverter to handle regmask machine operands. Note, this is a recommit of r236515 after fixing an error in r236514. The buildbot ran fast enough that it picked up r236514 prior to r236515 and threw an error. r236515 itself ran 'make check' without errors. Original commit message follows: A regmask (typically seen on a call) clobbers the set of registers it lists. The IfConverter, in UpdatePredRedefs, was handling register defs, but not regmasks. These are slightly different to a def in that we need to add both an implicit use and def to appease the machine verifier. Otherwise, uses after the if converted call could think they are reading an undefined register. Reviewed by Matthias Braun and Quentin Colombet. llvm-svn: 236550
2015-05-06 06:09:41 +08:00
if (Op.isRegMask()) {
// First handle regmasks. They clobber any entries in the mask which
// means that we need a def for those registers.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (LiveBeforeMI.count(Reg))
MIB.addReg(Reg, RegState::Implicit);
Fix IfConverter to handle regmask machine operands. Note, this is a recommit of r236515 after fixing an error in r236514. The buildbot ran fast enough that it picked up r236514 prior to r236515 and threw an error. r236515 itself ran 'make check' without errors. Original commit message follows: A regmask (typically seen on a call) clobbers the set of registers it lists. The IfConverter, in UpdatePredRedefs, was handling register defs, but not regmasks. These are slightly different to a def in that we need to add both an implicit use and def to appease the machine verifier. Otherwise, uses after the if converted call could think they are reading an undefined register. Reviewed by Matthias Braun and Quentin Colombet. llvm-svn: 236550
2015-05-06 06:09:41 +08:00
// We also need to add an implicit def of this register for the later
// use to read from.
// For the register allocator to have allocated a register clobbered
// by the call which is used later, it must be the case that
// the call doesn't return.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MIB.addReg(Reg, RegState::Implicit | RegState::Define);
Fix IfConverter to handle regmask machine operands. Note, this is a recommit of r236515 after fixing an error in r236514. The buildbot ran fast enough that it picked up r236514 prior to r236515 and threw an error. r236515 itself ran 'make check' without errors. Original commit message follows: A regmask (typically seen on a call) clobbers the set of registers it lists. The IfConverter, in UpdatePredRedefs, was handling register defs, but not regmasks. These are slightly different to a def in that we need to add both an implicit use and def to appease the machine verifier. Otherwise, uses after the if converted call could think they are reading an undefined register. Reviewed by Matthias Braun and Quentin Colombet. llvm-svn: 236550
2015-05-06 06:09:41 +08:00
continue;
}
assert(Op.isReg() && "Register operand required");
Handle dead defs in the if converter. We had code such as this: r2 = ... t2Bcc label1: ldr ... r2 label2; return r2<dead, def> The if converter was transforming this to r2<def> = ... return [pred] r2<dead,def> ldr <r2, kill> return which fails the machine verifier because the ldr now reads from a dead def. The fix here detects dead defs in stepForward and passes them back to the caller in the clobbers list. The caller then clears the dead flag from the def is the value is live. llvm-svn: 236660
2015-05-07 06:51:04 +08:00
if (Op.isDead()) {
// If we found a dead def, but it needs to be live, then remove the dead
// flag.
if (Redefs.contains(Op.getReg()))
Op.setIsDead(false);
}
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (LiveBeforeMI.count(Reg))
MIB.addReg(Reg, RegState::Implicit);
IfConversion: Add implicit uses for redefined regs with live subregisters Normally, if conversion would add implicit uses for redefined registers, e.g. R0<def> = add_if ..., R0<imp-use>. However, if only subregisters of R0 are known to be live but not R0 itself, such implicit uses will not be added, causing prior definitions of such subregisters and R0 itself to become dead. llvm-svn: 282626
2016-09-29 04:07:41 +08:00
else {
bool HasLiveSubReg = false;
for (MCSubRegIterator S(Reg, TRI); S.isValid(); ++S) {
if (!LiveBeforeMI.count(*S))
continue;
HasLiveSubReg = true;
break;
}
if (HasLiveSubReg)
MIB.addReg(Reg, RegState::Implicit);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
}
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Remove kill flags from operands with a registers in the \p DontKill set.
Convert register liveness tracking to work on a sub-register level instead of just register units. Reviewed by Andy llvm-svn: 197315
2013-12-14 14:52:56 +08:00
static void RemoveKills(MachineInstr &MI, const LivePhysRegs &DontKill) {
WIP: CodeGen: Use MachineInstr& in MachineInstrBundle.h, NFC Update APIs in MachineInstrBundle.h to take and return MachineInstr& instead of MachineInstr* when the instruction cannot be null. Besides being a nice cleanup, this is tacking toward a fix for PR26753. llvm-svn: 262141
2016-02-28 01:05:33 +08:00
for (MIBundleOperands O(MI); O.isValid(); ++O) {
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
if (!O->isReg() || !O->isKill())
continue;
Convert register liveness tracking to work on a sub-register level instead of just register units. Reviewed by Andy llvm-svn: 197315
2013-12-14 14:52:56 +08:00
if (DontKill.contains(O->getReg()))
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
O->setIsKill(false);
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
}
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Walks a range of machine instructions and removes kill flags for registers
/// in the \p DontKill set.
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
static void RemoveKills(MachineBasicBlock::iterator I,
MachineBasicBlock::iterator E,
Convert register liveness tracking to work on a sub-register level instead of just register units. Reviewed by Andy llvm-svn: 197315
2013-12-14 14:52:56 +08:00
const LivePhysRegs &DontKill,
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
const MCRegisterInfo &MCRI) {
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineInstr &MI : make_range(I, E))
RemoveKills(MI, DontKill);
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// If convert a simple (split, no rejoin) sub-CFG.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool IfConverter::IfConvertSimple(BBInfo &BBI, IfcvtKind Kind) {
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
BBInfo *CvtBBI = &TrueBBI;
BBInfo *NextBBI = &FalseBBI;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> Cond(BBI.BrCond.begin(), BBI.BrCond.end());
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICSimpleFalse)
Correctly mark early-exit on the false path. llvm-svn: 37387
2007-06-02 04:29:21 +08:00
std::swap(CvtBBI, NextBBI);
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &CvtMBB = *CvtBBI->BB;
MachineBasicBlock &NextMBB = *NextBBI->BB;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (CvtBBI->IsDone ||
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
(CvtBBI->CannotBeCopied && CvtMBB.pred_size() > 1)) {
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
// Something has changed. It's no longer safe to predicate this block.
BBI.IsAnalyzed = false;
CvtBBI->IsAnalyzed = false;
return false;
Correctly mark early-exit on the false path. llvm-svn: 37387
2007-06-02 04:29:21 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.hasAddressTaken())
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
// Conservatively abort if-conversion if BB's address is taken.
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICSimpleFalse)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(Cond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
LivePhysReg: Use reference instead of pointer in init(); NFC llvm-svn: 289002
2016-12-08 08:15:51 +08:00
Redefs.init(*TRI);
DontKill.init(*TRI);
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
// Initialize liveins to the first BB. These are potentiall redefined by
// predicated instructions.
Redefs.addLiveIns(CvtMBB);
Redefs.addLiveIns(NextMBB);
// Compute a set of registers which must not be killed by instructions in
// BB1: This is everything live-in to BB2.
DontKill.addLiveIns(NextMBB);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.pred_size() > 1) {
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Copy instructions in the true block, predicate them, and add them to
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
// the entry block.
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
CopyAndPredicateBlock(BBI, *CvtBBI, Cond);
Manually remove successors in if conversion when CopyAndPredicateBlock is used In the simple and triangle if-conversion cases, when CopyAndPredicateBlock is used because the to-be-predicated block has other predecessors, we need to explicitly remove the old copied block from the successors list. Normally if conversion relies on TII->AnalyzeBranch combined with BB->CorrectExtraCFGEdges to cleanup the successors list, but if the predicated block contained an un-analyzable branch (such as a now-predicated return), then this will fail. These extra successors were causing a problem on PPC because it was causing later passes (such as PPCEarlyReturm) to leave dead return-only basic blocks in the code. llvm-svn: 179227
2013-04-11 06:05:25 +08:00
// RemoveExtraEdges won't work if the block has an unanalyzable branch, so
// explicitly remove CvtBBI as a successor.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBI.BB->removeSuccessor(&CvtMBB, true);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
} else {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
RemoveKills(CvtMBB.begin(), CvtMBB.end(), DontKill, *TRI);
PredicateBlock(*CvtBBI, CvtMBB.end(), Cond);
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
// Merge converted block into entry block.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
MergeBlocks(BBI, *CvtBBI);
}
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
bool IterIfcvt = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (!canFallThroughTo(*BBI.BB, NextMBB)) {
InsertUncondBranch(*BBI.BB, NextMBB, TII);
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
BBI.HasFallThrough = false;
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
// Now ifcvt'd block will look like this:
// BB:
// ...
// t, f = cmp
// if t op
// b BBf
//
// We cannot further ifcvt this block because the unconditional branch
// will have to be predicated on the new condition, that will not be
// available if cmp executes.
IterIfcvt = false;
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
RemoveExtraEdges(BBI);
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
// Update block info. BB can be iteratively if-converted.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (!IterIfcvt)
BBI.IsDone = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InvalidatePreds(*BBI.BB);
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
CvtBBI->IsDone = true;
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
// FIXME: Must maintain LiveIns.
return true;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// If convert a triangle sub-CFG.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool IfConverter::IfConvertTriangle(BBInfo &BBI, IfcvtKind Kind) {
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
BBInfo *CvtBBI = &TrueBBI;
BBInfo *NextBBI = &FalseBBI;
Add a DebugLoc parameter to TargetInstrInfo::InsertBranch(). This addresses a longstanding deficiency noted in many FIXMEs scattered across all the targets. This effectively moves the problem up one level, replacing eleven FIXMEs in the targets with eight FIXMEs in CodeGen, plus one path through FastISel where we actually supply a DebugLoc, fixing Radar 7421831. llvm-svn: 106243
2010-06-18 06:43:56 +08:00
DebugLoc dl; // FIXME: this is nowhere
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> Cond(BBI.BrCond.begin(), BBI.BrCond.end());
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICTriangleFalse || Kind == ICTriangleFRev)
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
std::swap(CvtBBI, NextBBI);
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &CvtMBB = *CvtBBI->BB;
MachineBasicBlock &NextMBB = *NextBBI->BB;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (CvtBBI->IsDone ||
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
(CvtBBI->CannotBeCopied && CvtMBB.pred_size() > 1)) {
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
// Something has changed. It's no longer safe to predicate this block.
BBI.IsAnalyzed = false;
CvtBBI->IsAnalyzed = false;
return false;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
}
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.hasAddressTaken())
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
// Conservatively abort if-conversion if BB's address is taken.
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICTriangleFalse || Kind == ICTriangleFRev)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(Cond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICTriangleRev || Kind == ICTriangleFRev) {
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (reverseBranchCondition(*CvtBBI)) {
Optimized FCMP_OEQ and FCMP_UNE for x86. Where previously LLVM might emit code like this: ucomisd %xmm1, %xmm0 setne %al setp %cl orb %al, %cl jne .LBB4_2 it now emits this: ucomisd %xmm1, %xmm0 jne .LBB4_2 jp .LBB4_2 It has fewer instructions and uses fewer registers, but it does have more branches. And in the case that this code is followed by a non-fallthrough edge, it may be followed by a jmp instruction, resulting in three branch instructions in sequence. Some effort is made to avoid this situation. To achieve this, X86ISelLowering.cpp now recognizes FCMP_OEQ and FCMP_UNE in lowered form, and replace them with code that emits two branches, except in the case where it would require converting a fall-through edge to an explicit branch. Also, X86InstrInfo.cpp's branch analysis and transform code now knows now to handle blocks with multiple conditional branches. It uses loops instead of having fixed checks for up to two instructions. It can now analyze and transform code generated from FCMP_OEQ and FCMP_UNE. llvm-svn: 57873
2008-10-21 11:29:32 +08:00
// BB has been changed, modify its predecessors (except for this
// one) so they don't get ifcvt'ed based on bad intel.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
for (MachineBasicBlock *PBB : CvtMBB.predecessors()) {
Optimized FCMP_OEQ and FCMP_UNE for x86. Where previously LLVM might emit code like this: ucomisd %xmm1, %xmm0 setne %al setp %cl orb %al, %cl jne .LBB4_2 it now emits this: ucomisd %xmm1, %xmm0 jne .LBB4_2 jp .LBB4_2 It has fewer instructions and uses fewer registers, but it does have more branches. And in the case that this code is followed by a non-fallthrough edge, it may be followed by a jmp instruction, resulting in three branch instructions in sequence. Some effort is made to avoid this situation. To achieve this, X86ISelLowering.cpp now recognizes FCMP_OEQ and FCMP_UNE in lowered form, and replace them with code that emits two branches, except in the case where it would require converting a fall-through edge to an explicit branch. Also, X86InstrInfo.cpp's branch analysis and transform code now knows now to handle blocks with multiple conditional branches. It uses loops instead of having fixed checks for up to two instructions. It can now analyze and transform code generated from FCMP_OEQ and FCMP_UNE. llvm-svn: 57873
2008-10-21 11:29:32 +08:00
if (PBB == BBI.BB)
continue;
BBInfo &PBBI = BBAnalysis[PBB->getNumber()];
if (PBBI.IsEnqueued) {
PBBI.IsAnalyzed = false;
PBBI.IsEnqueued = false;
}
No really, clear predcessors states. llvm-svn: 37581
2007-06-15 07:34:09 +08:00
}
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
}
}
Ifcvt triangle: don't ifcvt 'true' BB if it has other predecessors; don't merge 'false' BB if it has other predecessors. llvm-svn: 37382
2007-06-01 15:41:07 +08:00
Tidy. llvm-svn: 106383
2010-06-19 13:33:57 +08:00
// Initialize liveins to the first BB. These are potentially redefined by
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
// predicated instructions.
LivePhysReg: Use reference instead of pointer in init(); NFC llvm-svn: 289002
2016-12-08 08:15:51 +08:00
Redefs.init(*TRI);
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
Redefs.addLiveIns(CvtMBB);
Redefs.addLiveIns(NextMBB);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
DontKill.clear();
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
bool HasEarlyExit = CvtBBI->FalseBB != nullptr;
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
BranchProbability CvtNext, CvtFalse, BBNext, BBCvt;
CodeGen: switch to a range based for loop Use a range based for loop instead of manual iteration. NFC. llvm-svn: 215287
2014-08-10 01:21:29 +08:00
PGO branch weight: update edge weights in IfConverter. This commit only handles IfConvertTriangle. To update edge weights of a successor, one interface is added to MachineBasicBlock: /// Set successor weight of a given iterator. setSuccWeight(succ_iterator I, uint32_t weight) An existing testing case test/CodeGen/Thumb2/v8_IT_5.ll is updated, since we now correctly update the edge weights, the cold block is placed at the end of the function and we jump to the cold block. llvm-svn: 200428
2014-01-30 07:18:47 +08:00
if (HasEarlyExit) {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// Get probabilities before modifying CvtMBB and BBI.BB.
CvtNext = MBPI->getEdgeProbability(&CvtMBB, &NextMBB);
CvtFalse = MBPI->getEdgeProbability(&CvtMBB, CvtBBI->FalseBB);
BBNext = MBPI->getEdgeProbability(BBI.BB, &NextMBB);
BBCvt = MBPI->getEdgeProbability(BBI.BB, &CvtMBB);
PGO branch weight: update edge weights in IfConverter. This commit only handles IfConvertTriangle. To update edge weights of a successor, one interface is added to MachineBasicBlock: /// Set successor weight of a given iterator. setSuccWeight(succ_iterator I, uint32_t weight) An existing testing case test/CodeGen/Thumb2/v8_IT_5.ll is updated, since we now correctly update the edge weights, the cold block is placed at the end of the function and we jump to the cold block. llvm-svn: 200428
2014-01-30 07:18:47 +08:00
}
CodeGen: switch to a range based for loop Use a range based for loop instead of manual iteration. NFC. llvm-svn: 215287
2014-08-10 01:21:29 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.pred_size() > 1) {
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Copy instructions in the true block, predicate them, and add them to
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
// the entry block.
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
CopyAndPredicateBlock(BBI, *CvtBBI, Cond, true);
Manually remove successors in if conversion when CopyAndPredicateBlock is used In the simple and triangle if-conversion cases, when CopyAndPredicateBlock is used because the to-be-predicated block has other predecessors, we need to explicitly remove the old copied block from the successors list. Normally if conversion relies on TII->AnalyzeBranch combined with BB->CorrectExtraCFGEdges to cleanup the successors list, but if the predicated block contained an un-analyzable branch (such as a now-predicated return), then this will fail. These extra successors were causing a problem on PPC because it was causing later passes (such as PPCEarlyReturm) to leave dead return-only basic blocks in the code. llvm-svn: 179227
2013-04-11 06:05:25 +08:00
// RemoveExtraEdges won't work if the block has an unanalyzable branch, so
// explicitly remove CvtBBI as a successor.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBI.BB->removeSuccessor(&CvtMBB, true);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
} else {
// Predicate the 'true' block after removing its branch.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
CvtBBI->NonPredSize -= TII->removeBranch(CvtMBB);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
PredicateBlock(*CvtBBI, CvtMBB.end(), Cond);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
// Now merge the entry of the triangle with the true block.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
MergeBlocks(BBI, *CvtBBI, false);
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
}
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
// If 'true' block has a 'false' successor, add an exit branch to it.
If the predicated block requires an early exit, end the block there and add a unconditional branch to false block. AnalyzeBranch() does not understand early exits. llvm-svn: 37430
2007-06-05 09:31:40 +08:00
if (HasEarlyExit) {
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> RevCond(CvtBBI->BrCond.begin(),
CvtBBI->BrCond.end());
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(RevCond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// Update the edge probability for both CvtBBI->FalseBB and NextBBI.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// NewNext = New_Prob(BBI.BB, NextMBB) =
// Prob(BBI.BB, NextMBB) +
// Prob(BBI.BB, CvtMBB) * Prob(CvtMBB, NextMBB)
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// NewFalse = New_Prob(BBI.BB, CvtBBI->FalseBB) =
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// Prob(BBI.BB, CvtMBB) * Prob(CvtMBB, CvtBBI->FalseBB)
auto NewTrueBB = getNextBlock(*BBI.BB);
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto NewNext = BBNext + BBCvt * CvtNext;
Use the range variant of find/find_if instead of unpacking begin/end If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278469
2016-08-12 11:55:06 +08:00
auto NewTrueBBIter = find(BBI.BB->successors(), NewTrueBB);
Fix a bug in IfConversion.cpp. The bug is introduced in r254377 which failed some tests on ARM, where a new probability is assigned to a successor but the provided BB may not be a successor. llvm-svn: 254463
2015-12-02 05:50:20 +08:00
if (NewTrueBBIter != BBI.BB->succ_end())
BBI.BB->setSuccProbability(NewTrueBBIter, NewNext);
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto NewFalse = BBCvt * CvtFalse;
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(*BBI.BB, CvtBBI->FalseBB, nullptr, RevCond, dl);
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
BBI.BB->addSuccessor(CvtBBI->FalseBB, NewFalse);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
// Merge in the 'false' block if the 'false' block has no other
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// predecessors. Otherwise, add an unconditional branch to 'false'.
Fix some subtle bugs: bug during succeessor copying; incorrectly updating states of ifcvted blocks. llvm-svn: 37429
2007-06-05 08:07:37 +08:00
bool FalseBBDead = false;
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
bool IterIfcvt = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
bool isFallThrough = canFallThroughTo(*BBI.BB, NextMBB);
ifcvt a triangle: don't merge ifcvt block with rejoin block if it can fall through to it. If merged, the resulting block is not a candidate for iterative ifcvting since it contains both predicated and non-predicated code. llvm-svn: 37487
2007-06-07 16:13:00 +08:00
if (!isFallThrough) {
// Only merge them if the true block does not fallthrough to the false
// block. By not merging them, we make it possible to iteratively
// ifcvt the blocks.
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (!HasEarlyExit &&
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
NextMBB.pred_size() == 1 && !NextBBI->HasFallThrough &&
!NextMBB.hasAddressTaken()) {
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
MergeBlocks(BBI, *NextBBI);
ifcvt a triangle: don't merge ifcvt block with rejoin block if it can fall through to it. If merged, the resulting block is not a candidate for iterative ifcvting since it contains both predicated and non-predicated code. llvm-svn: 37487
2007-06-07 16:13:00 +08:00
FalseBBDead = true;
} else {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InsertUncondBranch(*BBI.BB, NextMBB, TII);
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
BBI.HasFallThrough = false;
ifcvt a triangle: don't merge ifcvt block with rejoin block if it can fall through to it. If merged, the resulting block is not a candidate for iterative ifcvting since it contains both predicated and non-predicated code. llvm-svn: 37487
2007-06-07 16:13:00 +08:00
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
// Mixed predicated and unpredicated code. This cannot be iteratively
// predicated.
IterIfcvt = false;
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
RemoveExtraEdges(BBI);
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
// Update block info. BB can be iteratively if-converted.
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
if (!IterIfcvt)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsDone = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InvalidatePreds(*BBI.BB);
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
CvtBBI->IsDone = true;
Fix some subtle bugs: bug during succeessor copying; incorrectly updating states of ifcvted blocks. llvm-svn: 37429
2007-06-05 08:07:37 +08:00
if (FalseBBDead)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
NextBBI->IsDone = true;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
// FIXME: Must maintain LiveIns.
return true;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/// Common code shared between diamond conversions.
/// \p BBI, \p TrueBBI, and \p FalseBBI form the diamond shape.
/// \p NumDups1 - number of shared instructions at the beginning of \p TrueBBI
/// and FalseBBI
/// \p NumDups2 - number of shared instructions at the end of \p TrueBBI
/// and \p FalseBBI
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
/// \p RemoveBranch - Remove the common branch of the two blocks before
/// predicating. Only false for unanalyzable fallthrough
/// cases. The caller will replace the branch if necessary.
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/// \p MergeAddEdges - Add successor edges when merging blocks. Only false for
/// unanalyzable fallthrough
bool IfConverter::IfConvertDiamondCommon(
BBInfo &BBI, BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
bool RemoveBranch, bool MergeAddEdges) {
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (TrueBBI.IsDone || FalseBBI.IsDone ||
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
TrueBBI.BB->pred_size() > 1 || FalseBBI.BB->pred_size() > 1) {
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// Something has changed. It's no longer safe to predicate these blocks.
BBI.IsAnalyzed = false;
TrueBBI.IsAnalyzed = false;
FalseBBI.IsAnalyzed = false;
return false;
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
}
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
if (TrueBBI.BB->hasAddressTaken() || FalseBBI.BB->hasAddressTaken())
// Conservatively abort if-conversion if either BB has its address taken.
return false;
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
// Put the predicated instructions from the 'true' block before the
// instructions from the 'false' block, unless the true block would clobber
// the predicate, in which case, do the opposite.
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
BBInfo *BBI1 = &TrueBBI;
BBInfo *BBI2 = &FalseBBI;
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> RevCond(BBI.BrCond.begin(), BBI.BrCond.end());
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(RevCond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> *Cond1 = &BBI.BrCond;
SmallVector<MachineOperand, 4> *Cond2 = &RevCond;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
// Figure out the more profitable ordering.
bool DoSwap = false;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (TClobbersPred && !FClobbersPred)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
DoSwap = true;
IfConversion: Add assertions that both sides of a diamond don't pred-clobber. One side of a diamond may end with a predicate clobbering instruction. That side of the diamond has to be if-converted second. Both sides can't clobber the predicate or the ifconversion is invalid. This is checked elsewhere, but add an assert as a safety check. NFC llvm-svn: 280518
2016-09-03 02:29:28 +08:00
else if (!TClobbersPred && !FClobbersPred) {
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
if (TrueBBI.NonPredSize > FalseBBI.NonPredSize)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
DoSwap = true;
IfConversion: Add assertions that both sides of a diamond don't pred-clobber. One side of a diamond may end with a predicate clobbering instruction. That side of the diamond has to be if-converted second. Both sides can't clobber the predicate or the ifconversion is invalid. This is checked elsewhere, but add an assert as a safety check. NFC llvm-svn: 280518
2016-09-03 02:29:28 +08:00
} else if (TClobbersPred && FClobbersPred)
llvm_unreachable("Predicate info cannot be clobbered by both sides.");
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (DoSwap) {
Fix a couple of typos and be smarter about order of blocks when ifcvt a diamond. llvm-svn: 37449
2007-06-06 08:57:55 +08:00
std::swap(BBI1, BBI2);
std::swap(Cond1, Cond2);
}
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// Remove the conditional branch from entry to the blocks.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &MBB1 = *BBI1->BB;
MachineBasicBlock &MBB2 = *BBI2->BB;
If-conversion incorrectly calculates liveness of redefined registers Differential Revision: https://reviews.llvm.org/D23207 llvm-svn: 278383
2016-08-12 02:42:06 +08:00
// Initialize the Redefs:
// - BB2 live-in regs need implicit uses before being redefined by BB1
// instructions.
// - BB1 live-out regs need implicit uses before being redefined by BB2
// instructions. We start with BB1 live-ins so we have the live-out regs
// after tracking the BB1 instructions.
LivePhysReg: Use reference instead of pointer in init(); NFC llvm-svn: 289002
2016-12-08 08:15:51 +08:00
Redefs.init(*TRI);
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
Redefs.addLiveIns(MBB1);
Redefs.addLiveIns(MBB2);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// Remove the duplicated instructions at the beginnings of both paths.
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
// Skip dbg_value instructions
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock::iterator DI1 = MBB1.getFirstNonDebugInstr();
MachineBasicBlock::iterator DI2 = MBB2.getFirstNonDebugInstr();
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
BBI1->NonPredSize -= NumDups1;
BBI2->NonPredSize -= NumDups1;
new, no longer brain-dead, r106907 llvm-svn: 107060
2010-06-29 04:26:00 +08:00
// Skip past the dups on each side separately since there may be
// differing dbg_value entries.
for (unsigned i = 0; i < NumDups1; ++DI1) {
if (!DI1->isDebugValue())
++i;
}
make sure to handle dbg_value instructions in the middle of the block, not just at the head, when doing diamond if-conversion. rdar://7797940 llvm-svn: 106907
2010-06-26 07:05:46 +08:00
while (NumDups1 != 0) {
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
++DI2;
new, no longer brain-dead, r106907 llvm-svn: 107060
2010-06-29 04:26:00 +08:00
if (!DI2->isDebugValue())
--NumDups1;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
// Compute a set of registers which must not be killed by instructions in BB1:
// This is everything used+live in BB2 after the duplicated instructions. We
// can compute this set by simulating liveness backwards from the end of BB2.
LivePhysReg: Use reference instead of pointer in init(); NFC llvm-svn: 289002
2016-12-08 08:15:51 +08:00
DontKill.init(*TRI);
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
for (const MachineInstr &MI : make_range(MBB2.rbegin(), ++DI2.getReverse()))
DontKill.stepBackward(MI);
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
for (const MachineInstr &MI : make_range(MBB1.begin(), DI1)) {
SmallVector<std::pair<unsigned, const MachineOperand*>, 4> Dummy;
Redefs.stepForward(MI, Dummy);
}
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
}
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBI.BB->splice(BBI.BB->end(), &MBB1, MBB1.begin(), DI1);
MBB2.erase(MBB2.begin(), DI2);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
// The branches have been checked to match, so it is safe to remove the branch
// in BB1 and rely on the copy in BB2
#ifndef NDEBUG
// Unanalyzable branches must match exactly. Check that now.
if (!BBI1->IsBrAnalyzable)
verifySameBranchInstructions(&MBB1, &MBB2);
#endif
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI1->NonPredSize -= TII->removeBranch(*BBI1->BB);
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// Remove duplicated instructions.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
DI1 = MBB1.end();
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
for (unsigned i = 0; i != NumDups2; ) {
// NumDups2 only counted non-dbg_value instructions, so this won't
// run off the head of the list.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
assert(DI1 != MBB1.begin());
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
--DI1;
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
// skip dbg_value instructions
if (!DI1->isDebugValue())
++i;
}
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MBB1.erase(DI1, MBB1.end());
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
// Kill flags in the true block for registers living into the false block
// must be removed.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
RemoveKills(MBB1.begin(), MBB1.end(), DontKill, *TRI);
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
DI2 = BBI2->BB->end();
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
// The branches have been checked to match. Skip over the branch in the false
// block so that we don't try to predicate it.
if (RemoveBranch)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI2->NonPredSize -= TII->removeBranch(*BBI2->BB);
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
else {
do {
assert(DI2 != MBB2.begin());
DI2--;
} while (DI2->isBranch() || DI2->isDebugValue());
DI2++;
}
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
while (NumDups2 != 0) {
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
// NumDups2 only counted non-dbg_value instructions, so this won't
// run off the head of the list.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
assert(DI2 != MBB2.begin());
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
--DI2;
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
// skip dbg_value instructions
if (!DI2->isDebugValue())
--NumDups2;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
// Remember which registers would later be defined by the false block.
// This allows us not to predicate instructions in the true block that would
// later be re-defined. That is, rather than
// subeq r0, r1, #1
// addne r0, r1, #1
// generate:
// sub r0, r1, #1
// addne r0, r1, #1
SmallSet<unsigned, 4> RedefsByFalse;
SmallSet<unsigned, 4> ExtUses;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (TII->isProfitableToUnpredicate(MBB1, MBB2)) {
for (const MachineInstr &FI : make_range(MBB2.begin(), DI2)) {
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
if (FI.isDebugValue())
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
continue;
SmallVector<unsigned, 4> Defs;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (const MachineOperand &MO : FI.operands()) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!MO.isReg())
continue;
unsigned Reg = MO.getReg();
if (!Reg)
continue;
if (MO.isDef()) {
Defs.push_back(Reg);
} else if (!RedefsByFalse.count(Reg)) {
// These are defined before ctrl flow reach the 'false' instructions.
// They cannot be modified by the 'true' instructions.
Simplify logic now that r182490 is in place. No functional change intended. llvm-svn: 182531
2013-05-23 07:17:36 +08:00
for (MCSubRegIterator SubRegs(Reg, TRI, /*IncludeSelf=*/true);
SubRegs.isValid(); ++SubRegs)
Switch all register list clients to the new MC*Iterator interface. No functional change intended. Sorry for the churn. The iterator classes are supposed to help avoid giant commits like this one in the future. The TableGen-produced register lists are getting quite large, and it may be necessary to change the table representation. This makes it possible to do so without changing all clients (again). llvm-svn: 157854
2012-06-02 07:28:30 +08:00
ExtUses.insert(*SubRegs);
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
}
}
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (unsigned Reg : Defs) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!ExtUses.count(Reg)) {
Simplify logic now that r182490 is in place. No functional change intended. llvm-svn: 182531
2013-05-23 07:17:36 +08:00
for (MCSubRegIterator SubRegs(Reg, TRI, /*IncludeSelf=*/true);
SubRegs.isValid(); ++SubRegs)
Switch all register list clients to the new MC*Iterator interface. No functional change intended. Sorry for the churn. The iterator classes are supposed to help avoid giant commits like this one in the future. The TableGen-produced register lists are getting quite large, and it may be necessary to change the table representation. This makes it possible to do so without changing all clients (again). llvm-svn: 157854
2012-06-02 07:28:30 +08:00
RedefsByFalse.insert(*SubRegs);
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
}
}
}
}
// Predicate the 'true' block.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
PredicateBlock(*BBI1, MBB1.end(), *Cond1, &RedefsByFalse);
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// After predicating BBI1, if there is a predicated terminator in BBI1 and
// a non-predicated in BBI2, then we don't want to predicate the one from
// BBI2. The reason is that if we merged these blocks, we would end up with
// two predicated terminators in the same block.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (!MBB2.empty() && (DI2 == MBB2.end())) {
MachineBasicBlock::iterator BBI1T = MBB1.getFirstTerminator();
MachineBasicBlock::iterator BBI2T = MBB2.getFirstTerminator();
if (BBI1T != MBB1.end() && TII->isPredicated(*BBI1T) &&
BBI2T != MBB2.end() && !TII->isPredicated(*BBI2T))
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
--DI2;
}
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
// Predicate the 'false' block.
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
PredicateBlock(*BBI2, DI2, *Cond2);
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
// Merge the true block into the entry of the diamond.
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
MergeBlocks(BBI, *BBI1, MergeAddEdges);
MergeBlocks(BBI, *BBI2, MergeAddEdges);
return true;
}
/// If convert an almost-diamond sub-CFG where the true
/// and false blocks share a common tail.
bool IfConverter::IfConvertForkedDiamond(
BBInfo &BBI, IfcvtKind Kind,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred) {
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
// Save the debug location for later.
DebugLoc dl;
MachineBasicBlock::iterator TIE = TrueBBI.BB->getFirstTerminator();
if (TIE != TrueBBI.BB->end())
dl = TIE->getDebugLoc();
// Removing branches from both blocks is safe, because we have already
// determined that both blocks have the same branch instructions. The branch
// will be added back at the end, unpredicated.
if (!IfConvertDiamondCommon(
BBI, TrueBBI, FalseBBI,
NumDups1, NumDups2,
TClobbersPred, FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
/* RemoveBranch */ true, /* MergeAddEdges */ true))
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
return false;
// Add back the branch.
// Debug location saved above when removing the branch from BBI2
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(*BBI.BB, TrueBBI.TrueBB, TrueBBI.FalseBB,
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
TrueBBI.BrCond, dl);
RemoveExtraEdges(BBI);
// Update block info.
BBI.IsDone = TrueBBI.IsDone = FalseBBI.IsDone = true;
InvalidatePreds(*BBI.BB);
// FIXME: Must maintain LiveIns.
return true;
}
/// If convert a diamond sub-CFG.
bool IfConverter::IfConvertDiamond(BBInfo &BBI, IfcvtKind Kind,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred) {
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
MachineBasicBlock *TailBB = TrueBBI.TrueBB;
// True block must fall through or end with an unanalyzable terminator.
if (!TailBB) {
if (blockAlwaysFallThrough(TrueBBI))
TailBB = FalseBBI.TrueBB;
assert((TailBB || !TrueBBI.IsBrAnalyzable) && "Unexpected!");
}
if (!IfConvertDiamondCommon(
BBI, TrueBBI, FalseBBI,
NumDups1, NumDups2,
IfConversion: Fix bug introduced by rescanning diamonds. Passing the wrong values for predicate-clobbering. Simple to miss. Added an assert to make this easier to catch in the future. llvm-svn: 280517
2016-09-03 02:29:26 +08:00
TClobbersPred, FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
/* RemoveBranch */ TrueBBI.IsBrAnalyzable,
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/* MergeAddEdges */ TailBB == nullptr))
return false;
Fix a couple of typos and be smarter about order of blocks when ifcvt a diamond. llvm-svn: 37449
2007-06-06 08:57:55 +08:00
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// If the if-converted block falls through or unconditionally branches into
// the tail block, and the tail block does not have other predecessors, then
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// fold the tail block in as well. Otherwise, unless it falls through to the
// tail, add a unconditional branch to it.
if (TailBB) {
Added missing &. Fixes <rdar://problem/10393723> llvm-svn: 143753
2011-11-05 07:49:14 +08:00
BBInfo &TailBBI = BBAnalysis[TailBB->getNumber()];
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
bool CanMergeTail = !TailBBI.HasFallThrough &&
!TailBBI.BB->hasAddressTaken();
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// The if-converted block can still have a predicated terminator
// (e.g. a predicated return). If that is the case, we cannot merge
// it with the tail block.
MachineBasicBlock::const_iterator TI = BBI.BB->getFirstTerminator();
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (TI != BBI.BB->end() && TII->isPredicated(*TI))
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
CanMergeTail = false;
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
// There may still be a fall-through edge from BBI1 or BBI2 to TailBB;
// check if there are any other predecessors besides those.
unsigned NumPreds = TailBB->pred_size();
if (NumPreds > 1)
CanMergeTail = false;
else if (NumPreds == 1 && CanMergeTail) {
MachineBasicBlock::pred_iterator PI = TailBB->pred_begin();
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
if (*PI != TrueBBI.BB && *PI != FalseBBI.BB)
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
CanMergeTail = false;
}
if (CanMergeTail) {
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
MergeBlocks(BBI, TailBBI);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
TailBBI.IsDone = true;
} else {
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
BBI.BB->addSuccessor(TailBB, BranchProbability::getOne());
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InsertUncondBranch(*BBI.BB, *TailBB, TII);
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
BBI.HasFallThrough = false;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
}
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
// RemoveExtraEdges won't work if the block has an unanalyzable branch,
// which can happen here if TailBB is unanalyzable and is merged, so
// explicitly remove BBI1 and BBI2 as successors.
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
BBI.BB->removeSuccessor(TrueBBI.BB);
BBI.BB->removeSuccessor(FalseBBI.BB, /* NormalizeSuccessProbs */ true);
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
RemoveExtraEdges(BBI);
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
// Update block info.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsDone = TrueBBI.IsDone = FalseBBI.IsDone = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InvalidatePreds(*BBI.BB);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
// FIXME: Must maintain LiveIns.
return true;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
static bool MaySpeculate(const MachineInstr &MI,
MachineInstr: Remove unused parameter. llvm-svn: 237726
2015-05-20 05:22:20 +08:00
SmallSet<unsigned, 4> &LaterRedefs) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
bool SawStore = true;
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
if (!MI.isSafeToMove(nullptr, SawStore))
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
return false;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (const MachineOperand &MO : MI.operands()) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!MO.isReg())
continue;
unsigned Reg = MO.getReg();
if (!Reg)
continue;
if (MO.isDef() && !LaterRedefs.count(Reg))
return false;
}
return true;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Predicate instructions from the start of the block to the specified end with
/// the specified condition.
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
void IfConverter::PredicateBlock(BBInfo &BBI,
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
MachineBasicBlock::iterator E,
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
SmallVectorImpl<MachineOperand> &Cond,
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
SmallSet<unsigned, 4> *LaterRedefs) {
bool AnyUnpred = false;
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
bool MaySpec = LaterRedefs != nullptr;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineInstr &I : make_range(BBI.BB->begin(), E)) {
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
if (I.isDebugValue() || TII->isPredicated(I))
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
continue;
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
// It may be possible not to predicate an instruction if it's the 'true'
// side of a diamond and the 'false' side may re-define the instruction's
// defs.
MachineInstr: Remove unused parameter. llvm-svn: 237726
2015-05-20 05:22:20 +08:00
if (MaySpec && MaySpeculate(I, *LaterRedefs)) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
AnyUnpred = true;
continue;
}
// If any instruction is predicated, then every instruction after it must
// be predicated.
MaySpec = false;
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
if (!TII->PredicateInstruction(I, Cond)) {
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#ifndef NDEBUG
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
dbgs() << "Unable to predicate " << I << "!\n";
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#endif
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
llvm_unreachable(nullptr);
isBlockPredicable() always ignore terminal instructions; add comments. llvm-svn: 37126
2007-05-17 05:54:37 +08:00
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
Tidy. llvm-svn: 106383
2010-06-19 13:33:57 +08:00
// If the predicated instruction now redefines a register as the result of
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
// if-conversion, add an implicit kill.
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
UpdatePredRedefs(I, Redefs);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
Replace std::copy with a back inserter with vector append where feasible All of the cases were just appending from random access iterators to a vector. Using insert/append can grow the vector to the perfect size directly and moves the growing out of the loop. No intended functionalty change. llvm-svn: 230845
2015-02-28 18:11:12 +08:00
BBI.Predicate.append(Cond.begin(), Cond.end());
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
Fix some stupid bugs that have effectively disabled if-conversion. llvm-svn: 37575
2007-06-15 04:28:52 +08:00
BBI.IsAnalyzed = false;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
BBI.NonPredSize = 0;
Correct transfer predicate information. llvm-svn: 37524
2007-06-09 03:17:12 +08:00
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
++NumIfConvBBs;
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (AnyUnpred)
++NumUnpred;
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Copy and predicate instructions from source BB to the destination block.
/// Skip end of block branches if IgnoreBr is true.
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
void IfConverter::CopyAndPredicateBlock(BBInfo &ToBBI, BBInfo &FromBBI,
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVectorImpl<MachineOperand> &Cond,
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
bool IgnoreBr) {
Pool-allocation for MachineInstrs, MachineBasicBlocks, and MachineMemOperands. The pools are owned by MachineFunctions. This drastically reduces the number of calls to malloc/free made during the "Emit" phase of scheduling, as well as later phases in CodeGen. Combined with other changes, this speeds up the "instruction selection" phase of CodeGen by 10% in some cases. llvm-svn: 53212
2008-07-08 07:14:23 +08:00
MachineFunction &MF = *ToBBI.BB->getParent();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &FromMBB = *FromBBI.BB;
for (MachineInstr &I : FromMBB) {
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
// Do not copy the end of the block branches.
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (IgnoreBr && I.isBranch())
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
break;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
MachineInstr *MI = MF.CloneMachineInstr(&I);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
ToBBI.BB->insert(ToBBI.BB->end(), MI);
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
ToBBI.NonPredSize++;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
unsigned ExtraPredCost = TII->getPredicationCost(I);
unsigned NumCycles = SchedModel.computeInstrLatency(&I, false);
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
if (NumCycles > 1)
ToBBI.ExtraCost += NumCycles-1;
ToBBI.ExtraCost2 += ExtraPredCost;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
Fix PR7372: Conditional branches (at least on ARM) are treated as predicated, so when IfConverter::CopyAndPredicateBlock checks to see if it should ignore an instruction because it is a branch, it should not check if the branch is predicated. This case (when IgnoreBr is true) is only relevant from IfConvertTriangle, where new branches are inserted after the block has been copied and predicated. If the original branch is not removed, we end up with multiple conditional branches (possibly conflicting) at the end of the block. Aside from any immediate errors resulting from that, this confuses the AnalyzeBranch functions so that the branches are not analyzable. That in turn causes the IfConverter to think that the "Simple" pattern can be applied, and things go downhill fast because the "Simple" pattern does _not_ apply if the block can fall through. This is pretty fragile. If there are other degenerate cases where AnalyzeBranch fails, but where the block may still fall through, the IfConverter should not perform its "Simple" if-conversion. But, I don't know how to do that with the current AnalyzeBranch interface, so for now, the best thing seems to be to avoid creating branches that AnalyzeBranch cannot handle. Evan, please review! llvm-svn: 106291
2010-06-19 01:07:23 +08:00
if (!TII->isPredicated(I) && !MI->isDebugValue()) {
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (!TII->PredicateInstruction(*MI, Cond)) {
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#ifndef NDEBUG
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
dbgs() << "Unable to predicate " << I << "!\n";
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#endif
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
llvm_unreachable(nullptr);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
}
Tidy. llvm-svn: 106383
2010-06-19 13:33:57 +08:00
// If the predicated instruction now redefines a register as the result of
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
// if-conversion, add an implicit kill.
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
UpdatePredRedefs(*MI, Redefs);
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
// Some kill flags may not be correct anymore.
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
if (!DontKill.empty())
Convert register liveness tracking to work on a sub-register level instead of just register units. Reviewed by Andy llvm-svn: 197315
2013-12-14 14:52:56 +08:00
RemoveKills(*MI, DontKill);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
if (!IgnoreBr) {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
std::vector<MachineBasicBlock *> Succs(FromMBB.succ_begin(),
FromMBB.succ_end());
MachineBasicBlock *NBB = getNextBlock(FromMBB);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
MachineBasicBlock *FallThrough = FromBBI.HasFallThrough ? NBB : nullptr;
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock *Succ : Succs) {
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
// Fallthrough edge can't be transferred.
if (Succ == FallThrough)
continue;
ToBBI.BB->addSuccessor(Succ);
}
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
}
Replace std::copy with a back inserter with vector append where feasible All of the cases were just appending from random access iterators to a vector. Using insert/append can grow the vector to the perfect size directly and moves the growing out of the loop. No intended functionalty change. llvm-svn: 230845
2015-02-28 18:11:12 +08:00
ToBBI.Predicate.append(FromBBI.Predicate.begin(), FromBBI.Predicate.end());
ToBBI.Predicate.append(Cond.begin(), Cond.end());
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
ToBBI.ClobbersPred |= FromBBI.ClobbersPred;
ToBBI.IsAnalyzed = false;
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
++NumDupBBs;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Move all instructions from FromBB to the end of ToBB. This will leave
/// FromBB as an empty block, so remove all of its successor edges except for
/// the fall-through edge. If AddEdges is true, i.e., when FromBBI's branch is
/// being moved, add those successor edges to ToBBI.
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
void IfConverter::MergeBlocks(BBInfo &ToBBI, BBInfo &FromBBI, bool AddEdges) {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &FromMBB = *FromBBI.BB;
assert(!FromMBB.hasAddressTaken() &&
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
"Removing a BB whose address is taken!");
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// In case FromMBB contains terminators (e.g. return instruction),
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// first move the non-terminator instructions, then the terminators.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock::iterator FromTI = FromMBB.getFirstTerminator();
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
MachineBasicBlock::iterator ToTI = ToBBI.BB->getFirstTerminator();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
ToBBI.BB->splice(ToTI, &FromMBB, FromMBB.begin(), FromTI);
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// If FromBB has non-predicated terminator we should copy it at the end.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (FromTI != FromMBB.end() && !TII->isPredicated(*FromTI))
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
ToTI = ToBBI.BB->end();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
ToBBI.BB->splice(ToTI, &FromMBB, FromTI, FromMBB.end());
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
Fix PR25838. This is a quick fix to PR25838. The issue comes from the restriction that we cannot normalize probabilities containing both known and unknown ones. A patch that removes this restriction is under the review now: http://reviews.llvm.org/D15548 llvm-svn: 255867
2015-12-17 09:29:08 +08:00
// Force normalizing the successors' probabilities of ToBBI.BB to convert all
// unknown probabilities into known ones.
// FIXME: This usage is too tricky and in the future we would like to
// eliminate all unknown probabilities in MBB.
ToBBI.BB->normalizeSuccProbs();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
SmallVector<MachineBasicBlock *, 4> FromSuccs(FromMBB.succ_begin(),
FromMBB.succ_end());
MachineBasicBlock *NBB = getNextBlock(FromMBB);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
MachineBasicBlock *FallThrough = FromBBI.HasFallThrough ? NBB : nullptr;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// The edge probability from ToBBI.BB to FromMBB, which is only needed when
// AddEdges is true and FromMBB is a successor of ToBBI.BB.
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto To2FromProb = BranchProbability::getZero();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (AddEdges && ToBBI.BB->isSuccessor(&FromMBB)) {
To2FromProb = MBPI->getEdgeProbability(ToBBI.BB, &FromMBB);
// Set the edge probability from ToBBI.BB to FromMBB to zero to avoid the
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// edge probability being merged to other edges when this edge is removed
// later.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
ToBBI.BB->setSuccProbability(find(ToBBI.BB->successors(), &FromMBB),
Use the range variant of find/find_if instead of unpacking begin/end If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278469
2016-08-12 11:55:06 +08:00
BranchProbability::getZero());
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
}
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock *Succ : FromSuccs) {
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
// Fallthrough edge can't be transferred.
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
if (Succ == FallThrough)
continue;
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto NewProb = BranchProbability::getZero();
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
if (AddEdges) {
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// Calculate the edge probability for the edge from ToBBI.BB to Succ,
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// which is a portion of the edge probability from FromMBB to Succ. The
// portion ratio is the edge probability from ToBBI.BB to FromMBB (if
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// FromBBI is a successor of ToBBI.BB. See comment below for excepion).
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
NewProb = MBPI->getEdgeProbability(&FromMBB, Succ);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// To2FromProb is 0 when FromMBB is not a successor of ToBBI.BB. This
// only happens when if-converting a diamond CFG and FromMBB is the
// tail BB. In this case FromMBB post-dominates ToBBI.BB and hence we
// could just use the probabilities on FromMBB's out-edges when adding
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// new successors.
if (!To2FromProb.isZero())
NewProb *= To2FromProb;
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
}
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
FromMBB.removeSuccessor(Succ);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
if (AddEdges) {
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// If the edge from ToBBI.BB to Succ already exists, update the
Normalize MBB's successors' probabilities in several locations. This patch adds some missing calls to MBB::normalizeSuccProbs() in several locations where it should be called. Those places are found by checking if the sum of successors' probabilities is approximate one in MachineBlockPlacement pass with some instrumented code (not in this patch). Differential revision: http://reviews.llvm.org/D15259 llvm-svn: 255455
2015-12-13 17:26:17 +08:00
// probability of this edge by adding NewProb to it. An example is shown
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// below, in which A is ToBBI.BB and B is FromMBB. In this case we
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// don't have to set C as A's successor as it already is. We only need to
// update the edge probability on A->C. Note that B will not be
// immediately removed from A's successors. It is possible that B->D is
// not removed either if D is a fallthrough of B. Later the edge A->D
// (generated here) and B->D will be combined into one edge. To maintain
// correct edge probability of this combined edge, we need to set the edge
// probability of A->B to zero, which is already done above. The edge
// probability on A->D is calculated by scaling the original probability
// on A->B by the probability of B->D.
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
//
// Before ifcvt: After ifcvt (assume B->D is kept):
//
// A A
// /| /|\
// / B / B|
// | /| | ||
// |/ | | |/
// C D C D
//
if (ToBBI.BB->isSuccessor(Succ))
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
ToBBI.BB->setSuccProbability(
Use the range variant of find/find_if instead of unpacking begin/end If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278469
2016-08-12 11:55:06 +08:00
find(ToBBI.BB->successors(), Succ),
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
MBPI->getEdgeProbability(ToBBI.BB, Succ) + NewProb);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
else
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
ToBBI.BB->addSuccessor(Succ, NewProb);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
}
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Now FromBBI always falls through to the next block!
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (NBB && !FromMBB.isSuccessor(NBB))
FromMBB.addSuccessor(NBB);
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
Normalize MBB's successors' probabilities in several locations. This patch adds some missing calls to MBB::normalizeSuccProbs() in several locations where it should be called. Those places are found by checking if the sum of successors' probabilities is approximate one in MachineBlockPlacement pass with some instrumented code (not in this patch). Differential revision: http://reviews.llvm.org/D15259 llvm-svn: 255455
2015-12-13 17:26:17 +08:00
// Normalize the probabilities of ToBBI.BB's successors with all adjustment
// we've done above.
ToBBI.BB->normalizeSuccProbs();
Replace std::copy with a back inserter with vector append where feasible All of the cases were just appending from random access iterators to a vector. Using insert/append can grow the vector to the perfect size directly and moves the growing out of the loop. No intended functionalty change. llvm-svn: 230845
2015-02-28 18:11:12 +08:00
ToBBI.Predicate.append(FromBBI.Predicate.begin(), FromBBI.Predicate.end());
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
FromBBI.Predicate.clear();
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
ToBBI.NonPredSize += FromBBI.NonPredSize;
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
ToBBI.ExtraCost += FromBBI.ExtraCost;
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
ToBBI.ExtraCost2 += FromBBI.ExtraCost2;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
FromBBI.NonPredSize = 0;
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
FromBBI.ExtraCost = 0;
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
FromBBI.ExtraCost2 = 0;
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
ToBBI.ClobbersPred |= FromBBI.ClobbersPred;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
ToBBI.HasFallThrough = FromBBI.HasFallThrough;
Fix some stupid bugs that have effectively disabled if-conversion. llvm-svn: 37575
2007-06-15 04:28:52 +08:00
ToBBI.IsAnalyzed = false;
FromBBI.IsAnalyzed = false;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
FunctionPass *
CodeGen/Passes: Pass MachineFunction as functor arg; NFC Passing a MachineFunction as argument is more natural and avoids an unnecessary round-trip through the logic determining the correct Subtarget because MachineFunction already has a reference anyway. llvm-svn: 285039
2016-10-25 07:23:02 +08:00
llvm::createIfConverter(std::function<bool(const MachineFunction &)> Ftor) {
Move instances of std::function. Or replace with llvm::function_ref if it's never stored. NFC intended. llvm-svn: 272513
2016-06-13 00:13:55 +08:00
return new IfConverter(std::move(Ftor));
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
}