maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/CodeGen/IfConversion.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2363 lines
89 KiB
C++
Raw Normal View History

[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
//===- IfConversion.cpp - Machine code if conversion pass -----------------===//
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 16:50:56 +08:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
//
//===----------------------------------------------------------------------===//
//
[ifcnv] Add brief comment explaining what ifcnv is. llvm-svn: 265088
2016-04-01 09:09:03 +08:00
// This file implements the machine instruction level if-conversion pass, which
// tries to convert conditional branches into predicated instructions.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
//
//===----------------------------------------------------------------------===//
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "BranchFolding.h"
#include "llvm/ADT/STLExtras.h"
IfConversion: Handle inclusive ranges more carefully. This may affect calculations for thresholds, but is not a significant change in behavior. The problem was that an inclusive range must have an additonal flag to showr that it is empty, because otherwise begin == end implies that the range has one element, and it may not be possible to move past on either side. llvm-svn: 279166
2016-08-19 06:09:23 +08:00
#include "llvm/ADT/ScopeExit.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/ADT/SmallSet.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/SparseSet.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/ADT/Statistic.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/ADT/iterator_range.h"
Re-sort all of the includes with ./utils/sort_includes.py so that subsequent changes are easier to review. About to fix some layering issues, and wanted to separate out the necessary churn. Also comment and sink the include of "Windows.h" in three .inc files to match the usage in Memory.inc. llvm-svn: 198685
2014-01-07 19:48:04 +08:00
#include "llvm/CodeGen/LivePhysRegs.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/CodeGen/MachineBasicBlock.h"
[Branch probability] Recompute branch weights of tail-merged basic blocks. BranchFolderPass was not correctly setting the basic block branch weights when tail-merging created or merged blocks. This patch recomutes the weights of tail-merged blocks using the following formula: branch_weight(merged block to successor j) = sum(block_frequency(bb) * branch_probability(bb -> j)) bb is a block that is in the set of merged blocks. <rdar://problem/16256423> llvm-svn: 215135
2014-08-08 03:30:13 +08:00
#include "llvm/CodeGen/MachineBlockFrequencyInfo.h"
Fix typo in #include which revealed in the case-sensitive filesystem. llvm-svn: 136828
2011-08-04 06:53:41 +08:00
#include "llvm/CodeGen/MachineBranchProbabilityInfo.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/CodeGen/MachineFunction.h"
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
#include "llvm/CodeGen/MachineFunctionPass.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/CodeGen/MachineInstr.h"
Use MachineInstrBuilder in a few CodeGen passes. This automatically passes a context pointer to MI->addOperand(). llvm-svn: 170711
2012-12-21 02:08:06 +08:00
#include "llvm/CodeGen/MachineInstrBuilder.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/CodeGen/MachineModuleInfo.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/CodeGen/MachineOperand.h"
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
#include "llvm/CodeGen/MachineRegisterInfo.h"
Target/TargetInstrInfo.h -> CodeGen/TargetInstrInfo.h to match layering This header includes CodeGen headers, and is not, itself, included by any Target headers, so move it into CodeGen to match the layering of its implementation. llvm-svn: 317647
2017-11-08 09:01:31 +08:00
#include "llvm/CodeGen/TargetInstrInfo.h"
Fix a bunch more layering of CodeGen headers that are in Target All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
2017-11-17 09:07:10 +08:00
#include "llvm/CodeGen/TargetLowering.h"
#include "llvm/CodeGen/TargetRegisterInfo.h"
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
#include "llvm/CodeGen/TargetSchedule.h"
Fix a bunch more layering of CodeGen headers that are in Target All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
2017-11-17 09:07:10 +08:00
#include "llvm/CodeGen/TargetSubtargetInfo.h"
Sink all InitializePasses.h includes This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
2019-11-14 05:15:01 +08:00
#include "llvm/IR/Attributes.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/IR/DebugLoc.h"
Sink all InitializePasses.h includes This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
2019-11-14 05:15:01 +08:00
#include "llvm/InitializePasses.h"
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include "llvm/MC/MCRegisterInfo.h"
#include "llvm/Pass.h"
#include "llvm/Support/BranchProbability.h"
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
#include "llvm/Support/CommandLine.h"
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
#include "llvm/Support/Debug.h"
Convert more assert(0)+abort() -> LLVM_UNREACHABLE, and abort()/exit() -> llvm_report_error(). llvm-svn: 75363
2009-07-11 21:10:19 +08:00
#include "llvm/Support/ErrorHandling.h"
#include "llvm/Support/raw_ostream.h"
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
#include <algorithm>
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include <cassert>
#include <functional>
#include <iterator>
#include <memory>
Apply clang-tidy's misc-move-constructor-init throughout LLVM. No functionality change intended, maybe a tiny performance improvement. llvm-svn: 270997
2016-05-27 22:27:24 +08:00
#include <utility>
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
#include <vector>
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
using namespace llvm;
CodeGen: Rename DEBUG_TYPE to match passnames Rename the DEBUG_TYPE to match the names of corresponding passes where it makes sense. Also establish the pattern of simply referencing DEBUG_TYPE instead of repeating the passname where possible. llvm-svn: 303921
2017-05-26 05:26:32 +08:00
#define DEBUG_TYPE "if-converter"
[Modules] Remove potential ODR violations by sinking the DEBUG_TYPE define below all header includes in the lib/CodeGen/... tree. While the current modules implementation doesn't check for this kind of ODR violation yet, it is likely to grow support for it in the future. It also removes one layer of macro pollution across all the included headers. Other sub-trees will follow. llvm-svn: 206837
2014-04-22 10:02:50 +08:00
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
// Hidden options for help debugging.
static cl::opt<int> IfCvtFnStart("ifcvt-fn-start", cl::init(-1), cl::Hidden);
static cl::opt<int> IfCvtFnStop("ifcvt-fn-stop", cl::init(-1), cl::Hidden);
static cl::opt<int> IfCvtLimit("ifcvt-limit", cl::init(-1), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableSimple("disable-ifcvt-simple",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableSimpleF("disable-ifcvt-simple-false",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangle("disable-ifcvt-triangle",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangleR("disable-ifcvt-triangle-rev",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangleF("disable-ifcvt-triangle-false",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableTriangleFR("disable-ifcvt-triangle-false-rev",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
static cl::opt<bool> DisableDiamond("disable-ifcvt-diamond",
simplify some code using new predicates llvm-svn: 45689
2008-01-07 13:40:58 +08:00
cl::init(false), cl::Hidden);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
static cl::opt<bool> DisableForkedDiamond("disable-ifcvt-forked-diamond",
cl::init(false), cl::Hidden);
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
static cl::opt<bool> IfCvtBranchFold("ifcvt-branch-fold",
cl::init(true), cl::Hidden);
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
STATISTIC(NumSimple, "Number of simple if-conversions performed");
STATISTIC(NumSimpleFalse, "Number of simple (F) if-conversions performed");
STATISTIC(NumTriangle, "Number of triangle if-conversions performed");
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
STATISTIC(NumTriangleRev, "Number of triangle (R) if-conversions performed");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
STATISTIC(NumTriangleFalse,"Number of triangle (F) if-conversions performed");
STATISTIC(NumTriangleFRev, "Number of triangle (F/R) if-conversions performed");
STATISTIC(NumDiamonds, "Number of diamond if-conversions performed");
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
STATISTIC(NumForkedDiamonds, "Number of forked-diamond if-conversions performed");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
STATISTIC(NumIfConvBBs, "Number of if-converted blocks");
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
STATISTIC(NumDupBBs, "Number of duplicated blocks");
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
STATISTIC(NumUnpred, "Number of true blocks of diamonds unpredicated");
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
namespace {
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
Remove VISIBILITY_HIDDEN from class/struct found inside anonymous namespaces. Chris claims we should never have visibility_hidden inside any .cpp file but that's still not true even after this commit. llvm-svn: 85042
2009-10-25 14:33:48 +08:00
class IfConverter : public MachineFunctionPass {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
enum IfcvtKind {
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
ICNotClassfied, // BB data valid, but not classified.
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
ICSimpleFalse, // Same as ICSimple, but on the false path.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
ICSimple, // BB is entry of an one split, no rejoin sub-CFG.
ICTriangleFRev, // Same as ICTriangleFalse, but false path rev condition.
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
ICTriangleRev, // Same as ICTriangle, but true path rev condition.
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
ICTriangleFalse, // Same as ICTriangle, but on the false path.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
ICTriangle, // BB is entry of a triangle sub-CFG.
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
ICDiamond, // BB is entry of a diamond sub-CFG.
ICForkedDiamond // BB is entry of an almost diamond sub-CFG, with a
// common tail that can be shared.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
};
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// One per MachineBasicBlock, this is used to cache the result
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
/// if-conversion feasibility analysis. This includes results from
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
/// TargetInstrInfo::analyzeBranch() (i.e. TBB, FBB, and Cond), and its
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
/// classification, and common tail block of its successors (if it's a
Some restructuring in preparation for most aggressive if-conversion. llvm-svn: 37231
2007-05-19 02:14:37 +08:00
/// diamond shape), its size, whether it's predicable, and whether any
/// instruction can clobber the 'would-be' predicate.
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
///
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
/// IsDone - True if BB is not to be considered for ifcvt.
/// IsBeingAnalyzed - True if BB is currently being analyzed.
/// IsAnalyzed - True if BB has been analyzed (info is still valid).
/// IsEnqueued - True if BB has been enqueued to be ifcvt'ed.
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
/// IsBrAnalyzable - True if analyzeBranch() returns false.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
/// HasFallThrough - True if BB may fallthrough to the following BB.
/// IsUnpredicable - True if BB is known to be unpredicable.
Somehow this wasn't committed last time. M_CLOBBERS_PRED is gone. llvm-svn: 38495
2007-07-11 01:50:43 +08:00
/// ClobbersPred - True if BB could modify predicates (e.g. has
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
/// cmp, call, etc.)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
/// NonPredSize - Number of non-predicated instructions.
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
/// ExtraCost - Extra cost for multi-cycle instructions.
/// ExtraCost2 - Some instructions are slower when predicated
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
/// BB - Corresponding MachineBasicBlock.
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
/// TrueBB / FalseBB- See analyzeBranch().
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
/// BrCond - Conditions for end of block conditional branches.
/// Predicate - Predicate used in the BB.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
struct BBInfo {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool IsDone : 1;
bool IsBeingAnalyzed : 1;
bool IsAnalyzed : 1;
bool IsEnqueued : 1;
bool IsBrAnalyzable : 1;
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool IsBrReversible : 1;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool HasFallThrough : 1;
bool IsUnpredicable : 1;
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
bool CannotBeCopied : 1;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool ClobbersPred : 1;
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
unsigned NonPredSize = 0;
unsigned ExtraCost = 0;
unsigned ExtraCost2 = 0;
MachineBasicBlock *BB = nullptr;
MachineBasicBlock *TrueBB = nullptr;
MachineBasicBlock *FalseBB = nullptr;
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> BrCond;
SmallVector<MachineOperand, 4> Predicate;
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
BBInfo() : IsDone(false), IsBeingAnalyzed(false),
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
IsAnalyzed(false), IsEnqueued(false), IsBrAnalyzable(false),
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
IsBrReversible(false), HasFallThrough(false),
IsUnpredicable(false), CannotBeCopied(false),
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
ClobbersPred(false) {}
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
};
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Record information about pending if-conversions to attempt:
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
/// BBI - Corresponding BBInfo.
/// Kind - Type of block. See IfcvtKind.
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
/// NeedSubsumption - True if the to-be-predicated BB has already been
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
/// predicated.
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
/// NumDups - Number of instructions that would be duplicated due
/// to this if-conversion. (For diamonds, the number of
/// identical instructions at the beginnings of both
/// paths).
/// NumDups2 - For diamonds, the number of identical instructions
/// at the ends of both paths.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
struct IfcvtToken {
BBInfo &BBI;
IfcvtKind Kind;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
unsigned NumDups;
unsigned NumDups2;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool NeedSubsumption : 1;
bool TClobbersPred : 1;
bool FClobbersPred : 1;
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
IfcvtToken(BBInfo &b, IfcvtKind k, bool s, unsigned d, unsigned d2 = 0,
bool tc = false, bool fc = false)
: BBI(b), Kind(k), NumDups(d), NumDups2(d2), NeedSubsumption(s),
TClobbersPred(tc), FClobbersPred(fc) {}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
};
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Results of if-conversion feasibility analysis indexed by basic block
/// number.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
std::vector<BBInfo> BBAnalysis;
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
TargetSchedModel SchedModel;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Split TargetLowering into a CodeGen and a SelectionDAG part. This fixes some of the cycles between libCodeGen and libSelectionDAG. It's still a complete mess but as long as the edges consist of virtual call it doesn't cause breakage. BasicTTI did static calls and thus broke some build configurations. llvm-svn: 172246
2013-01-12 04:05:37 +08:00
const TargetLoweringBase *TLI;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
const TargetInstrInfo *TII;
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
const TargetRegisterInfo *TRI;
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
const MachineBranchProbabilityInfo *MBPI;
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
MachineRegisterInfo *MRI;
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
Convert register liveness tracking to work on a sub-register level instead of just register units. Reviewed by Andy llvm-svn: 197315
2013-12-14 14:52:56 +08:00
LivePhysRegs Redefs;
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
bool PreRegAlloc;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
bool MadeChange;
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
int FnNum = -1;
CodeGen/Passes: Pass MachineFunction as functor arg; NFC Passing a MachineFunction as argument is more natural and avoids an unnecessary round-trip through the logic determining the correct Subtarget because MachineFunction already has a reference anyway. llvm-svn: 285039
2016-10-25 07:23:02 +08:00
std::function<bool(const MachineFunction &)> PredicateFtor;
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
public:
static char ID;
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
CodeGen/Passes: Pass MachineFunction as functor arg; NFC Passing a MachineFunction as argument is more natural and avoids an unnecessary round-trip through the logic determining the correct Subtarget because MachineFunction already has a reference anyway. llvm-svn: 285039
2016-10-25 07:23:02 +08:00
IfConverter(std::function<bool(const MachineFunction &)> Ftor = nullptr)
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
: MachineFunctionPass(ID), PredicateFtor(std::move(Ftor)) {
Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which must be called in the pass's constructor. This function uses static dependency declarations to recursively initialize the pass's dependencies. Clients that only create passes through the createFooPass() APIs will require no changes. Clients that want to use the CommandLine options for passes will need to manually call the appropriate initialization functions in PassInitialization.h before parsing commandline arguments. I have tested this with all standard configurations of clang and llvm-gcc on Darwin. It is possible that there are problems with the static dependencies that will only be visible with non-standard options. If you encounter any crash in pass registration/creation, please send the testcase to me directly. llvm-svn: 116820
2010-10-20 01:21:58 +08:00
initializeIfConverterPass(*PassRegistry::getPassRegistry());
}
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
[C++11] Add 'override' keyword to virtual methods that override their base class. llvm-svn: 203220
2014-03-07 17:26:03 +08:00
void getAnalysisUsage(AnalysisUsage &AU) const override {
[Branch probability] Recompute branch weights of tail-merged basic blocks. BranchFolderPass was not correctly setting the basic block branch weights when tail-merging created or merged blocks. This patch recomutes the weights of tail-merged blocks using the following formula: branch_weight(merged block to successor j) = sum(block_frequency(bb) * branch_probability(bb -> j)) bb is a block that is in the set of merged blocks. <rdar://problem/16256423> llvm-svn: 215135
2014-08-08 03:30:13 +08:00
AU.addRequired<MachineBlockFrequencyInfo>();
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
AU.addRequired<MachineBranchProbabilityInfo>();
Give the if-converter access to MachineLoopInfo, and use it to generate plausible branch prediction estimates. llvm-svn: 114981
2010-09-29 04:42:15 +08:00
MachineFunctionPass::getAnalysisUsage(AU);
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
[C++11] Add 'override' keyword to virtual methods that override their base class. llvm-svn: 203220
2014-03-07 17:26:03 +08:00
bool runOnMachineFunction(MachineFunction &MF) override;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Add MachineFunctionProperty checks for AllVRegsAllocated for target passes Summary: This adds the same checks that were added in r264593 to all target-specific passes that run after register allocation. Reviewers: qcolombet Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D18525 llvm-svn: 265313
2016-04-05 01:09:25 +08:00
MachineFunctionProperties getRequiredProperties() const override {
return MachineFunctionProperties().set(
MachineFunctionProperties/MIRParser: Rename AllVRegsAllocated->NoVRegs, compute it Rename AllVRegsAllocated to NoVRegs. This avoids the connotation of running after register and simply describes that no vregs are used in a machine function. With that we can simply compute the property and do not need to dump/parse it in .mir files. Differential Revision: http://reviews.llvm.org/D23850 llvm-svn: 279698
2016-08-25 09:27:13 +08:00
MachineFunctionProperties::Property::NoVRegs);
Add MachineFunctionProperty checks for AllVRegsAllocated for target passes Summary: This adds the same checks that were added in r264593 to all target-specific passes that run after register allocation. Reviewers: qcolombet Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D18525 llvm-svn: 265313
2016-04-05 01:09:25 +08:00
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
private:
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
bool reverseBranchCondition(BBInfo &BBI) const;
Thread the determination of branch prediction hit rates back through the if-conversion heuristic APIs. For now, stick with a constant estimate of 90% (branch predictors are good!), but we might find that we want to provide more nuanced estimates in the future. llvm-svn: 115364
2010-10-02 06:45:50 +08:00
bool ValidSimple(BBInfo &TrueBBI, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const;
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
bool ValidTriangle(BBInfo &TrueBBI, BBInfo &FalseBBI,
Thread the determination of branch prediction hit rates back through the if-conversion heuristic APIs. For now, stick with a constant estimate of 90% (branch predictors are good!), but we might find that we want to provide more nuanced estimates in the future. llvm-svn: 115364
2010-10-02 06:45:50 +08:00
bool FalseBranch, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool CountDuplicatedInstructions(
MachineBasicBlock::iterator &TIB, MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE, MachineBasicBlock::iterator &FIE,
unsigned &Dups1, unsigned &Dups2,
MachineBasicBlock &TBB, MachineBasicBlock &FBB,
bool SkipUnconditionalBranches) const;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
bool ValidDiamond(BBInfo &TrueBBI, BBInfo &FalseBBI,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const;
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool ValidForkedDiamond(BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const;
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
void AnalyzeBranches(BBInfo &BBI);
void ScanInstructions(BBInfo &BBI,
MachineBasicBlock::iterator &Begin,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
MachineBasicBlock::iterator &End,
bool BranchUnpredicable = false) const;
bool RescanInstructions(
MachineBasicBlock::iterator &TIB, MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE, MachineBasicBlock::iterator &FIE,
BBInfo &TrueBBI, BBInfo &FalseBBI) const;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
void AnalyzeBlock(MachineBasicBlock &MBB,
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
std::vector<std::unique_ptr<IfcvtToken>> &Tokens);
[CodeGen] Fix inconsistent declaration parameter name llvm-svn: 337200
2018-07-17 02:51:40 +08:00
bool FeasibilityAnalysis(BBInfo &BBI, SmallVectorImpl<MachineOperand> &Pred,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool isTriangle = false, bool RevBranch = false,
bool hasCommonTail = false);
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
void AnalyzeBlocks(MachineFunction &MF,
std::vector<std::unique_ptr<IfcvtToken>> &Tokens);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
void InvalidatePreds(MachineBasicBlock &MBB);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool IfConvertSimple(BBInfo &BBI, IfcvtKind Kind);
bool IfConvertTriangle(BBInfo &BBI, IfcvtKind Kind);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool IfConvertDiamondCommon(BBInfo &BBI, BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
bool RemoveBranch, bool MergeAddEdges);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
bool IfConvertDiamond(BBInfo &BBI, IfcvtKind Kind,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
unsigned NumDups1, unsigned NumDups2,
bool TClobbers, bool FClobbers);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool IfConvertForkedDiamond(BBInfo &BBI, IfcvtKind Kind,
unsigned NumDups1, unsigned NumDups2,
bool TClobbers, bool FClobbers);
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
void PredicateBlock(BBInfo &BBI,
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
MachineBasicBlock::iterator E,
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
SmallVectorImpl<MachineOperand> &Cond,
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SmallSet<MCPhysReg, 4> *LaterRedefs = nullptr);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
void CopyAndPredicateBlock(BBInfo &ToBBI, BBInfo &FromBBI,
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVectorImpl<MachineOperand> &Cond,
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
bool IgnoreBr = false);
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
void MergeBlocks(BBInfo &ToBBI, BBInfo &FromBBI, bool AddEdges = true);
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
bool MeetIfcvtSizeLimit(MachineBasicBlock &BB,
unsigned Cycle, unsigned Extra,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const {
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
return Cycle > 0 && TII->isProfitableToIfCvt(BB, Cycle, Extra,
Use BranchProbability instead of floating points in IfConverter. llvm-svn: 134858
2011-07-10 10:58:07 +08:00
Prediction);
Change if-conversion block size limit checks to add some flexibility. llvm-svn: 106901
2010-06-26 06:42:03 +08:00
}
[IfCvt][ARM] Optimise diamond if-conversion for code size Currently, the heuristics the if-conversion pass uses for diamond if-conversion are based on execution time, with no consideration for code size. This adds a new set of heuristics to be used when optimising for code size. This is mostly target-independent, because the if-conversion pass can see the code size of the instructions which it is removing. For thumb, there are a few passes (insertion of IT instructions, selection of narrow branches, and selection of CBZ instructions) which are run after if conversion and affect these heuristics, so I've added target hooks to better predict the code-size effect of a proposed if-conversion. Differential revision: https://reviews.llvm.org/D67350 llvm-svn: 374301
2019-10-10 17:58:28 +08:00
bool MeetIfcvtSizeLimit(BBInfo &TBBInfo, BBInfo &FBBInfo,
MachineBasicBlock &CommBB, unsigned Dups,
BranchProbability Prediction, bool Forked) const {
const MachineFunction &MF = *TBBInfo.BB->getParent();
if (MF.getFunction().hasMinSize()) {
MachineBasicBlock::iterator TIB = TBBInfo.BB->begin();
MachineBasicBlock::iterator FIB = FBBInfo.BB->begin();
MachineBasicBlock::iterator TIE = TBBInfo.BB->end();
MachineBasicBlock::iterator FIE = FBBInfo.BB->end();
unsigned Dups1, Dups2;
if (!CountDuplicatedInstructions(TIB, FIB, TIE, FIE, Dups1, Dups2,
*TBBInfo.BB, *FBBInfo.BB,
/*SkipUnconditionalBranches*/ true))
llvm_unreachable("should already have been checked by ValidDiamond");
unsigned BranchBytes = 0;
unsigned CommonBytes = 0;
// Count common instructions at the start of the true and false blocks.
for (auto &I : make_range(TBBInfo.BB->begin(), TIB)) {
LLVM_DEBUG(dbgs() << "Common inst: " << I);
CommonBytes += TII->getInstSizeInBytes(I);
}
for (auto &I : make_range(FBBInfo.BB->begin(), FIB)) {
LLVM_DEBUG(dbgs() << "Common inst: " << I);
CommonBytes += TII->getInstSizeInBytes(I);
}
// Count instructions at the end of the true and false blocks, after
// the ones we plan to predicate. Analyzable branches will be removed
// (unless this is a forked diamond), and all other instructions are
// common between the two blocks.
for (auto &I : make_range(TIE, TBBInfo.BB->end())) {
if (I.isBranch() && TBBInfo.IsBrAnalyzable && !Forked) {
LLVM_DEBUG(dbgs() << "Saving branch: " << I);
BranchBytes += TII->predictBranchSizeForIfCvt(I);
} else {
LLVM_DEBUG(dbgs() << "Common inst: " << I);
CommonBytes += TII->getInstSizeInBytes(I);
}
}
for (auto &I : make_range(FIE, FBBInfo.BB->end())) {
if (I.isBranch() && FBBInfo.IsBrAnalyzable && !Forked) {
LLVM_DEBUG(dbgs() << "Saving branch: " << I);
BranchBytes += TII->predictBranchSizeForIfCvt(I);
} else {
LLVM_DEBUG(dbgs() << "Common inst: " << I);
CommonBytes += TII->getInstSizeInBytes(I);
}
}
for (auto &I : CommBB.terminators()) {
if (I.isBranch()) {
LLVM_DEBUG(dbgs() << "Saving branch: " << I);
BranchBytes += TII->predictBranchSizeForIfCvt(I);
}
}
// The common instructions in one branch will be eliminated, halving
// their code size.
CommonBytes /= 2;
// Count the instructions which we need to predicate.
unsigned NumPredicatedInstructions = 0;
for (auto &I : make_range(TIB, TIE)) {
if (!I.isDebugInstr()) {
LLVM_DEBUG(dbgs() << "Predicating: " << I);
NumPredicatedInstructions++;
}
}
for (auto &I : make_range(FIB, FIE)) {
if (!I.isDebugInstr()) {
LLVM_DEBUG(dbgs() << "Predicating: " << I);
NumPredicatedInstructions++;
}
}
// Even though we're optimising for size at the expense of performance,
// avoid creating really long predicated blocks.
if (NumPredicatedInstructions > 15)
return false;
// Some targets (e.g. Thumb2) need to insert extra instructions to
// start predicated blocks.
unsigned ExtraPredicateBytes = TII->extraSizeToPredicateInstructions(
MF, NumPredicatedInstructions);
LLVM_DEBUG(dbgs() << "MeetIfcvtSizeLimit(BranchBytes=" << BranchBytes
<< ", CommonBytes=" << CommonBytes
<< ", NumPredicatedInstructions="
<< NumPredicatedInstructions
<< ", ExtraPredicateBytes=" << ExtraPredicateBytes
<< ")\n");
return (BranchBytes + CommonBytes) > ExtraPredicateBytes;
} else {
unsigned TCycle = TBBInfo.NonPredSize + TBBInfo.ExtraCost - Dups;
unsigned FCycle = FBBInfo.NonPredSize + FBBInfo.ExtraCost - Dups;
bool Res = TCycle > 0 && FCycle > 0 &&
TII->isProfitableToIfCvt(
*TBBInfo.BB, TCycle, TBBInfo.ExtraCost2, *FBBInfo.BB,
FCycle, FBBInfo.ExtraCost2, Prediction);
LLVM_DEBUG(dbgs() << "MeetIfcvtSizeLimit(TCycle=" << TCycle
<< ", FCycle=" << FCycle
<< ", TExtra=" << TBBInfo.ExtraCost2 << ", FExtra="
<< FBBInfo.ExtraCost2 << ") = " << Res << "\n");
return Res;
}
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true if Block ends without a terminator.
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
bool blockAlwaysFallThrough(BBInfo &BBI) const {
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
return BBI.IsBrAnalyzable && BBI.TrueBB == nullptr;
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Used to sort if-conversion candidates.
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
static bool IfcvtTokenCmp(const std::unique_ptr<IfcvtToken> &C1,
const std::unique_ptr<IfcvtToken> &C2) {
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
int Incr1 = (C1->Kind == ICDiamond)
? -(int)(C1->NumDups + C1->NumDups2) : (int)C1->NumDups;
int Incr2 = (C2->Kind == ICDiamond)
? -(int)(C2->NumDups + C2->NumDups2) : (int)C2->NumDups;
if (Incr1 > Incr2)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
return true;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
else if (Incr1 == Incr2) {
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Favors subsumption.
Simplify expressions involving boolean constants with clang-tidy Patch by Richard (legalize at xmission dot com). Differential Revision: http://reviews.llvm.org/D8154 llvm-svn: 231617
2015-03-09 09:57:13 +08:00
if (!C1->NeedSubsumption && C2->NeedSubsumption)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
return true;
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
else if (C1->NeedSubsumption == C2->NeedSubsumption) {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
// Favors diamond over triangle, etc.
if ((unsigned)C1->Kind < (unsigned)C2->Kind)
return true;
else if (C1->Kind == C2->Kind)
return C1->BBI.BB->getNumber() < C2->BBI.BB->getNumber();
}
}
return false;
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
};
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
} // end anonymous namespace
char IfConverter::ID = 0;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Codegen pass definition cleanup. No functionality. Moving toward a uniform style of pass definition to allow easier target configuration. Globally declare Pass ID. Globally declare pass initializer. Use INITIALIZE_PASS consistently. Add a call to the initializer from CodeGen.cpp. Remove redundant "createPass" functions and "getPassName" methods. While cleaning up declarations, cleaned up comments (sorry for large diff). llvm-svn: 150100
2012-02-09 05:23:13 +08:00
char &llvm::IfConverterID = IfConverter::ID;
CodeGen: Rename DEBUG_TYPE to match passnames Rename the DEBUG_TYPE to match the names of corresponding passes where it makes sense. Also establish the pattern of simply referencing DEBUG_TYPE instead of repeating the passname where possible. llvm-svn: 303921
2017-05-26 05:26:32 +08:00
INITIALIZE_PASS_BEGIN(IfConverter, DEBUG_TYPE, "If Converter", false, false)
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
INITIALIZE_PASS_DEPENDENCY(MachineBranchProbabilityInfo)
CodeGen: Rename DEBUG_TYPE to match passnames Rename the DEBUG_TYPE to match the names of corresponding passes where it makes sense. Also establish the pattern of simply referencing DEBUG_TYPE instead of repeating the passname where possible. llvm-svn: 303921
2017-05-26 05:26:32 +08:00
INITIALIZE_PASS_END(IfConverter, DEBUG_TYPE, "If Converter", false, false)
Revert r85346 change to control tail merging by CodeGenOpt::Level. I'm going to redo this using the OptimizeForSize function attribute. llvm-svn: 85426
2009-10-29 04:46:46 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
bool IfConverter::runOnMachineFunction(MachineFunction &MF) {
MachineFunction: Return reference from getFunction(); NFC The Function can never be nullptr so we can return a reference. llvm-svn: 320884
2017-12-16 06:22:58 +08:00
if (skipFunction(MF.getFunction()) || (PredicateFtor && !PredicateFtor(MF)))
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
return false;
The subtarget is cached on the MachineFunction. Access it directly. llvm-svn: 227173
2015-01-27 15:31:29 +08:00
const TargetSubtargetInfo &ST = MF.getSubtarget();
TLI = ST.getTargetLowering();
TII = ST.getInstrInfo();
TRI = ST.getRegisterInfo();
Reapply "[MBP] Reduce code size by running tail merging in MBP."" This reapplies commit r271930, r271915, r271923. They hit a bug in Thumb which is fixed in r272258 now. The original message: The code layout that TailMerging (inside BranchFolding) works on is not the final layout optimized based on the branch probability. Generally, after BlockPlacement, many new merging opportunities emerge. This patch calls Tail Merging after MBP and calls MBP again if Tail Merging merges anything. llvm-svn: 272267
2016-06-09 23:24:29 +08:00
BranchFolder::MBFIWrapper MBFI(getAnalysis<MachineBlockFrequencyInfo>());
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
MBPI = &getAnalysis<MachineBranchProbabilityInfo>();
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
MRI = &MF.getRegInfo();
[TargetSchedule] shrink interface for init(); NFCI The TargetSchedModel is always initialized using the TargetSubtargetInfo's MCSchedModel and TargetInstrInfo, so we don't need to extract those and pass 3 parameters to init(). Differential Revision: https://reviews.llvm.org/D44789 llvm-svn: 329540
2018-04-09 03:56:04 +08:00
SchedModel.init(&ST);
IfConverter: Use TargetSchedule for instruction latencies For targets that have instruction itineraries this means no change. Targets that move over to the new schedule model will use be able the new schedule module for instruction latencies in the if-converter (the logic is such that if there is no itineary we will use the new sched model for the latencies). Before, we queried "TTI->getInstructionLatency()" for the instruction latency and the extra prediction cost. Now, we query the TargetSchedule abstraction for the instruction latency and TargetInstrInfo for the extra predictation cost. The TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if an itinerary exists, otherwise it will use the new schedule model. ATTENTION: Out of tree targets! (I will also send out an email later to LLVMDev) This means, if your target implements unsigned getInstrLatency(const InstrItineraryData *ItinData, const MachineInstr *MI, unsigned *PredCost); and returns a value for "PredCost", you now also need to implement unsigned getPredictationCost(const MachineInstr *MI); (if your target uses the IfConversion.cpp pass) radar://15077010 llvm-svn: 191671
2013-09-30 23:28:56 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
if (!TII) return false;
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
PreRegAlloc = MRI->isSSA();
bool BFChange = false;
if (!PreRegAlloc) {
// Tail merge tend to expose more if-conversion opportunities.
Revert "[PGO][PGSO] Instrument the code gen / target passes." This reverts commit 9a0b5e14075a1f42a72eedb66fd4fde7985d37ac. This seems to break buildbots.
2019-12-07 04:17:32 +08:00
BranchFolder BF(true, false, MBFI, *MBPI);
[NewPM] Port MachineModuleInfo to the new pass manager. Existing clients are converted to use MachineModuleInfoWrapperPass. The new interface is for defining a new pass manager API in CodeGen. Reviewers: fedor.sergeev, philip.pfaffe, chandlerc, arsenm Reviewed By: arsenm, fedor.sergeev Differential Revision: https://reviews.llvm.org/D64183 llvm-svn: 373240
2019-10-01 01:54:50 +08:00
auto *MMIWP = getAnalysisIfAvailable<MachineModuleInfoWrapperPass>();
BFChange = BF.OptimizeFunction(
MF, TII, ST.getRegisterInfo(),
MMIWP ? &MMIWP->getMMI() : nullptr);
Start implementing pre-ra if-converter: using speculation and selects to eliminate branches. llvm-svn: 158234
2012-06-09 05:53:50 +08:00
}
Allow ARM if-converter to be run after post allocation scheduling. - This fixed a number of bugs in if-converter, tail merging, and post-allocation scheduler. If-converter now runs branch folding / tail merging first to maximize if-conversion opportunities. - Also changed the t2IT instruction slightly. It now defines the ITSTATE register which is read by instructions in the IT block. - Added Thumb2 specific hazard recognizer to ensure the scheduler doesn't change the instruction ordering in the IT block (since IT mask has been finalized). It also ensures no other instructions can be scheduled between instructions in the IT block. This is not yet enabled. llvm-svn: 106344
2010-06-19 07:09:54 +08:00
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << "\nIfcvt: function (" << ++FnNum << ") \'"
<< MF.getName() << "\'");
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
if (FnNum < IfCvtFnStart || (IfCvtFnStop != -1 && FnNum > IfCvtFnStop)) {
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << " skipped\n");
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
return false;
}
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << "\n");
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
MF.RenumberBlocks();
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
BBAnalysis.resize(MF.getNumBlockIDs());
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
std::vector<std::unique_ptr<IfcvtToken>> Tokens;
If true / false blocks fallthrough before ifcvt, add unconditional branches to ifcvt'd block. llvm-svn: 37200
2007-05-18 09:55:58 +08:00
MadeChange = false;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
unsigned NumIfCvts = NumSimple + NumSimpleFalse + NumTriangle +
NumTriangleRev + NumTriangleFalse + NumTriangleFRev + NumDiamonds;
while (IfCvtLimit == -1 || (int)NumIfCvts < IfCvtLimit) {
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Do an initial analysis for each basic block and find all the potential
// candidates to perform if-conversion.
IfConversion's AnalyzeBlocks method always returns false; clean it up. llvm-svn: 106027
2010-06-16 02:57:15 +08:00
bool Change = false;
AnalyzeBlocks(MF, Tokens);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
while (!Tokens.empty()) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
std::unique_ptr<IfcvtToken> Token = std::move(Tokens.back());
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Tokens.pop_back();
BBInfo &BBI = Token->BBI;
IfcvtKind Kind = Token->Kind;
fix leakage of IfcvtTokens llvm-svn: 58690
2008-11-04 21:02:59 +08:00
unsigned NumDups = Token->NumDups;
Fix typo. Patch by nlewycky. llvm-svn: 58709
2008-11-05 02:05:30 +08:00
unsigned NumDups2 = Token->NumDups2;
fix leakage of IfcvtTokens llvm-svn: 58690
2008-11-04 21:02:59 +08:00
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
// If the block has been evicted out of the queue or it has already been
// marked dead (due to it being predicated), then skip it.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (BBI.IsDone)
BBI.IsEnqueued = false;
if (!BBI.IsEnqueued)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
continue;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Fix some stupid bugs that have effectively disabled if-conversion. llvm-svn: 37575
2007-06-15 04:28:52 +08:00
BBI.IsEnqueued = false;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
bool RetVal = false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
switch (Kind) {
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
default: llvm_unreachable("Unexpected!");
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
case ICSimple:
Minor statistics counting bug. llvm-svn: 37451
2007-06-06 09:12:44 +08:00
case ICSimpleFalse: {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool isFalse = Kind == ICSimpleFalse;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if ((isFalse && DisableSimpleF) || (!isFalse && DisableSimple)) break;
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << "Ifcvt (Simple"
<< (Kind == ICSimpleFalse ? " false" : "")
<< "): " << printMBBReference(*BBI.BB) << " ("
<< ((Kind == ICSimpleFalse) ? BBI.FalseBB->getNumber()
: BBI.TrueBB->getNumber())
<< ") ");
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
RetVal = IfConvertSimple(BBI, Kind);
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
Update gcc 4.3 warnings fix patch with recent head changes llvm-svn: 47368
2008-02-20 19:10:28 +08:00
if (RetVal) {
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (isFalse) ++NumSimpleFalse;
else ++NumSimple;
Update gcc 4.3 warnings fix patch with recent head changes llvm-svn: 47368
2008-02-20 19:10:28 +08:00
}
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
break;
Minor statistics counting bug. llvm-svn: 37451
2007-06-06 09:12:44 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
case ICTriangle:
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
case ICTriangleRev:
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
case ICTriangleFalse:
Fix the build. llvm-svn: 37537
2007-06-10 08:19:17 +08:00
case ICTriangleFRev: {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool isFalse = Kind == ICTriangleFalse;
bool isRev = (Kind == ICTriangleRev || Kind == ICTriangleFRev);
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
if (DisableTriangle && !isFalse && !isRev) break;
if (DisableTriangleR && !isFalse && isRev) break;
if (DisableTriangleF && isFalse && !isRev) break;
if (DisableTriangleFR && isFalse && isRev) break;
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << "Ifcvt (Triangle");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if (isFalse)
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << " false");
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
if (isRev)
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << " rev");
LLVM_DEBUG(dbgs() << "): " << printMBBReference(*BBI.BB)
<< " (T:" << BBI.TrueBB->getNumber()
<< ",F:" << BBI.FalseBB->getNumber() << ") ");
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
RetVal = IfConvertTriangle(BBI, Kind);
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if (RetVal) {
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
if (isFalse) {
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (isRev) ++NumTriangleFRev;
else ++NumTriangleFalse;
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
} else {
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (isRev) ++NumTriangleRev;
else ++NumTriangle;
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
}
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
break;
Fix the build. llvm-svn: 37537
2007-06-10 08:19:17 +08:00
}
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
case ICDiamond:
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
if (DisableDiamond) break;
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << "Ifcvt (Diamond): " << printMBBReference(*BBI.BB)
<< " (T:" << BBI.TrueBB->getNumber()
<< ",F:" << BBI.FalseBB->getNumber() << ") ");
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
RetVal = IfConvertDiamond(BBI, Kind, NumDups, NumDups2,
Token->TClobbersPred,
Token->FClobbersPred);
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
if (RetVal) ++NumDiamonds;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
break;
[CodeGen] Fix some Clang-tidy modernize-use-default-member-init and Include What You Use warnings; other minor fixes (NFC). llvm-svn: 314046
2017-09-23 07:46:57 +08:00
case ICForkedDiamond:
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
if (DisableForkedDiamond) break;
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << "Ifcvt (Forked Diamond): "
<< printMBBReference(*BBI.BB)
<< " (T:" << BBI.TrueBB->getNumber()
<< ",F:" << BBI.FalseBB->getNumber() << ") ");
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
RetVal = IfConvertForkedDiamond(BBI, Kind, NumDups, NumDups2,
Token->TClobbersPred,
Token->FClobbersPred);
Rename DEBUG macro to LLVM_DEBUG. The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-14 20:53:11 +08:00
LLVM_DEBUG(dbgs() << (RetVal ? "succeeded!" : "failed!") << "\n");
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
if (RetVal) ++NumForkedDiamonds;
break;
}
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
[IfConversion] More simple, correct dead/kill liveness handling Patch by Jesper Antonsson. Differential Revision: https://reviews.llvm.org/D37611 llvm-svn: 313268
2017-09-14 23:53:11 +08:00
if (RetVal && MRI->tracksLiveness())
recomputeLivenessFlags(*BBI.BB);
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
Change |= RetVal;
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
NumIfCvts = NumSimple + NumSimpleFalse + NumTriangle + NumTriangleRev +
NumTriangleFalse + NumTriangleFRev + NumDiamonds;
if (IfCvtLimit != -1 && (int)NumIfCvts >= IfCvtLimit)
Hidden options to help debugging ifcvt issues. llvm-svn: 37523
2007-06-09 03:10:51 +08:00
break;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
if (!Change)
break;
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
MadeChange |= Change;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
If true / false blocks fallthrough before ifcvt, add unconditional branches to ifcvt'd block. llvm-svn: 37200
2007-05-18 09:55:58 +08:00
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Tokens.clear();
If true / false blocks fallthrough before ifcvt, add unconditional branches to ifcvt'd block. llvm-svn: 37200
2007-05-18 09:55:58 +08:00
BBAnalysis.clear();
Fix an inverted condition. llvm-svn: 106330
2010-06-19 06:17:13 +08:00
if (MadeChange && IfCvtBranchFold) {
Revert "[PGO][PGSO] Instrument the code gen / target passes." This reverts commit 9a0b5e14075a1f42a72eedb66fd4fde7985d37ac. This seems to break buildbots.
2019-12-07 04:17:32 +08:00
BranchFolder BF(false, false, MBFI, *MBPI);
[NewPM] Port MachineModuleInfo to the new pass manager. Existing clients are converted to use MachineModuleInfoWrapperPass. The new interface is for defining a new pass manager API in CodeGen. Reviewers: fedor.sergeev, philip.pfaffe, chandlerc, arsenm Reviewed By: arsenm, fedor.sergeev Differential Revision: https://reviews.llvm.org/D64183 llvm-svn: 373240
2019-10-01 01:54:50 +08:00
auto *MMIWP = getAnalysisIfAvailable<MachineModuleInfoWrapperPass>();
BF.OptimizeFunction(
MF, TII, MF.getSubtarget().getRegisterInfo(),
MMIWP ? &MMIWP->getMMI() : nullptr);
Run branch folding if if-converter make some transformations. llvm-svn: 80994
2009-09-04 15:47:40 +08:00
}
Allow ARM if-converter to be run after post allocation scheduling. - This fixed a number of bugs in if-converter, tail merging, and post-allocation scheduler. If-converter now runs branch folding / tail merging first to maximize if-conversion opportunities. - Also changed the t2IT instruction slightly. It now defines the ITSTATE register which is read by instructions in the IT block. - Added Thumb2 specific hazard recognizer to ensure the scheduler doesn't change the instruction ordering in the IT block (since IT mask has been finalized). It also ensures no other instructions can be scheduled between instructions in the IT block. This is not yet enabled. llvm-svn: 106344
2010-06-19 07:09:54 +08:00
MadeChange |= BFChange;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
return MadeChange;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// BB has a fallthrough. Find its 'false' successor given its 'true' successor.
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
static MachineBasicBlock *findFalseBlock(MachineBasicBlock *BB,
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
MachineBasicBlock *TrueBB) {
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock *SuccBB : BB->successors()) {
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
if (SuccBB != TrueBB)
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
return SuccBB;
}
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
return nullptr;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Reverse the condition of the end of the block branch. Swap block's 'true'
/// and 'false' successors.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
bool IfConverter::reverseBranchCondition(BBInfo &BBI) const {
Add a DebugLoc parameter to TargetInstrInfo::InsertBranch(). This addresses a longstanding deficiency noted in many FIXMEs scattered across all the targets. This effectively moves the problem up one level, replacing eleven FIXMEs in the targets with eight FIXMEs in CodeGen, plus one path through FastISel where we actually supply a DebugLoc, fixing Radar 7421831. llvm-svn: 106243
2010-06-18 06:43:56 +08:00
DebugLoc dl; // FIXME: this is nowhere
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (!TII->reverseBranchCondition(BBI.BrCond)) {
TII->removeBranch(*BBI.BB);
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(*BBI.BB, BBI.FalseBB, BBI.TrueBB, BBI.BrCond, dl);
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
std::swap(BBI.TrueBB, BBI.FalseBB);
return true;
}
return false;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns the next block in the function blocks ordering. If it is the end,
/// returns NULL.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
static inline MachineBasicBlock *getNextBlock(MachineBasicBlock &MBB) {
MachineFunction::iterator I = MBB.getIterator();
MachineFunction::iterator E = MBB.getParent()->end();
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
if (++I == E)
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
return nullptr;
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
return &*I;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true if the 'true' block (along with its predecessor) forms a valid
/// simple shape for ifcvt. It also returns the number of instructions that the
/// ifcvt would need to duplicate if performed in Dups.
Give the if-converter access to MachineLoopInfo, and use it to generate plausible branch prediction estimates. llvm-svn: 114981
2010-09-29 04:42:15 +08:00
bool IfConverter::ValidSimple(BBInfo &TrueBBI, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = 0;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
Avoid if-converting simple block that ends with unconditional branch or fallthrough unless it branches / falls to the 'false' block. Not profitable, may end up increasing code size. llvm-svn: 37660
2007-06-20 05:45:13 +08:00
if (TrueBBI.IsBrAnalyzable)
return false;
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
if (TrueBBI.BB->pred_size() > 1) {
if (TrueBBI.CannotBeCopied ||
Give the if-converter access to MachineLoopInfo, and use it to generate plausible branch prediction estimates. llvm-svn: 114981
2010-09-29 04:42:15 +08:00
!TII->isProfitableToDupForIfCvt(*TrueBBI.BB, TrueBBI.NonPredSize,
Use BranchProbability instead of floating points in IfConverter. llvm-svn: 134858
2011-07-10 10:58:07 +08:00
Prediction))
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = TrueBBI.NonPredSize;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Avoid if-converting simple block that ends with unconditional branch or fallthrough unless it branches / falls to the 'false' block. Not profitable, may end up increasing code size. llvm-svn: 37660
2007-06-20 05:45:13 +08:00
return true;
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true if the 'true' and 'false' blocks (along with their common
/// predecessor) forms a valid triangle shape for ifcvt. If 'FalseBranch' is
/// true, it checks if 'true' block's false branch branches to the 'false' block
/// rather than the other way around. It also returns the number of instructions
/// that the ifcvt would need to duplicate if performed in 'Dups'.
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
bool IfConverter::ValidTriangle(BBInfo &TrueBBI, BBInfo &FalseBBI,
Thread the determination of branch prediction hit rates back through the if-conversion heuristic APIs. For now, stick with a constant estimate of 90% (branch predictors are good!), but we might find that we want to provide more nuanced estimates in the future. llvm-svn: 115364
2010-10-02 06:45:50 +08:00
bool FalseBranch, unsigned &Dups,
Pass BranchProbability/BlockMass by value instead of const& as they are small. NFC. llvm-svn: 247357
2015-09-11 07:10:42 +08:00
BranchProbability Prediction) const {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = 0;
[IfConversion] Disallow TBB == FBB for valid triangles Summary: Previously the case EBB | \_ | | | TBB | / FBB was treated as a valid triangle also when TBB and FBB was the same basic block. This could then lead to an invalid CFG when we removed the edge from EBB to TBB, since that meant we would also remove the edge from EBB to FBB. Since TBB == FBB is quite a degenerated case of a triangle, we now don't treat it as a valid triangle anymore, and thus we will avoid the trouble with updating the CFG. Reviewers: efriedma, dmgreen, kparzysz Reviewed By: efriedma Subscribers: bjope, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D67832 llvm-svn: 372943
2019-09-26 14:35:55 +08:00
if (TrueBBI.BB == FalseBBI.BB)
return false;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
if (TrueBBI.BB->pred_size() > 1) {
if (TrueBBI.CannotBeCopied)
return false;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
unsigned Size = TrueBBI.NonPredSize;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (TrueBBI.IsBrAnalyzable) {
Use empty() instead of comparing size() with zero. llvm-svn: 46514
2008-01-29 21:02:09 +08:00
if (TrueBBI.TrueBB && TrueBBI.BrCond.empty())
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Ends with an unconditional branch. It will be removed.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
--Size;
else {
MachineBasicBlock *FExit = FalseBranch
? TrueBBI.TrueBB : TrueBBI.FalseBB;
if (FExit)
// Require a conditional branch
++Size;
}
}
Use BranchProbability instead of floating points in IfConverter. llvm-svn: 134858
2011-07-10 10:58:07 +08:00
if (!TII->isProfitableToDupForIfCvt(*TrueBBI.BB, Size, Prediction))
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Dups = Size;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
MachineBasicBlock *TExit = FalseBranch ? TrueBBI.FalseBB : TrueBBI.TrueBB;
if (!TExit && blockAlwaysFallThrough(TrueBBI)) {
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
MachineFunction::iterator I = TrueBBI.BB->getIterator();
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
if (++I == TrueBBI.BB->getParent()->end())
return false;
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
TExit = &*I;
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
return TExit && TExit == FalseBBI.BB;
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
IfConversion: Document countDuplicatedInstructions. NFC llvm-svn: 277903
2016-08-06 09:52:33 +08:00
/// Count duplicated instructions and move the iterators to show where they
/// are.
/// @param TIB True Iterator Begin
/// @param FIB False Iterator Begin
/// These two iterators initially point to the first instruction of the two
/// blocks, and finally point to the first non-shared instruction.
/// @param TIE True Iterator End
/// @param FIE False Iterator End
/// These two iterators initially point to End() for the two blocks() and
/// finally point to the first shared instruction in the tail.
/// Upon return [TIB, TIE), and [FIB, FIE) mark the un-duplicated portions of
/// two blocks.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
/// @param Dups1 count of duplicated instructions at the beginning of the 2
/// blocks.
/// @param Dups2 count of duplicated instructions at the end of the 2 blocks.
/// @param SkipUnconditionalBranches if true, Don't make sure that
/// unconditional branches at the end of the blocks are the same. True is
/// passed when the blocks are analyzable to allow for fallthrough to be
/// handled.
/// @return false if the shared portion prevents if conversion.
bool IfConverter::CountDuplicatedInstructions(
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
MachineBasicBlock::iterator &TIB,
MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE,
MachineBasicBlock::iterator &FIE,
unsigned &Dups1, unsigned &Dups2,
MachineBasicBlock &TBB, MachineBasicBlock &FBB,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool SkipUnconditionalBranches) const {
When the "true" and "false" blocks of a diamond if-conversion are the same, do not double-count the duplicate instructions by counting once from the beginning and again from the end. Keep track of where the duplicates from the beginning ended and don't go past that point when counting duplicates at the end. Radar 8589805. This change causes one of the MC/ARM/simple-fp-encoding tests to produce different (better!) code without the vmovne instruction being tested. I changed the test to produce vmovne and vmoveq instructions but moving between register files in the opposite direction. That's not quite the same but predicated versions of those instructions weren't being tested before, so at least the test coverage is not any worse, just different. llvm-svn: 117333
2010-10-26 08:02:24 +08:00
while (TIB != TIE && FIB != FIE) {
Teach iff-converter to properly count # of dups. It was not skipping over dbg_value's which resulted in non-duplicated instructions being deleted. rdar://8104384. llvm-svn: 106323
2010-06-19 05:52:57 +08:00
// Skip dbg_value instructions. These do not count.
[codegen] Add generic functions to skip debug values. Summary: This commits moves skipDebugInstructionsForward and skipDebugInstructionsBackward from lib/CodeGen/IfConversion.cpp to include/llvm/CodeGen/MachineBasicBlock.h and updates some codgen files to use them. This refactoring was suggested in https://reviews.llvm.org/D27688 and I thought it's best to do the refactoring in a separate review, but I could also put both changes in a single review if that's preferred. Also, the names for the functions aren't the snappiest and I would be happy to rename them if anybody has suggestions. Reviewers: eli.friedman, iteratee, aprantl, MatzeB Subscribers: MatzeB, llvm-commits Differential Revision: https://reviews.llvm.org/D27782 llvm-svn: 289933
2016-12-16 19:10:26 +08:00
TIB = skipDebugInstructionsForward(TIB, TIE);
FIB = skipDebugInstructionsForward(FIB, FIE);
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (TIB == TIE || FIB == FIE)
IfConversion: factor out 2 functions to skip debug instrs. NFC Skipping debug instructions occurrs repeatedly, factor it out. llvm-svn: 277902
2016-08-06 09:52:31 +08:00
break;
CodeGen: Change MachineInstr to use MachineInstr&, NFC Change MachineInstr API to prefer MachineInstr& over MachineInstr* whenever the parameter is expected to be non-null. Slowly inching toward being able to fix PR26753. llvm-svn: 262149
2016-02-28 04:01:33 +08:00
if (!TIB->isIdenticalTo(*FIB))
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
break;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// A pred-clobbering instruction in the shared portion prevents
// if-conversion.
std::vector<MachineOperand> PredDefs;
if (TII->DefinesPredicate(*TIB, PredDefs))
return false;
IfConversion: Don't count branches in # of duplicates. If the entire blocks match, we would count the branch instructions toward the number of duplicated instructions. This doesn't match what we do elsewhere, and was causing a bug. llvm-svn: 280448
2016-09-02 09:20:06 +08:00
// If we get all the way to the branch instructions, don't count them.
if (!TIB->isBranch())
++Dups1;
When the "true" and "false" blocks of a diamond if-conversion are the same, do not double-count the duplicate instructions by counting once from the beginning and again from the end. Keep track of where the duplicates from the beginning ended and don't go past that point when counting duplicates at the end. Radar 8589805. This change causes one of the MC/ARM/simple-fp-encoding tests to produce different (better!) code without the vmovne instruction being tested. I changed the test to produce vmovne and vmoveq instructions but moving between register files in the opposite direction. That's not quite the same but predicated versions of those instructions weren't being tested before, so at least the test coverage is not any worse, just different. llvm-svn: 117333
2010-10-26 08:02:24 +08:00
++TIB;
++FIB;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// Check for already containing all of the block.
if (TIB == TIE || FIB == FIE)
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
return true;
IfConversion: Handle inclusive ranges more carefully. This may affect calculations for thresholds, but is not a significant change in behavior. The problem was that an inclusive range must have an additonal flag to showr that it is empty, because otherwise begin == end implies that the range has one element, and it may not be possible to move past on either side. llvm-svn: 279166
2016-08-19 06:09:23 +08:00
// Now, in preparation for counting duplicate instructions at the ends of the
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
// blocks, switch to reverse_iterators. Note that getReverse() returns an
// iterator that points to the same instruction, unlike std::reverse_iterator.
// We have to do our own shifting so that we get the same range.
MachineBasicBlock::reverse_iterator RTIE = std::next(TIE.getReverse());
MachineBasicBlock::reverse_iterator RFIE = std::next(FIE.getReverse());
const MachineBasicBlock::reverse_iterator RTIB = std::next(TIB.getReverse());
const MachineBasicBlock::reverse_iterator RFIB = std::next(FIB.getReverse());
IfConversion: Handle inclusive ranges more carefully. This may affect calculations for thresholds, but is not a significant change in behavior. The problem was that an inclusive range must have an additonal flag to showr that it is empty, because otherwise begin == end implies that the range has one element, and it may not be possible to move past on either side. llvm-svn: 279166
2016-08-19 06:09:23 +08:00
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
if (!TBB.succ_empty() || !FBB.succ_empty()) {
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (SkipUnconditionalBranches) {
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
while (RTIE != RTIB && RTIE->isUnconditionalBranch())
++RTIE;
while (RFIE != RFIB && RFIE->isUnconditionalBranch())
++RFIE;
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
}
When the "true" and "false" blocks of a diamond if-conversion are the same, do not double-count the duplicate instructions by counting once from the beginning and again from the end. Keep track of where the duplicates from the beginning ended and don't go past that point when counting duplicates at the end. Radar 8589805. This change causes one of the MC/ARM/simple-fp-encoding tests to produce different (better!) code without the vmovne instruction being tested. I changed the test to produce vmovne and vmoveq instructions but moving between register files in the opposite direction. That's not quite the same but predicated versions of those instructions weren't being tested before, so at least the test coverage is not any worse, just different. llvm-svn: 117333
2010-10-26 08:02:24 +08:00
}
// Count duplicate instructions at the ends of the blocks.
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
while (RTIE != RTIB && RFIE != RFIB) {
Teach iff-converter to properly count # of dups. It was not skipping over dbg_value's which resulted in non-duplicated instructions being deleted. rdar://8104384. llvm-svn: 106323
2010-06-19 05:52:57 +08:00
// Skip dbg_value instructions. These do not count.
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
// Note that these are reverse iterators going forward.
RTIE = skipDebugInstructionsForward(RTIE, RTIB);
RFIE = skipDebugInstructionsForward(RFIE, RFIB);
if (RTIE == RTIB || RFIE == RFIB)
IfConversion: factor out 2 functions to skip debug instrs. NFC Skipping debug instructions occurrs repeatedly, factor it out. llvm-svn: 277902
2016-08-06 09:52:31 +08:00
break;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (!RTIE->isIdenticalTo(*RFIE))
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
break;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// We have to verify that any branch instructions are the same, and then we
// don't count them toward the # of duplicate instructions.
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (!RTIE->isBranch())
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
++Dups2;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
++RTIE;
++RFIE;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
TIE = std::next(RTIE.getReverse());
FIE = std::next(RFIE.getReverse());
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
return true;
}
/// RescanInstructions - Run ScanInstructions on a pair of blocks.
/// @param TIB - True Iterator Begin, points to first non-shared instruction
/// @param FIB - False Iterator Begin, points to first non-shared instruction
/// @param TIE - True Iterator End, points past last non-shared instruction
/// @param FIE - False Iterator End, points past last non-shared instruction
/// @param TrueBBI - BBInfo to update for the true block.
/// @param FalseBBI - BBInfo to update for the false block.
/// @returns - false if either block cannot be predicated or if both blocks end
/// with a predicate-clobbering instruction.
bool IfConverter::RescanInstructions(
MachineBasicBlock::iterator &TIB, MachineBasicBlock::iterator &FIB,
MachineBasicBlock::iterator &TIE, MachineBasicBlock::iterator &FIE,
BBInfo &TrueBBI, BBInfo &FalseBBI) const {
bool BranchUnpredicable = true;
TrueBBI.IsUnpredicable = FalseBBI.IsUnpredicable = false;
ScanInstructions(TrueBBI, TIB, TIE, BranchUnpredicable);
if (TrueBBI.IsUnpredicable)
return false;
ScanInstructions(FalseBBI, FIB, FIE, BranchUnpredicable);
if (FalseBBI.IsUnpredicable)
return false;
if (TrueBBI.ClobbersPred && FalseBBI.ClobbersPred)
return false;
return true;
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
}
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
#ifndef NDEBUG
static void verifySameBranchInstructions(
MachineBasicBlock *MBB1,
MachineBasicBlock *MBB2) {
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
const MachineBasicBlock::reverse_iterator B1 = MBB1->rend();
const MachineBasicBlock::reverse_iterator B2 = MBB2->rend();
MachineBasicBlock::reverse_iterator E1 = MBB1->rbegin();
MachineBasicBlock::reverse_iterator E2 = MBB2->rbegin();
while (E1 != B1 && E2 != B2) {
skipDebugInstructionsForward(E1, B1);
skipDebugInstructionsForward(E2, B2);
if (E1 == B1 && E2 == B2)
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
break;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (E1 == B1) {
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
assert(!E2->isBranch() && "Branch mis-match, one block is empty.");
break;
}
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
if (E2 == B2) {
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
assert(!E1->isBranch() && "Branch mis-match, one block is empty.");
break;
}
if (E1->isBranch() || E2->isBranch())
assert(E1->isIdenticalTo(*E2) &&
"Branch mis-match, branch instructions don't match.");
else
break;
[IfConversion] Use reverse_iterator to simplify. NFC This simplifies skipping debug instructions and shrinking ranges. llvm-svn: 293202
2017-01-27 04:02:47 +08:00
++E1;
++E2;
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
}
}
#endif
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/// ValidForkedDiamond - Returns true if the 'true' and 'false' blocks (along
/// with their common predecessor) form a diamond if a common tail block is
/// extracted.
/// While not strictly a diamond, this pattern would form a diamond if
/// tail-merging had merged the shared tails.
/// EBB
/// _/ \_
/// | |
/// TBB FBB
/// / \ / \
/// FalseBB TrueBB FalseBB
/// Currently only handles analyzable branches.
/// Specifically excludes actual diamonds to avoid overlap.
bool IfConverter::ValidForkedDiamond(
BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const {
Dups1 = Dups2 = 0;
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone ||
FalseBBI.IsBeingAnalyzed || FalseBBI.IsDone)
return false;
if (!TrueBBI.IsBrAnalyzable || !FalseBBI.IsBrAnalyzable)
return false;
// Don't IfConvert blocks that can't be folded into their predecessor.
if (TrueBBI.BB->pred_size() > 1 || FalseBBI.BB->pred_size() > 1)
return false;
// This function is specifically looking for conditional tails, as
// unconditional tails are already handled by the standard diamond case.
if (TrueBBI.BrCond.size() == 0 ||
FalseBBI.BrCond.size() == 0)
return false;
MachineBasicBlock *TT = TrueBBI.TrueBB;
MachineBasicBlock *TF = TrueBBI.FalseBB;
MachineBasicBlock *FT = FalseBBI.TrueBB;
MachineBasicBlock *FF = FalseBBI.FalseBB;
if (!TT)
TT = getNextBlock(*TrueBBI.BB);
if (!TF)
TF = getNextBlock(*TrueBBI.BB);
if (!FT)
FT = getNextBlock(*FalseBBI.BB);
if (!FF)
FF = getNextBlock(*FalseBBI.BB);
if (!TT || !TF)
return false;
// Check successors. If they don't match, bail.
if (!((TT == FT && TF == FF) || (TF == FT && TT == FF)))
return false;
bool FalseReversed = false;
if (TF == FT && TT == FF) {
// If the branches are opposing, but we can't reverse, don't do it.
if (!FalseBBI.IsBrReversible)
return false;
FalseReversed = true;
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
reverseBranchCondition(FalseBBI);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
}
auto UnReverseOnExit = make_scope_exit([&]() {
if (FalseReversed)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
reverseBranchCondition(FalseBBI);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
});
// Count duplicate instructions at the beginning of the true and false blocks.
MachineBasicBlock::iterator TIB = TrueBBI.BB->begin();
MachineBasicBlock::iterator FIB = FalseBBI.BB->begin();
MachineBasicBlock::iterator TIE = TrueBBI.BB->end();
MachineBasicBlock::iterator FIE = FalseBBI.BB->end();
if(!CountDuplicatedInstructions(TIB, FIB, TIE, FIE, Dups1, Dups2,
*TrueBBI.BB, *FalseBBI.BB,
/* SkipUnconditionalBranches */ true))
return false;
TrueBBICalc.BB = TrueBBI.BB;
FalseBBICalc.BB = FalseBBI.BB;
[IfCvt][ARM] Optimise diamond if-conversion for code size Currently, the heuristics the if-conversion pass uses for diamond if-conversion are based on execution time, with no consideration for code size. This adds a new set of heuristics to be used when optimising for code size. This is mostly target-independent, because the if-conversion pass can see the code size of the instructions which it is removing. For thumb, there are a few passes (insertion of IT instructions, selection of narrow branches, and selection of CBZ instructions) which are run after if conversion and affect these heuristics, so I've added target hooks to better predict the code-size effect of a proposed if-conversion. Differential revision: https://reviews.llvm.org/D67350 llvm-svn: 374301
2019-10-10 17:58:28 +08:00
TrueBBICalc.IsBrAnalyzable = TrueBBI.IsBrAnalyzable;
FalseBBICalc.IsBrAnalyzable = FalseBBI.IsBrAnalyzable;
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
if (!RescanInstructions(TIB, FIB, TIE, FIE, TrueBBICalc, FalseBBICalc))
return false;
// The size is used to decide whether to if-convert, and the shared portions
// are subtracted off. Because of the subtraction, we just use the size that
// was calculated by the original ScanInstructions, as it is correct.
TrueBBICalc.NonPredSize = TrueBBI.NonPredSize;
FalseBBICalc.NonPredSize = FalseBBI.NonPredSize;
return true;
}
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
/// ValidDiamond - Returns true if the 'true' and 'false' blocks (along
/// with their common predecessor) forms a valid diamond shape for ifcvt.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool IfConverter::ValidDiamond(
BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned &Dups1, unsigned &Dups2,
BBInfo &TrueBBICalc, BBInfo &FalseBBICalc) const {
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
Dups1 = Dups2 = 0;
if (TrueBBI.IsBeingAnalyzed || TrueBBI.IsDone ||
FalseBBI.IsBeingAnalyzed || FalseBBI.IsDone)
return false;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
MachineBasicBlock *TT = TrueBBI.TrueBB;
MachineBasicBlock *FT = FalseBBI.TrueBB;
if (!TT && blockAlwaysFallThrough(TrueBBI))
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
TT = getNextBlock(*TrueBBI.BB);
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
if (!FT && blockAlwaysFallThrough(FalseBBI))
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
FT = getNextBlock(*FalseBBI.BB);
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
if (TT != FT)
return false;
if (!TT && (TrueBBI.IsBrAnalyzable || FalseBBI.IsBrAnalyzable))
return false;
if (TrueBBI.BB->pred_size() > 1 || FalseBBI.BB->pred_size() > 1)
return false;
// FIXME: Allow true block to have an early exit?
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (TrueBBI.FalseBB || FalseBBI.FalseBB)
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
return false;
// Count duplicate instructions at the beginning and end of the true and
// false blocks.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// Skip unconditional branches only if we are considering an analyzable
// diamond. Otherwise the branches must be the same.
bool SkipUnconditionalBranches =
TrueBBI.IsBrAnalyzable && FalseBBI.IsBrAnalyzable;
Codegen: IfConversion: Factor out a function to count dup instrs. Factor out countDuplicatedInstructions to Count duplicated instructions at the beginning and end of a diamond pattern. This is in prep for adding support for diamonds that need to be tail-merged. llvm-svn: 276910
2016-07-28 04:19:33 +08:00
MachineBasicBlock::iterator TIB = TrueBBI.BB->begin();
MachineBasicBlock::iterator FIB = FalseBBI.BB->begin();
MachineBasicBlock::iterator TIE = TrueBBI.BB->end();
MachineBasicBlock::iterator FIE = FalseBBI.BB->end();
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if(!CountDuplicatedInstructions(TIB, FIB, TIE, FIE, Dups1, Dups2,
*TrueBBI.BB, *FalseBBI.BB,
SkipUnconditionalBranches))
return false;
TrueBBICalc.BB = TrueBBI.BB;
FalseBBICalc.BB = FalseBBI.BB;
[IfCvt][ARM] Optimise diamond if-conversion for code size Currently, the heuristics the if-conversion pass uses for diamond if-conversion are based on execution time, with no consideration for code size. This adds a new set of heuristics to be used when optimising for code size. This is mostly target-independent, because the if-conversion pass can see the code size of the instructions which it is removing. For thumb, there are a few passes (insertion of IT instructions, selection of narrow branches, and selection of CBZ instructions) which are run after if conversion and affect these heuristics, so I've added target hooks to better predict the code-size effect of a proposed if-conversion. Differential revision: https://reviews.llvm.org/D67350 llvm-svn: 374301
2019-10-10 17:58:28 +08:00
TrueBBICalc.IsBrAnalyzable = TrueBBI.IsBrAnalyzable;
FalseBBICalc.IsBrAnalyzable = FalseBBI.IsBrAnalyzable;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (!RescanInstructions(TIB, FIB, TIE, FIE, TrueBBICalc, FalseBBICalc))
return false;
// The size is used to decide whether to if-convert, and the shared portions
// are subtracted off. Because of the subtraction, we just use the size that
// was calculated by the original ScanInstructions, as it is correct.
TrueBBICalc.NonPredSize = TrueBBI.NonPredSize;
FalseBBICalc.NonPredSize = FalseBBI.NonPredSize;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
return true;
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
/// AnalyzeBranches - Look at the branches at the end of a block to determine if
/// the block is predicable.
void IfConverter::AnalyzeBranches(BBInfo &BBI) {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (BBI.IsDone)
return;
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
BBI.TrueBB = BBI.FalseBB = nullptr;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.BrCond.clear();
BBI.IsBrAnalyzable =
Rename AnalyzeBranch* to analyzeBranch*. Summary: NFC. Rename AnalyzeBranch/AnalyzeBranchPredicate to analyzeBranch/analyzeBranchPredicate to follow LLVM coding style and be consistent with TargetInstrInfo's analyzeCompare and analyzeSelect. Reviewers: tstellarAMD, mcrosier Subscribers: mcrosier, jholewinski, jfb, arsenm, dschuff, jyknight, dsanders, nemanjai Differential Revision: https://reviews.llvm.org/D22409 llvm-svn: 275564
2016-07-15 22:41:04 +08:00
!TII->analyzeBranch(*BBI.BB, BBI.TrueBB, BBI.FalseBB, BBI.BrCond);
[IfConversion] Correctly handle cases where analyzeBranch fails. If analyzeBranch fails, on some targets, the out parameters point to some blocks in the function. But we can't use that information, so make sure to clear it out. (In some places in IfConversion, we assume that any block with a TrueBB is analyzable.) The change to the testcase makes it trigger a bug on builds without this fix: IfConvertDiamond tries to perform a followup "merge" operation, which isn't legal, and we somehow end up with a branch to a deleted MBB. I'm not sure how this doesn't crash the compiler. Differential Revision: https://reviews.llvm.org/D67306 llvm-svn: 371434
2019-09-10 02:29:27 +08:00
if (!BBI.IsBrAnalyzable) {
BBI.TrueBB = nullptr;
BBI.FalseBB = nullptr;
BBI.BrCond.clear();
}
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
SmallVector<MachineOperand, 4> RevCond(BBI.BrCond.begin(), BBI.BrCond.end());
BBI.IsBrReversible = (RevCond.size() == 0) ||
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
!TII->reverseBranchCondition(RevCond);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
BBI.HasFallThrough = BBI.IsBrAnalyzable && BBI.FalseBB == nullptr;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (BBI.BrCond.size()) {
// No false branch. This BB must end with a conditional branch and a
// fallthrough.
if (!BBI.FalseBB)
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
BBI.FalseBB = findFalseBlock(BBI.BB, BBI.TrueBB);
ifcvt should ignore cfg where true and false successors are the same. llvm-svn: 73423
2009-06-16 05:24:34 +08:00
if (!BBI.FalseBB) {
// Malformed bcc? True and false blocks are the same?
BBI.IsUnpredicable = true;
}
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
}
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
}
/// ScanInstructions - Scan all the instructions in the block to determine if
/// the block is predicable. In most cases, that means all the instructions
/// in the block are isPredicable(). Also checks if the block contains any
/// instruction which can clobber a predicate (e.g. condition code register).
/// If so, the block is not predicable unless it's the last instruction.
void IfConverter::ScanInstructions(BBInfo &BBI,
MachineBasicBlock::iterator &Begin,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
MachineBasicBlock::iterator &End,
bool BranchUnpredicable) const {
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
if (BBI.IsDone || BBI.IsUnpredicable)
return;
bool AlreadyPredicated = !BBI.Predicate.empty();
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.NonPredSize = 0;
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
BBI.ExtraCost = 0;
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
BBI.ExtraCost2 = 0;
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.ClobbersPred = false;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineInstr &MI : make_range(Begin, End)) {
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (MI.isDebugInstr())
Make if-conversion ignore dbg_value instructions in its analysis. rdar://7797940 llvm-svn: 105498
2010-06-05 07:01:26 +08:00
continue;
[ifcnv] Don't duplicate blocks that contain convergent instructions. It's unsafe to duplicate blocks that contain convergent instructions during ifcnv. See the patch for details. Reviewers: hfinkel Differential Revision: http://reviews.llvm.org/D17518 llvm-svn: 266404
2016-04-15 09:38:41 +08:00
// It's unsafe to duplicate convergent instructions in this context, so set
// BBI.CannotBeCopied to true if MI is convergent. To see why, consider the
// following CFG, which is subject to our "simple" transformation.
//
// BB0 // if (c1) goto BB1; else goto BB2;
// / \
// BB1 |
// | BB2 // if (c2) goto TBB; else goto FBB;
// | / |
// | / |
// TBB |
// | |
// | FBB
// |
// exit
//
// Suppose we want to move TBB's contents up into BB1 and BB2 (in BB1 they'd
// be unconditional, and in BB2, they'd be predicated upon c2), and suppose
// TBB contains a convergent instruction. This is safe iff doing so does
// not add a control-flow dependency to the convergent instruction -- i.e.,
// it's safe iff the set of control flows that leads us to the convergent
// instruction does not get smaller after the transformation.
//
// Originally we executed TBB if c1 || c2. After the transformation, there
// are two copies of TBB's instructions. We get to the first if c1, and we
// get to the second if !c1 && c2.
//
// There are clearly fewer ways to satisfy the condition "c1" than
// "c1 || c2". Since we've shrunk the set of control flows which lead to
// our convergent instruction, the transformation is unsafe.
if (MI.isNotDuplicable() || MI.isConvergent())
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
BBI.CannotBeCopied = true;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
bool isPredicated = TII->isPredicated(MI);
bool isCondBr = BBI.IsBrAnalyzable && MI.isConditionalBranch();
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (BranchUnpredicable && MI.isBranch()) {
BBI.IsUnpredicable = true;
return;
}
[ARMv8] Prevent generation of deprecated IT blocks on ARMv8 in Thumb mode. IT blocks can only be one instruction lonf, and can only contain a subset of the 16 instructions. Patch by Artyom Skrobov! llvm-svn: 190309
2013-09-09 22:21:49 +08:00
// A conditional branch is not predicable, but it may be eliminated.
if (isCondBr)
continue;
if (!isPredicated) {
BBI.NonPredSize++;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
unsigned ExtraPredCost = TII->getPredicationCost(MI);
unsigned NumCycles = SchedModel.computeInstrLatency(&MI, false);
[ARMv8] Prevent generation of deprecated IT blocks on ARMv8 in Thumb mode. IT blocks can only be one instruction lonf, and can only contain a subset of the 16 instructions. Patch by Artyom Skrobov! llvm-svn: 190309
2013-09-09 22:21:49 +08:00
if (NumCycles > 1)
BBI.ExtraCost += NumCycles-1;
BBI.ExtraCost2 += ExtraPredCost;
} else if (!AlreadyPredicated) {
// FIXME: This instruction is already predicated before the
// if-conversion pass. It's probably something like a conditional move.
// Mark this block unpredicable for now.
BBI.IsUnpredicable = true;
return;
Teach if-conversion about instructions that were already predicated, e.g. conditional move. llvm-svn: 37964
2007-07-07 07:24:39 +08:00
}
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (BBI.ClobbersPred && !isPredicated) {
// Predicate modification instruction should end the block (except for
// already predicated instructions and end of block branches).
// Predicate may have been modified, the subsequent (currently)
Teach if-conversion about instructions that were already predicated, e.g. conditional move. llvm-svn: 37964
2007-07-07 07:24:39 +08:00
// unpredicated instructions cannot be correctly predicated.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsUnpredicable = true;
return;
}
Somehow this wasn't committed last time. M_CLOBBERS_PRED is gone. llvm-svn: 38495
2007-07-11 01:50:43 +08:00
// FIXME: Make use of PredDefs? e.g. ADDC, SUBC sets predicates but are
// still potentially predicable.
std::vector<MachineOperand> PredDefs;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (TII->DefinesPredicate(MI, PredDefs))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.ClobbersPred = true;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (!TII->isPredicable(MI)) {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsUnpredicable = true;
return;
}
}
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Determine if the block is a suitable candidate to be predicated by the
/// specified predicate.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
/// @param BBI BBInfo for the block to check
/// @param Pred Predicate array for the branch that leads to BBI
/// @param isTriangle true if the Analysis is for a triangle
/// @param RevBranch true if Reverse(Pred) leads to BBI (e.g. BBI is the false
/// case
/// @param hasCommonTail true if BBI shares a tail with a sibling block that
/// contains any instruction that would make the block unpredicable.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
bool IfConverter::FeasibilityAnalysis(BBInfo &BBI,
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVectorImpl<MachineOperand> &Pred,
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
bool isTriangle, bool RevBranch,
bool hasCommonTail) {
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
// If the block is dead or unpredicable, then it cannot be predicated.
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
// Two blocks may share a common unpredicable tail, but this doesn't prevent
// them from being if-converted. The non-shared portion is assumed to have
// been checked
if (BBI.IsDone || (BBI.IsUnpredicable && !hasCommonTail))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
[CodeGen][IfCvt] Don't re-ifcvt blocks with unanalyzable terminators. If we couldn't analyze its terminator (i.e., it's an indirectbr, or some other weirdness), we can't safely re-if-convert a predicated block, because we can't tell whether the predicated terminator can fallthrough (it does). Currently, we would completely ignore the fallthrough successor. In the added testcase, this means we used to generate: ... @ %entry: cmp r5, #21 ittt ne @ %cc1f: cmpne r7, #42 @ %cc2t: strne.w r5, [r8] movne pc, r10 @ %cc1t: ... Whereas the successor of %cc1f was originally %bb1. With the fix, we get the correct: ... @ %entry: cmp r5, #21 itt eq @ %cc1t: streq.w r5, [r11] moveq pc, r0 @ %cc1f: cmp r7, #42 itt ne @ %cc2t: strne.w r5, [r8] movne pc, r10 @ %bb1: ... rdar://20192768 Differential Revision: http://reviews.llvm.org/D8509 llvm-svn: 232872
2015-03-21 09:23:15 +08:00
// If it is already predicated but we couldn't analyze its terminator, the
// latter might fallthrough, but we can't determine where to.
// Conservatively avoid if-converting again.
if (BBI.Predicate.size() && !BBI.IsBrAnalyzable)
return false;
Fix a bug in IfConverter with nested predicates. Prior to this patch, IfConverter may widen the cases where a sequence of instructions were executed because of the way it uses nested predicates. This result in incorrect execution. For instance, Let A be a basic block that flows conditionally into B and B be a predicated block. B can be predicated with A.BrToBPredicate into A iff B.Predicate is less "permissive" than A.BrToBPredicate, i.e., iff A.BrToBPredicate subsumes B.Predicate. The IfConverter was checking the opposite: B.Predicate subsumes A.BrToBPredicate. <rdar://problem/14379453> llvm-svn: 187071
2013-07-25 04:20:37 +08:00
// If it is already predicated, check if the new predicate subsumes
// its predicate.
if (BBI.Predicate.size() && !TII->SubsumesPredicate(Pred, BBI.Predicate))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (!hasCommonTail && BBI.BrCond.size()) {
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (!isTriangle)
return false;
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Test predicate subsumption.
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> RevPred(Pred.begin(), Pred.end());
SmallVector<MachineOperand, 4> Cond(BBI.BrCond.begin(), BBI.BrCond.end());
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (RevBranch) {
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(Cond))
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
return false;
}
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(RevPred) ||
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
!TII->SubsumesPredicate(Cond, RevPred))
return false;
}
return true;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Analyze the structure of the sub-CFG starting from the specified block.
/// Record its successors and whether it looks like an if-conversion candidate.
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
void IfConverter::AnalyzeBlock(
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &MBB, std::vector<std::unique_ptr<IfcvtToken>> &Tokens) {
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
struct BBState {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBState(MachineBasicBlock &MBB) : MBB(&MBB), SuccsAnalyzed(false) {}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
MachineBasicBlock *MBB;
/// This flag is true if MBB's successors have been analyzed.
bool SuccsAnalyzed;
};
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Push MBB to the stack.
SmallVector<BBState, 16> BBStack(1, MBB);
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
while (!BBStack.empty()) {
BBState &State = BBStack.back();
MachineBasicBlock *BB = State.MBB;
BBInfo &BBI = BBAnalysis[BB->getNumber()];
Silly boog. llvm-svn: 37328
2007-05-25 08:59:01 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (!State.SuccsAnalyzed) {
if (BBI.IsAnalyzed || BBI.IsBeingAnalyzed) {
BBStack.pop_back();
continue;
}
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
BBI.BB = BB;
BBI.IsBeingAnalyzed = true;
isBlockPredicable() always ignore terminal instructions; add comments. llvm-svn: 37126
2007-05-17 05:54:37 +08:00
IfConverter: Split ScanInstructions into 2 functions. ScanInstructions is now 2 functions: AnalyzeBranches and ScanInstructions. ScanInstructions also now takes a pair of arguments delimiting the instructions to be scanned. This will be used for forked diamond support to re-scan only a portion of the block. llvm-svn: 277904
2016-08-06 09:52:34 +08:00
AnalyzeBranches(BBI);
MachineBasicBlock::iterator Begin = BBI.BB->begin();
MachineBasicBlock::iterator End = BBI.BB->end();
ScanInstructions(BBI, Begin, End);
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Unanalyzable or ends with fallthrough or unconditional branch, or if is
// not considered for ifcvt anymore.
if (!BBI.IsBrAnalyzable || BBI.BrCond.empty() || BBI.IsDone) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
ifcvt should ignore cfg where true and false successors are the same. llvm-svn: 73423
2009-06-16 05:24:34 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Do not ifcvt if either path is a back edge to the entry block.
if (BBI.TrueBB == BB || BBI.FalseBB == BB) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Do not ifcvt if true and false fallthrough blocks are the same.
if (!BBI.FalseBB) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
// Push the False and True blocks to the stack.
State.SuccsAnalyzed = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBStack.push_back(*BBI.FalseBB);
BBStack.push_back(*BBI.TrueBB);
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
continue;
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (TrueBBI.IsDone && FalseBBI.IsDone) {
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
continue;
}
Use MachineBranchProbabilityInfo in If-Conversion instead of its own heuristics. llvm-svn: 136826
2011-08-04 06:34:43 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
SmallVector<MachineOperand, 4>
RevCond(BBI.BrCond.begin(), BBI.BrCond.end());
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
bool CanRevCond = !TII->reverseBranchCondition(RevCond);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
unsigned Dups = 0;
unsigned Dups2 = 0;
bool TNeedSub = !TrueBBI.Predicate.empty();
bool FNeedSub = !FalseBBI.Predicate.empty();
bool Enqueued = false;
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
BranchProbability Prediction = MBPI->getEdgeProbability(BB, TrueBBI.BB);
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (CanRevCond) {
BBInfo TrueBBICalc, FalseBBICalc;
[IfCvt][ARM] Optimise diamond if-conversion for code size Currently, the heuristics the if-conversion pass uses for diamond if-conversion are based on execution time, with no consideration for code size. This adds a new set of heuristics to be used when optimising for code size. This is mostly target-independent, because the if-conversion pass can see the code size of the instructions which it is removing. For thumb, there are a few passes (insertion of IT instructions, selection of narrow branches, and selection of CBZ instructions) which are run after if conversion and affect these heuristics, so I've added target hooks to better predict the code-size effect of a proposed if-conversion. Differential revision: https://reviews.llvm.org/D67350 llvm-svn: 374301
2019-10-10 17:58:28 +08:00
auto feasibleDiamond = [&](bool Forked) {
bool MeetsSize = MeetIfcvtSizeLimit(TrueBBICalc, FalseBBICalc, *BB,
Dups + Dups2, Prediction, Forked);
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
bool TrueFeasible = FeasibilityAnalysis(TrueBBI, BBI.BrCond,
/* IsTriangle */ false, /* RevCond */ false,
/* hasCommonTail */ true);
bool FalseFeasible = FeasibilityAnalysis(FalseBBI, RevCond,
/* IsTriangle */ false, /* RevCond */ false,
/* hasCommonTail */ true);
return MeetsSize && TrueFeasible && FalseFeasible;
};
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (ValidDiamond(TrueBBI, FalseBBI, Dups, Dups2,
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
TrueBBICalc, FalseBBICalc)) {
[IfCvt][ARM] Optimise diamond if-conversion for code size Currently, the heuristics the if-conversion pass uses for diamond if-conversion are based on execution time, with no consideration for code size. This adds a new set of heuristics to be used when optimising for code size. This is mostly target-independent, because the if-conversion pass can see the code size of the instructions which it is removing. For thumb, there are a few passes (insertion of IT instructions, selection of narrow branches, and selection of CBZ instructions) which are run after if conversion and affect these heuristics, so I've added target hooks to better predict the code-size effect of a proposed if-conversion. Differential revision: https://reviews.llvm.org/D67350 llvm-svn: 374301
2019-10-10 17:58:28 +08:00
if (feasibleDiamond(false)) {
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
// Diamond:
// EBB
// / \_
// | |
// TBB FBB
// \ /
// TailBB
// Note TailBB can be empty.
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
Tokens.push_back(std::make_unique<IfcvtToken>(
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
BBI, ICDiamond, TNeedSub | FNeedSub, Dups, Dups2,
(bool) TrueBBICalc.ClobbersPred, (bool) FalseBBICalc.ClobbersPred));
Enqueued = true;
}
} else if (ValidForkedDiamond(TrueBBI, FalseBBI, Dups, Dups2,
TrueBBICalc, FalseBBICalc)) {
[IfCvt][ARM] Optimise diamond if-conversion for code size Currently, the heuristics the if-conversion pass uses for diamond if-conversion are based on execution time, with no consideration for code size. This adds a new set of heuristics to be used when optimising for code size. This is mostly target-independent, because the if-conversion pass can see the code size of the instructions which it is removing. For thumb, there are a few passes (insertion of IT instructions, selection of narrow branches, and selection of CBZ instructions) which are run after if conversion and affect these heuristics, so I've added target hooks to better predict the code-size effect of a proposed if-conversion. Differential revision: https://reviews.llvm.org/D67350 llvm-svn: 374301
2019-10-10 17:58:28 +08:00
if (feasibleDiamond(true)) {
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
// ForkedDiamond:
// if TBB and FBB have a common tail that includes their conditional
// branch instructions, then we can If Convert this pattern.
// EBB
// _/ \_
// | |
// TBB FBB
// / \ / \
// FalseBB TrueBB FalseBB
//
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
Tokens.push_back(std::make_unique<IfcvtToken>(
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
BBI, ICForkedDiamond, TNeedSub | FNeedSub, Dups, Dups2,
(bool) TrueBBICalc.ClobbersPred, (bool) FalseBBICalc.ClobbersPred));
Enqueued = true;
}
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
}
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (ValidTriangle(TrueBBI, FalseBBI, false, Dups, Prediction) &&
MeetIfcvtSizeLimit(*TrueBBI.BB, TrueBBI.NonPredSize + TrueBBI.ExtraCost,
TrueBBI.ExtraCost2, Prediction) &&
FeasibilityAnalysis(TrueBBI, BBI.BrCond, true)) {
// Triangle:
// EBB
// | \_
// | |
// | TBB
// | /
// FBB
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
std::make_unique<IfcvtToken>(BBI, ICTriangle, TNeedSub, Dups));
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Enqueued = true;
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (ValidTriangle(TrueBBI, FalseBBI, true, Dups, Prediction) &&
MeetIfcvtSizeLimit(*TrueBBI.BB, TrueBBI.NonPredSize + TrueBBI.ExtraCost,
TrueBBI.ExtraCost2, Prediction) &&
FeasibilityAnalysis(TrueBBI, BBI.BrCond, true, true)) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
std::make_unique<IfcvtToken>(BBI, ICTriangleRev, TNeedSub, Dups));
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Enqueued = true;
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (ValidSimple(TrueBBI, Dups, Prediction) &&
MeetIfcvtSizeLimit(*TrueBBI.BB, TrueBBI.NonPredSize + TrueBBI.ExtraCost,
TrueBBI.ExtraCost2, Prediction) &&
FeasibilityAnalysis(TrueBBI, BBI.BrCond)) {
// Simple (split, no rejoin):
// EBB
// | \_
// | |
// | TBB---> exit
// |
// FBB
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
std::make_unique<IfcvtToken>(BBI, ICSimple, TNeedSub, Dups));
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
Enqueued = true;
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
}
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
if (CanRevCond) {
// Try the other path...
if (ValidTriangle(FalseBBI, TrueBBI, false, Dups,
Prediction.getCompl()) &&
MeetIfcvtSizeLimit(*FalseBBI.BB,
FalseBBI.NonPredSize + FalseBBI.ExtraCost,
FalseBBI.ExtraCost2, Prediction.getCompl()) &&
FeasibilityAnalysis(FalseBBI, RevCond, true)) {
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
Tokens.push_back(std::make_unique<IfcvtToken>(BBI, ICTriangleFalse,
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
FNeedSub, Dups));
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
Enqueued = true;
}
if (ValidTriangle(FalseBBI, TrueBBI, true, Dups,
Prediction.getCompl()) &&
MeetIfcvtSizeLimit(*FalseBBI.BB,
FalseBBI.NonPredSize + FalseBBI.ExtraCost,
FalseBBI.ExtraCost2, Prediction.getCompl()) &&
FeasibilityAnalysis(FalseBBI, RevCond, true, true)) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
std::make_unique<IfcvtToken>(BBI, ICTriangleFRev, FNeedSub, Dups));
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
Enqueued = true;
}
if (ValidSimple(FalseBBI, Dups, Prediction.getCompl()) &&
MeetIfcvtSizeLimit(*FalseBBI.BB,
FalseBBI.NonPredSize + FalseBBI.ExtraCost,
FalseBBI.ExtraCost2, Prediction.getCompl()) &&
FeasibilityAnalysis(FalseBBI, RevCond)) {
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
Tokens.push_back(
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
std::make_unique<IfcvtToken>(BBI, ICSimpleFalse, FNeedSub, Dups));
[If Converter] Convert recursion to iteration. This commit makes changes to IfConverter::AnalyzeBlock to use iteration instead of recursion. Previously, this function would get called recursively a large number of times and eventually segfault when a function with the following CFG was compiled: BB0: if (condition0) goto BB1 goto BB2 BB1: goto BB2 BB2: if (condition1) goto BB3 goto BB4 BB3: ... (repeat until BB7488) rdar://problem/21386145 Differential Revision: http://reviews.llvm.org/D10587 llvm-svn: 240589
2015-06-25 04:34:35 +08:00
Enqueued = true;
}
}
BBI.IsEnqueued = Enqueued;
BBI.IsBeingAnalyzed = false;
BBI.IsAnalyzed = true;
BBStack.pop_back();
}
Some restructuring in preparation for most aggressive if-conversion. llvm-svn: 37231
2007-05-19 02:14:37 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Analyze all blocks and find entries for all if-conversion candidates.
[ifcnv] Use unique_ptr in IfConversion. NFC Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D17466 llvm-svn: 261541
2016-02-23 01:51:28 +08:00
void IfConverter::AnalyzeBlocks(
MachineFunction &MF, std::vector<std::unique_ptr<IfcvtToken>> &Tokens) {
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock &MBB : MF)
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
AnalyzeBlock(MBB, Tokens);
Change traversal order to bottom up in preparation for more aggressive if-conversion. llvm-svn: 37365
2007-05-31 03:49:19 +08:00
Allow multiple ifcvt candidates to share children blocks; add some debugging code. llvm-svn: 37379
2007-06-01 08:12:12 +08:00
// Sort to favor more complex ifcvt scheme.
Use llvm::stable_sort While touching the code, simplify if feasible. llvm-svn: 358996
2019-04-23 22:51:27 +08:00
llvm::stable_sort(Tokens, IfcvtTokenCmp);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Returns true either if ToMBB is the next block after MBB or that all the
/// intervening blocks are empty (given MBB can fall through to its next block).
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
static bool canFallThroughTo(MachineBasicBlock &MBB, MachineBasicBlock &ToMBB) {
MachineFunction::iterator PI = MBB.getIterator();
[C++11] Replace llvm::next and llvm::prior with std::next and std::prev. Remove the old functions. llvm-svn: 202636
2014-03-02 20:27:27 +08:00
MachineFunction::iterator I = std::next(PI);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineFunction::iterator TI = ToMBB.getIterator();
MachineFunction::iterator E = MBB.getParent()->end();
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
while (I != TI) {
// Check isSuccessor to avoid case where the next block is empty, but
// it's not a successor.
CodeGen: Remove more ilist iterator implicit conversions, NFC llvm-svn: 249879
2015-10-10 03:13:58 +08:00
if (I == E || !I->empty() || !PI->isSuccessor(&*I))
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
return false;
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
PI = I++;
}
[IfConversion] Add missing check in IfConversion/canFallThroughTo Summary: When trying to figure out if MBB could fallthrough to ToMBB (possibly by falling through a bunch of other MBBs) we didn't actually check if there was fallthrough between the last two blocks in the chain. Reviewers: kparzysz, iteratee, MatzeB Reviewed By: kparzysz, iteratee Subscribers: javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D32996 llvm-svn: 302650
2017-05-10 21:06:13 +08:00
// Finally see if the last I is indeed a successor to PI.
return PI->isSuccessor(&*I);
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Invalidate predecessor BB info so it would be re-analyzed to determine if it
/// can be if-converted. If predecessor is already enqueued, dequeue it!
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
void IfConverter::InvalidatePreds(MachineBasicBlock &MBB) {
for (const MachineBasicBlock *Predecessor : MBB.predecessors()) {
CodeGen: switch to a range based for loop Use a range based for loop instead of manual iteration. NFC. llvm-svn: 215287
2014-08-10 01:21:29 +08:00
BBInfo &PBBI = BBAnalysis[Predecessor->getNumber()];
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (PBBI.IsDone || PBBI.BB == &MBB)
If BB is predicated, invalidate its predecessor(s) which would if-convert it. It needs to be re-analyzed. llvm-svn: 37580
2007-06-15 07:13:19 +08:00
continue;
No really, clear predcessors states. llvm-svn: 37581
2007-06-15 07:34:09 +08:00
PBBI.IsAnalyzed = false;
PBBI.IsEnqueued = false;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
}
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Inserts an unconditional branch from \p MBB to \p ToMBB.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
static void InsertUncondBranch(MachineBasicBlock &MBB, MachineBasicBlock &ToMBB,
If there is an empty block between a source and its successor block, it still requires a unconditional branch. llvm-svn: 37344
2007-05-30 06:31:16 +08:00
const TargetInstrInfo *TII) {
Add a DebugLoc parameter to TargetInstrInfo::InsertBranch(). This addresses a longstanding deficiency noted in many FIXMEs scattered across all the targets. This effectively moves the problem up one level, replacing eleven FIXMEs in the targets with eight FIXMEs in CodeGen, plus one path through FastISel where we actually supply a DebugLoc, fixing Radar 7421831. llvm-svn: 106243
2010-06-18 06:43:56 +08:00
DebugLoc dl; // FIXME: this is nowhere
Fix SmallVector's size calculation so that a size of 0 is handled correctly, and change a few SmallVector uses to use size 0 to more clearly reflect their intent. llvm-svn: 55181
2008-08-23 00:07:55 +08:00
SmallVector<MachineOperand, 0> NoCond;
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(MBB, &ToMBB, nullptr, NoCond, dl);
If there is an empty block between a source and its successor block, it still requires a unconditional branch. llvm-svn: 37344
2007-05-30 06:31:16 +08:00
}
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
/// Behaves like LiveRegUnits::StepForward() but also adds implicit uses to all
[IfConversion] Bugfix: Don't use undef flag while adding use operands. IfConversion used to always add the undef flag when adding a use operand on a newly predicated instruction. This would be an operand for the register being conditionally redefined. Due to the undef flag, the liveness of this register prior to the predicated instruction would get lost. This patch changes this so that such use operands are added only when the register is live, without the undef flag. This was reverted but pushed again now, for details follow link below. Reviewed by Quentin Colombet. http://reviews.llvm.org/D209077 llvm-svn: 277571
2016-08-03 13:46:35 +08:00
/// values defined in MI which are also live/used by MI.
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
static void UpdatePredRedefs(MachineInstr &MI, LivePhysRegs &Redefs) {
CodeGen: Minor cleanups to use MachineInstr::getMF. NFC Since r315388 we have a shorter way to say this, so we'll replace MI->getParent()->getParent() with MI->getMF() in a few places. llvm-svn: 315390
2017-10-11 07:50:49 +08:00
const TargetRegisterInfo *TRI = MI.getMF()->getSubtarget().getRegisterInfo();
[IfConversion] Bugfix: Don't use undef flag while adding use operands. IfConversion used to always add the undef flag when adding a use operand on a newly predicated instruction. This would be an operand for the register being conditionally redefined. Due to the undef flag, the liveness of this register prior to the predicated instruction would get lost. This patch changes this so that such use operands are added only when the register is live, without the undef flag. This was reverted but pushed again now, for details follow link below. Reviewed by Quentin Colombet. http://reviews.llvm.org/D209077 llvm-svn: 277571
2016-08-03 13:46:35 +08:00
// Before stepping forward past MI, remember which regs were live
// before MI. This is needed to set the Undef flag only when reg is
// dead.
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SparseSet<MCPhysReg, identity<MCPhysReg>> LiveBeforeMI;
[IfConversion] Bugfix: Don't use undef flag while adding use operands. IfConversion used to always add the undef flag when adding a use operand on a newly predicated instruction. This would be an operand for the register being conditionally redefined. Due to the undef flag, the liveness of this register prior to the predicated instruction would get lost. This patch changes this so that such use operands are added only when the register is live, without the undef flag. This was reverted but pushed again now, for details follow link below. Reviewed by Quentin Colombet. http://reviews.llvm.org/D209077 llvm-svn: 277571
2016-08-03 13:46:35 +08:00
LiveBeforeMI.setUniverse(TRI->getNumRegs());
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (unsigned Reg : Redefs)
[IfConversion] Bugfix: Don't use undef flag while adding use operands. IfConversion used to always add the undef flag when adding a use operand on a newly predicated instruction. This would be an operand for the register being conditionally redefined. Due to the undef flag, the liveness of this register prior to the predicated instruction would get lost. This patch changes this so that such use operands are added only when the register is live, without the undef flag. This was reverted but pushed again now, for details follow link below. Reviewed by Quentin Colombet. http://reviews.llvm.org/D209077 llvm-svn: 277571
2016-08-03 13:46:35 +08:00
LiveBeforeMI.insert(Reg);
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SmallVector<std::pair<MCPhysReg, const MachineOperand*>, 4> Clobbers;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
Redefs.stepForward(MI, Clobbers);
Refactor UpdatePredRedefs and StepForward to avoid duplication. NFC Note, this is a reapplication of r236515 with a fix to not assert on non-register operands, but instead only handle them until the subsequent commit. Original commit message follows. The code was basically the same here already. Just added an out parameter for a vector of seen defs so that UpdatePredRedefs can call StepForward first, then do its own post processing on the seen defs. Will be used in the next commit to also handle regmasks. llvm-svn: 236538
2015-05-06 04:14:22 +08:00
// Now add the implicit uses for each of the clobbered values.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
for (auto Clobber : Clobbers) {
Refactor UpdatePredRedefs and StepForward to avoid duplication. NFC Note, this is a reapplication of r236515 with a fix to not assert on non-register operands, but instead only handle them until the subsequent commit. Original commit message follows. The code was basically the same here already. Just added an out parameter for a vector of seen defs so that UpdatePredRedefs can call StepForward first, then do its own post processing on the seen defs. Will be used in the next commit to also handle regmasks. llvm-svn: 236538
2015-05-06 04:14:22 +08:00
// FIXME: Const cast here is nasty, but better than making StepForward
// take a mutable instruction instead of const.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
unsigned Reg = Clobber.first;
MachineOperand &Op = const_cast<MachineOperand&>(*Clobber.second);
Handle dead defs in the if converter. We had code such as this: r2 = ... t2Bcc label1: ldr ... r2 label2; return r2<dead, def> The if converter was transforming this to r2<def> = ... return [pred] r2<dead,def> ldr <r2, kill> return which fails the machine verifier because the ldr now reads from a dead def. The fix here detects dead defs in stepForward and passes them back to the caller in the clobbers list. The caller then clears the dead flag from the def is the value is live. llvm-svn: 236660
2015-05-07 06:51:04 +08:00
MachineInstr *OpMI = Op.getParent();
CodeGen: Minor cleanups to use MachineInstr::getMF. NFC Since r315388 we have a shorter way to say this, so we'll replace MI->getParent()->getParent() with MI->getMF() in a few places. llvm-svn: 315390
2017-10-11 07:50:49 +08:00
MachineInstrBuilder MIB(*OpMI->getMF(), OpMI);
Fix IfConverter to handle regmask machine operands. Note, this is a recommit of r236515 after fixing an error in r236514. The buildbot ran fast enough that it picked up r236514 prior to r236515 and threw an error. r236515 itself ran 'make check' without errors. Original commit message follows: A regmask (typically seen on a call) clobbers the set of registers it lists. The IfConverter, in UpdatePredRedefs, was handling register defs, but not regmasks. These are slightly different to a def in that we need to add both an implicit use and def to appease the machine verifier. Otherwise, uses after the if converted call could think they are reading an undefined register. Reviewed by Matthias Braun and Quentin Colombet. llvm-svn: 236550
2015-05-06 06:09:41 +08:00
if (Op.isRegMask()) {
// First handle regmasks. They clobber any entries in the mask which
// means that we need a def for those registers.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (LiveBeforeMI.count(Reg))
MIB.addReg(Reg, RegState::Implicit);
Fix IfConverter to handle regmask machine operands. Note, this is a recommit of r236515 after fixing an error in r236514. The buildbot ran fast enough that it picked up r236514 prior to r236515 and threw an error. r236515 itself ran 'make check' without errors. Original commit message follows: A regmask (typically seen on a call) clobbers the set of registers it lists. The IfConverter, in UpdatePredRedefs, was handling register defs, but not regmasks. These are slightly different to a def in that we need to add both an implicit use and def to appease the machine verifier. Otherwise, uses after the if converted call could think they are reading an undefined register. Reviewed by Matthias Braun and Quentin Colombet. llvm-svn: 236550
2015-05-06 06:09:41 +08:00
// We also need to add an implicit def of this register for the later
// use to read from.
// For the register allocator to have allocated a register clobbered
// by the call which is used later, it must be the case that
// the call doesn't return.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MIB.addReg(Reg, RegState::Implicit | RegState::Define);
Fix IfConverter to handle regmask machine operands. Note, this is a recommit of r236515 after fixing an error in r236514. The buildbot ran fast enough that it picked up r236514 prior to r236515 and threw an error. r236515 itself ran 'make check' without errors. Original commit message follows: A regmask (typically seen on a call) clobbers the set of registers it lists. The IfConverter, in UpdatePredRedefs, was handling register defs, but not regmasks. These are slightly different to a def in that we need to add both an implicit use and def to appease the machine verifier. Otherwise, uses after the if converted call could think they are reading an undefined register. Reviewed by Matthias Braun and Quentin Colombet. llvm-svn: 236550
2015-05-06 06:09:41 +08:00
continue;
}
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (LiveBeforeMI.count(Reg))
MIB.addReg(Reg, RegState::Implicit);
IfConversion: Add implicit uses for redefined regs with live subregisters Normally, if conversion would add implicit uses for redefined registers, e.g. R0<def> = add_if ..., R0<imp-use>. However, if only subregisters of R0 are known to be live but not R0 itself, such implicit uses will not be added, causing prior definitions of such subregisters and R0 itself to become dead. llvm-svn: 282626
2016-09-29 04:07:41 +08:00
else {
bool HasLiveSubReg = false;
for (MCSubRegIterator S(Reg, TRI); S.isValid(); ++S) {
if (!LiveBeforeMI.count(*S))
continue;
HasLiveSubReg = true;
break;
}
if (HasLiveSubReg)
MIB.addReg(Reg, RegState::Implicit);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
}
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// If convert a simple (split, no rejoin) sub-CFG.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool IfConverter::IfConvertSimple(BBInfo &BBI, IfcvtKind Kind) {
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
BBInfo *CvtBBI = &TrueBBI;
BBInfo *NextBBI = &FalseBBI;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> Cond(BBI.BrCond.begin(), BBI.BrCond.end());
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICSimpleFalse)
Correctly mark early-exit on the false path. llvm-svn: 37387
2007-06-02 04:29:21 +08:00
std::swap(CvtBBI, NextBBI);
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &CvtMBB = *CvtBBI->BB;
MachineBasicBlock &NextMBB = *NextBBI->BB;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (CvtBBI->IsDone ||
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
(CvtBBI->CannotBeCopied && CvtMBB.pred_size() > 1)) {
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
// Something has changed. It's no longer safe to predicate this block.
BBI.IsAnalyzed = false;
CvtBBI->IsAnalyzed = false;
return false;
Correctly mark early-exit on the false path. llvm-svn: 37387
2007-06-02 04:29:21 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.hasAddressTaken())
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
// Conservatively abort if-conversion if BB's address is taken.
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICSimpleFalse)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(Cond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
LivePhysReg: Use reference instead of pointer in init(); NFC llvm-svn: 289002
2016-12-08 08:15:51 +08:00
Redefs.init(*TRI);
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
[NFC] fix trivial typos in comments llvm-svn: 350690
2019-01-09 13:11:10 +08:00
// Initialize liveins to the first BB. These are potentially redefined by
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
// predicated instructions.
Redefs.addLiveIns(CvtMBB);
Redefs.addLiveIns(NextMBB);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
[IfConversion] Hoist removeBranch calls out of if/else clauses [NFC] Summary: Also added a comment. Pulled out of https://reviews.llvm.org/D34099. Reviewers: iteratee Reviewed By: iteratee Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34388 llvm-svn: 306279
2017-06-26 17:33:04 +08:00
// Remove the branches from the entry so we can add the contents of the true
// block to it.
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.pred_size() > 1) {
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Copy instructions in the true block, predicate them, and add them to
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
// the entry block.
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
CopyAndPredicateBlock(BBI, *CvtBBI, Cond);
Manually remove successors in if conversion when CopyAndPredicateBlock is used In the simple and triangle if-conversion cases, when CopyAndPredicateBlock is used because the to-be-predicated block has other predecessors, we need to explicitly remove the old copied block from the successors list. Normally if conversion relies on TII->AnalyzeBranch combined with BB->CorrectExtraCFGEdges to cleanup the successors list, but if the predicated block contained an un-analyzable branch (such as a now-predicated return), then this will fail. These extra successors were causing a problem on PPC because it was causing later passes (such as PPCEarlyReturm) to leave dead return-only basic blocks in the code. llvm-svn: 179227
2013-04-11 06:05:25 +08:00
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
// Keep the CFG updated.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBI.BB->removeSuccessor(&CvtMBB, true);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
} else {
[IfConversion] Hoist removeBranch calls out of if/else clauses [NFC] Summary: Also added a comment. Pulled out of https://reviews.llvm.org/D34099. Reviewers: iteratee Reviewed By: iteratee Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34388 llvm-svn: 306279
2017-06-26 17:33:04 +08:00
// Predicate the instructions in the true block.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
PredicateBlock(*CvtBBI, CvtMBB.end(), Cond);
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
// Merge converted block into entry block. The BB to Cvt edge is removed
// by MergeBlocks.
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
MergeBlocks(BBI, *CvtBBI);
}
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
bool IterIfcvt = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (!canFallThroughTo(*BBI.BB, NextMBB)) {
InsertUncondBranch(*BBI.BB, NextMBB, TII);
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
BBI.HasFallThrough = false;
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
// Now ifcvt'd block will look like this:
// BB:
// ...
// t, f = cmp
// if t op
// b BBf
//
// We cannot further ifcvt this block because the unconditional branch
// will have to be predicated on the new condition, that will not be
// available if cmp executes.
IterIfcvt = false;
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
// Update block info. BB can be iteratively if-converted.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
if (!IterIfcvt)
BBI.IsDone = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InvalidatePreds(*BBI.BB);
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
CvtBBI->IsDone = true;
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
// FIXME: Must maintain LiveIns.
return true;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// If convert a triangle sub-CFG.
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
bool IfConverter::IfConvertTriangle(BBInfo &BBI, IfcvtKind Kind) {
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
BBInfo *CvtBBI = &TrueBBI;
BBInfo *NextBBI = &FalseBBI;
Add a DebugLoc parameter to TargetInstrInfo::InsertBranch(). This addresses a longstanding deficiency noted in many FIXMEs scattered across all the targets. This effectively moves the problem up one level, replacing eleven FIXMEs in the targets with eight FIXMEs in CodeGen, plus one path through FastISel where we actually supply a DebugLoc, fixing Radar 7421831. llvm-svn: 106243
2010-06-18 06:43:56 +08:00
DebugLoc dl; // FIXME: this is nowhere
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> Cond(BBI.BrCond.begin(), BBI.BrCond.end());
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICTriangleFalse || Kind == ICTriangleFRev)
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
std::swap(CvtBBI, NextBBI);
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &CvtMBB = *CvtBBI->BB;
MachineBasicBlock &NextMBB = *NextBBI->BB;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (CvtBBI->IsDone ||
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
(CvtBBI->CannotBeCopied && CvtMBB.pred_size() > 1)) {
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
// Something has changed. It's no longer safe to predicate this block.
BBI.IsAnalyzed = false;
CvtBBI->IsAnalyzed = false;
return false;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
}
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.hasAddressTaken())
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
// Conservatively abort if-conversion if BB's address is taken.
return false;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICTriangleFalse || Kind == ICTriangleFRev)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(Cond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Not every predicable block can be safely duplicated. llvm-svn: 37607
2007-06-16 05:18:05 +08:00
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (Kind == ICTriangleRev || Kind == ICTriangleFRev) {
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (reverseBranchCondition(*CvtBBI)) {
Optimized FCMP_OEQ and FCMP_UNE for x86. Where previously LLVM might emit code like this: ucomisd %xmm1, %xmm0 setne %al setp %cl orb %al, %cl jne .LBB4_2 it now emits this: ucomisd %xmm1, %xmm0 jne .LBB4_2 jp .LBB4_2 It has fewer instructions and uses fewer registers, but it does have more branches. And in the case that this code is followed by a non-fallthrough edge, it may be followed by a jmp instruction, resulting in three branch instructions in sequence. Some effort is made to avoid this situation. To achieve this, X86ISelLowering.cpp now recognizes FCMP_OEQ and FCMP_UNE in lowered form, and replace them with code that emits two branches, except in the case where it would require converting a fall-through edge to an explicit branch. Also, X86InstrInfo.cpp's branch analysis and transform code now knows now to handle blocks with multiple conditional branches. It uses loops instead of having fixed checks for up to two instructions. It can now analyze and transform code generated from FCMP_OEQ and FCMP_UNE. llvm-svn: 57873
2008-10-21 11:29:32 +08:00
// BB has been changed, modify its predecessors (except for this
// one) so they don't get ifcvt'ed based on bad intel.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
for (MachineBasicBlock *PBB : CvtMBB.predecessors()) {
Optimized FCMP_OEQ and FCMP_UNE for x86. Where previously LLVM might emit code like this: ucomisd %xmm1, %xmm0 setne %al setp %cl orb %al, %cl jne .LBB4_2 it now emits this: ucomisd %xmm1, %xmm0 jne .LBB4_2 jp .LBB4_2 It has fewer instructions and uses fewer registers, but it does have more branches. And in the case that this code is followed by a non-fallthrough edge, it may be followed by a jmp instruction, resulting in three branch instructions in sequence. Some effort is made to avoid this situation. To achieve this, X86ISelLowering.cpp now recognizes FCMP_OEQ and FCMP_UNE in lowered form, and replace them with code that emits two branches, except in the case where it would require converting a fall-through edge to an explicit branch. Also, X86InstrInfo.cpp's branch analysis and transform code now knows now to handle blocks with multiple conditional branches. It uses loops instead of having fixed checks for up to two instructions. It can now analyze and transform code generated from FCMP_OEQ and FCMP_UNE. llvm-svn: 57873
2008-10-21 11:29:32 +08:00
if (PBB == BBI.BB)
continue;
BBInfo &PBBI = BBAnalysis[PBB->getNumber()];
if (PBBI.IsEnqueued) {
PBBI.IsAnalyzed = false;
PBBI.IsEnqueued = false;
}
No really, clear predcessors states. llvm-svn: 37581
2007-06-15 07:34:09 +08:00
}
Now if-converting all 4 variants of triangles. llvm-svn: 37565
2007-06-13 07:54:05 +08:00
}
}
Ifcvt triangle: don't ifcvt 'true' BB if it has other predecessors; don't merge 'false' BB if it has other predecessors. llvm-svn: 37382
2007-06-01 15:41:07 +08:00
Tidy. llvm-svn: 106383
2010-06-19 13:33:57 +08:00
// Initialize liveins to the first BB. These are potentially redefined by
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
// predicated instructions.
LivePhysReg: Use reference instead of pointer in init(); NFC llvm-svn: 289002
2016-12-08 08:15:51 +08:00
Redefs.init(*TRI);
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
Redefs.addLiveIns(CvtMBB);
Redefs.addLiveIns(NextMBB);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
bool HasEarlyExit = CvtBBI->FalseBB != nullptr;
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
BranchProbability CvtNext, CvtFalse, BBNext, BBCvt;
CodeGen: switch to a range based for loop Use a range based for loop instead of manual iteration. NFC. llvm-svn: 215287
2014-08-10 01:21:29 +08:00
PGO branch weight: update edge weights in IfConverter. This commit only handles IfConvertTriangle. To update edge weights of a successor, one interface is added to MachineBasicBlock: /// Set successor weight of a given iterator. setSuccWeight(succ_iterator I, uint32_t weight) An existing testing case test/CodeGen/Thumb2/v8_IT_5.ll is updated, since we now correctly update the edge weights, the cold block is placed at the end of the function and we jump to the cold block. llvm-svn: 200428
2014-01-30 07:18:47 +08:00
if (HasEarlyExit) {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// Get probabilities before modifying CvtMBB and BBI.BB.
CvtNext = MBPI->getEdgeProbability(&CvtMBB, &NextMBB);
CvtFalse = MBPI->getEdgeProbability(&CvtMBB, CvtBBI->FalseBB);
BBNext = MBPI->getEdgeProbability(BBI.BB, &NextMBB);
BBCvt = MBPI->getEdgeProbability(BBI.BB, &CvtMBB);
PGO branch weight: update edge weights in IfConverter. This commit only handles IfConvertTriangle. To update edge weights of a successor, one interface is added to MachineBasicBlock: /// Set successor weight of a given iterator. setSuccWeight(succ_iterator I, uint32_t weight) An existing testing case test/CodeGen/Thumb2/v8_IT_5.ll is updated, since we now correctly update the edge weights, the cold block is placed at the end of the function and we jump to the cold block. llvm-svn: 200428
2014-01-30 07:18:47 +08:00
}
CodeGen: switch to a range based for loop Use a range based for loop instead of manual iteration. NFC. llvm-svn: 215287
2014-08-10 01:21:29 +08:00
[IfConversion] Hoist removeBranch calls out of if/else clauses [NFC] Summary: Also added a comment. Pulled out of https://reviews.llvm.org/D34099. Reviewers: iteratee Reviewed By: iteratee Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D34388 llvm-svn: 306279
2017-06-26 17:33:04 +08:00
// Remove the branches from the entry so we can add the contents of the true
// block to it.
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (CvtMBB.pred_size() > 1) {
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// Copy instructions in the true block, predicate them, and add them to
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
// the entry block.
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
CopyAndPredicateBlock(BBI, *CvtBBI, Cond, true);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
} else {
// Predicate the 'true' block after removing its branch.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
CvtBBI->NonPredSize -= TII->removeBranch(CvtMBB);
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
PredicateBlock(*CvtBBI, CvtMBB.end(), Cond);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
[IfConversion] Keep the CFG updated incrementally in IfConvertTriangle Summary: Instead of using RemoveExtraEdges (which uses analyzeBranch, which cannot always be trusted) at the end to fixup the CFG we keep the CFG updated as we go along and remove or add branches and merge blocks. This way we won't have any problems if the involved MBBs contain unanalyzable instructions. This fixes PR32721. In that case we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). Reviewers: kparzysz, iteratee, MatzeB Reviewed By: iteratee Subscribers: aemerson, rengolin, javed.absar, llvm-commits Differential Revision: https://reviews.llvm.org/D33037 llvm-svn: 302876
2017-05-12 14:28:58 +08:00
// Now merge the entry of the triangle with the true block.
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
MergeBlocks(BBI, *CvtBBI, false);
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
}
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
// Keep the CFG updated.
BBI.BB->removeSuccessor(&CvtMBB, true);
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
// If 'true' block has a 'false' successor, add an exit branch to it.
If the predicated block requires an early exit, end the block there and add a unconditional branch to false block. AnalyzeBranch() does not understand early exits. llvm-svn: 37430
2007-06-05 09:31:40 +08:00
if (HasEarlyExit) {
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> RevCond(CvtBBI->BrCond.begin(),
CvtBBI->BrCond.end());
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(RevCond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// Update the edge probability for both CvtBBI->FalseBB and NextBBI.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// NewNext = New_Prob(BBI.BB, NextMBB) =
// Prob(BBI.BB, NextMBB) +
// Prob(BBI.BB, CvtMBB) * Prob(CvtMBB, NextMBB)
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// NewFalse = New_Prob(BBI.BB, CvtBBI->FalseBB) =
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// Prob(BBI.BB, CvtMBB) * Prob(CvtMBB, CvtBBI->FalseBB)
auto NewTrueBB = getNextBlock(*BBI.BB);
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto NewNext = BBNext + BBCvt * CvtNext;
Use the range variant of find/find_if instead of unpacking begin/end If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278469
2016-08-12 11:55:06 +08:00
auto NewTrueBBIter = find(BBI.BB->successors(), NewTrueBB);
Fix a bug in IfConversion.cpp. The bug is introduced in r254377 which failed some tests on ARM, where a new probability is assigned to a successor but the provided BB may not be a successor. llvm-svn: 254463
2015-12-02 05:50:20 +08:00
if (NewTrueBBIter != BBI.BB->succ_end())
BBI.BB->setSuccProbability(NewTrueBBIter, NewNext);
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto NewFalse = BBCvt * CvtFalse;
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(*BBI.BB, CvtBBI->FalseBB, nullptr, RevCond, dl);
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
BBI.BB->addSuccessor(CvtBBI->FalseBB, NewFalse);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
// Merge in the 'false' block if the 'false' block has no other
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// predecessors. Otherwise, add an unconditional branch to 'false'.
Fix some subtle bugs: bug during succeessor copying; incorrectly updating states of ifcvted blocks. llvm-svn: 37429
2007-06-05 08:07:37 +08:00
bool FalseBBDead = false;
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
bool IterIfcvt = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
bool isFallThrough = canFallThroughTo(*BBI.BB, NextMBB);
ifcvt a triangle: don't merge ifcvt block with rejoin block if it can fall through to it. If merged, the resulting block is not a candidate for iterative ifcvting since it contains both predicated and non-predicated code. llvm-svn: 37487
2007-06-07 16:13:00 +08:00
if (!isFallThrough) {
// Only merge them if the true block does not fallthrough to the false
// block. By not merging them, we make it possible to iteratively
// ifcvt the blocks.
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (!HasEarlyExit &&
Revert "[IfConversion] Keep the CFG updated incrementally in IfConvertTriangle" The reverted change introdued assertions ala: "MachineBasicBlock::succ_iterator llvm::MachineBasicBlock::removeSuccessor(succ_iterator, bool): Assertion `I != Successors.end() && "Not a current successor!"' Mikael, the original committer, wrote me that he is working on a fix, but that it likely will take some time to get this resolved. As this bug is one of the last two issues that keep the AOSP buildbot from turning green, I revert the original commit r302876. I am looking forward to see this recommitted after the assertion has been resolved. llvm-svn: 304128
2017-05-29 14:12:18 +08:00
NextMBB.pred_size() == 1 && !NextBBI->HasFallThrough &&
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
!NextMBB.hasAddressTaken()) {
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
MergeBlocks(BBI, *NextBBI);
ifcvt a triangle: don't merge ifcvt block with rejoin block if it can fall through to it. If merged, the resulting block is not a candidate for iterative ifcvting since it contains both predicated and non-predicated code. llvm-svn: 37487
2007-06-07 16:13:00 +08:00
FalseBBDead = true;
} else {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InsertUncondBranch(*BBI.BB, NextMBB, TII);
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
BBI.HasFallThrough = false;
ifcvt a triangle: don't merge ifcvt block with rejoin block if it can fall through to it. If merged, the resulting block is not a candidate for iterative ifcvting since it contains both predicated and non-predicated code. llvm-svn: 37487
2007-06-07 16:13:00 +08:00
}
Allow more cmp / bcc to be predicated; clean up triangle ifcvt checking code. llvm-svn: 37518
2007-06-08 17:36:04 +08:00
// Mixed predicated and unpredicated code. This cannot be iteratively
// predicated.
IterIfcvt = false;
If a unconditional branch is added to branch to the false path during ifcvt, the predicated block cannot be iteratively ifcvted. llvm-svn: 37456
2007-06-06 10:08:52 +08:00
}
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
// Update block info. BB can be iteratively if-converted.
Fix 80col violations, remove trailing whitespace, and clarify a comment. llvm-svn: 106057
2010-06-16 06:18:54 +08:00
if (!IterIfcvt)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsDone = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InvalidatePreds(*BBI.BB);
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
CvtBBI->IsDone = true;
Fix some subtle bugs: bug during succeessor copying; incorrectly updating states of ifcvted blocks. llvm-svn: 37429
2007-06-05 08:07:37 +08:00
if (FalseBBDead)
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
NextBBI->IsDone = true;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
// FIXME: Must maintain LiveIns.
return true;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/// Common code shared between diamond conversions.
/// \p BBI, \p TrueBBI, and \p FalseBBI form the diamond shape.
/// \p NumDups1 - number of shared instructions at the beginning of \p TrueBBI
/// and FalseBBI
/// \p NumDups2 - number of shared instructions at the end of \p TrueBBI
/// and \p FalseBBI
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
/// \p RemoveBranch - Remove the common branch of the two blocks before
/// predicating. Only false for unanalyzable fallthrough
/// cases. The caller will replace the branch if necessary.
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/// \p MergeAddEdges - Add successor edges when merging blocks. Only false for
/// unanalyzable fallthrough
bool IfConverter::IfConvertDiamondCommon(
BBInfo &BBI, BBInfo &TrueBBI, BBInfo &FalseBBI,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
bool RemoveBranch, bool MergeAddEdges) {
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
if (TrueBBI.IsDone || FalseBBI.IsDone ||
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
TrueBBI.BB->pred_size() > 1 || FalseBBI.BB->pred_size() > 1) {
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// Something has changed. It's no longer safe to predicate these blocks.
BBI.IsAnalyzed = false;
TrueBBI.IsAnalyzed = false;
FalseBBI.IsAnalyzed = false;
return false;
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
}
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
if (TrueBBI.BB->hasAddressTaken() || FalseBBI.BB->hasAddressTaken())
// Conservatively abort if-conversion if either BB has its address taken.
return false;
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
// Put the predicated instructions from the 'true' block before the
// instructions from the 'false' block, unless the true block would clobber
// the predicate, in which case, do the opposite.
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
BBInfo *BBI1 = &TrueBBI;
BBInfo *BBI2 = &FalseBBI;
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> RevCond(BBI.BrCond.begin(), BBI.BrCond.end());
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
if (TII->reverseBranchCondition(RevCond))
Convert assert(0) to llvm_unreachable llvm-svn: 149816
2012-02-05 16:31:47 +08:00
llvm_unreachable("Unable to reverse branch condition!");
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVector<MachineOperand, 4> *Cond1 = &BBI.BrCond;
SmallVector<MachineOperand, 4> *Cond2 = &RevCond;
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
// Figure out the more profitable ordering.
bool DoSwap = false;
IfConversion: Rescan diamonds. The cost of predicating a diamond is only the instructions that are not shared between the two branches. Additionally If a predicate clobbering instruction occurs in the shared portion of the branches (e.g. a cond move), it may still be possible to if convert the sub-cfg. This change handles these two facts by rescanning the non-shared portion of a diamond sub-cfg to recalculate both the predication cost and whether both blocks are pred-clobbering. Fixed 2 bugs before recommitting. Branch instructions must be compared and found identical before diamond conversion. Also, predicate-clobbering instructions in the shared prefix disqualifies a potential diamond conversion. Includes tests for both. llvm-svn: 279670
2016-08-25 05:34:24 +08:00
if (TClobbersPred && !FClobbersPred)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
DoSwap = true;
IfConversion: Add assertions that both sides of a diamond don't pred-clobber. One side of a diamond may end with a predicate clobbering instruction. That side of the diamond has to be if-converted second. Both sides can't clobber the predicate or the ifconversion is invalid. This is checked elsewhere, but add an assert as a safety check. NFC llvm-svn: 280518
2016-09-03 02:29:28 +08:00
else if (!TClobbersPred && !FClobbersPred) {
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
if (TrueBBI.NonPredSize > FalseBBI.NonPredSize)
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
DoSwap = true;
IfConversion: Add assertions that both sides of a diamond don't pred-clobber. One side of a diamond may end with a predicate clobbering instruction. That side of the diamond has to be if-converted second. Both sides can't clobber the predicate or the ifconversion is invalid. This is checked elsewhere, but add an assert as a safety check. NFC llvm-svn: 280518
2016-09-03 02:29:28 +08:00
} else if (TClobbersPred && FClobbersPred)
llvm_unreachable("Predicate info cannot be clobbered by both sides.");
Really turn if-converter loose: 1. Consider all possible ifcvt cases at once. No longer restricted to bottom up iterative approach. 2. Sort all possible cases based on a cost function. Perform the most profitable ones first invalidate others that target the same blocks. 3. Fixed a number of bugs related to block duplication. llvm-svn: 37613
2007-06-16 17:34:52 +08:00
if (DoSwap) {
Fix a couple of typos and be smarter about order of blocks when ifcvt a diamond. llvm-svn: 37449
2007-06-06 08:57:55 +08:00
std::swap(BBI1, BBI2);
std::swap(Cond1, Cond2);
}
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// Remove the conditional branch from entry to the blocks.
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI.NonPredSize -= TII->removeBranch(*BBI.BB);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &MBB1 = *BBI1->BB;
MachineBasicBlock &MBB2 = *BBI2->BB;
If-conversion incorrectly calculates liveness of redefined registers Differential Revision: https://reviews.llvm.org/D23207 llvm-svn: 278383
2016-08-12 02:42:06 +08:00
// Initialize the Redefs:
// - BB2 live-in regs need implicit uses before being redefined by BB1
// instructions.
// - BB1 live-out regs need implicit uses before being redefined by BB2
// instructions. We start with BB1 live-ins so we have the live-out regs
// after tracking the BB1 instructions.
LivePhysReg: Use reference instead of pointer in init(); NFC llvm-svn: 289002
2016-12-08 08:15:51 +08:00
Redefs.init(*TRI);
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
Redefs.addLiveIns(MBB1);
Redefs.addLiveIns(MBB2);
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// Remove the duplicated instructions at the beginnings of both paths.
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
// Skip dbg_value instructions.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock::iterator DI1 = MBB1.getFirstNonDebugInstr();
MachineBasicBlock::iterator DI2 = MBB2.getFirstNonDebugInstr();
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
BBI1->NonPredSize -= NumDups1;
BBI2->NonPredSize -= NumDups1;
new, no longer brain-dead, r106907 llvm-svn: 107060
2010-06-29 04:26:00 +08:00
// Skip past the dups on each side separately since there may be
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
// differing dbg_value entries. NumDups1 can include a "return"
// instruction, if it's not marked as "branch".
new, no longer brain-dead, r106907 llvm-svn: 107060
2010-06-29 04:26:00 +08:00
for (unsigned i = 0; i < NumDups1; ++DI1) {
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
if (DI1 == MBB1.end())
break;
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (!DI1->isDebugInstr())
new, no longer brain-dead, r106907 llvm-svn: 107060
2010-06-29 04:26:00 +08:00
++i;
}
make sure to handle dbg_value instructions in the middle of the block, not just at the head, when doing diamond if-conversion. rdar://7797940 llvm-svn: 106907
2010-06-26 07:05:46 +08:00
while (NumDups1 != 0) {
[DebugInfo][If-Converter] Update call site info during the optimization During the If-Converter optimization pay attention when copying or deleting call instructions in order to keep call site information in valid state. Reviewers: aprantl, vsk, efriedma Reviewed By: vsk, efriedma Differential Revision: https://reviews.llvm.org/D66955 llvm-svn: 374068
2019-10-08 23:43:12 +08:00
// Since this instruction is going to be deleted, update call
// site info state if the instruction is call instruction.
if (DI2->isCall(MachineInstr::IgnoreBundle))
MBB2.getParent()->eraseCallSiteInfo(&*DI2);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
++DI2;
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
if (DI2 == MBB2.end())
break;
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (!DI2->isDebugInstr())
new, no longer brain-dead, r106907 llvm-svn: 107060
2010-06-29 04:26:00 +08:00
--NumDups1;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
if (MRI->tracksLiveness()) {
for (const MachineInstr &MI : make_range(MBB1.begin(), DI1)) {
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SmallVector<std::pair<MCPhysReg, const MachineOperand*>, 4> Dummy;
CodeGen: Assert that liveness is up to date when reading block live-ins. Add an assert that checks whether liveins are up to date before they are used. - Do not print liveins into .mir files anymore in situations where they are out of date anyway. - The assert in the RegisterScavenger is superseded by the new one in livein_begin(). - Skip parts of the liveness updating logic in IfConversion.cpp when liveness isn't tracked anymore (just enough to avoid hitting the new assert()). Differential Revision: https://reviews.llvm.org/D27562 llvm-svn: 291169
2017-01-06 04:01:19 +08:00
Redefs.stepForward(MI, Dummy);
}
Remove kill flags after if conversion if necessary When if converting something like: true: ... = R0<kill> false: ... = R0<kill> then the instructions of the true block must not have a <kill> flag anymore, as the instruction of the false block follow and do still read the R0 value. Specifically this patch determines the set of register live-in in the false block (possibly after simulating the liveness changes of the duplicated instructions). Each of these live-in registers mustn't be killed. llvm-svn: 192482
2013-10-12 03:04:37 +08:00
}
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
BBI.BB->splice(BBI.BB->end(), &MBB1, MBB1.begin(), DI1);
MBB2.erase(MBB2.begin(), DI2);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
// The branches have been checked to match, so it is safe to remove the
// branch in BB1 and rely on the copy in BB2. The complication is that
// the blocks may end with a return instruction, which may or may not
// be marked as "branch". If it's not, then it could be included in
// "dups1", leaving the blocks potentially empty after moving the common
// duplicates.
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
#ifndef NDEBUG
// Unanalyzable branches must match exactly. Check that now.
if (!BBI1->IsBrAnalyzable)
verifySameBranchInstructions(&MBB1, &MBB2);
#endif
[IfConversion] Fix diamond conversion with unanalyzable branches. The code was incorrectly counting the number of identical instructions, and therefore tried to predicate an instruction which should not have been predicated. This could have various effects: a compiler crash, an assembler failure, a miscompile, or just generating an extra, unnecessary instruction. Instead of depending on TargetInstrInfo::removeBranch, which only works on analyzable branches, just remove all branch instructions. Fixes https://bugs.llvm.org/show_bug.cgi?id=43121 and https://bugs.llvm.org/show_bug.cgi?id=41121 . Differential Revision: https://reviews.llvm.org/D67203 llvm-svn: 371111
2019-09-06 04:02:38 +08:00
// Remove duplicated instructions from the tail of MBB1: any branch
// instructions, and the common instructions counted by NumDups2.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
DI1 = MBB1.end();
[IfConversion] Fix diamond conversion with unanalyzable branches. The code was incorrectly counting the number of identical instructions, and therefore tried to predicate an instruction which should not have been predicated. This could have various effects: a compiler crash, an assembler failure, a miscompile, or just generating an extra, unnecessary instruction. Instead of depending on TargetInstrInfo::removeBranch, which only works on analyzable branches, just remove all branch instructions. Fixes https://bugs.llvm.org/show_bug.cgi?id=43121 and https://bugs.llvm.org/show_bug.cgi?id=41121 . Differential Revision: https://reviews.llvm.org/D67203 llvm-svn: 371111
2019-09-06 04:02:38 +08:00
while (DI1 != MBB1.begin()) {
MachineBasicBlock::iterator Prev = std::prev(DI1);
if (!Prev->isBranch() && !Prev->isDebugInstr())
break;
DI1 = Prev;
}
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
for (unsigned i = 0; i != NumDups2; ) {
// NumDups2 only counted non-dbg_value instructions, so this won't
// run off the head of the list.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
assert(DI1 != MBB1.begin());
[DebugInfo][If-Converter] Update call site info during the optimization During the If-Converter optimization pay attention when copying or deleting call instructions in order to keep call site information in valid state. Reviewers: aprantl, vsk, efriedma Reviewed By: vsk, efriedma Differential Revision: https://reviews.llvm.org/D66955 llvm-svn: 374068
2019-10-08 23:43:12 +08:00
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
--DI1;
[DebugInfo][If-Converter] Update call site info during the optimization During the If-Converter optimization pay attention when copying or deleting call instructions in order to keep call site information in valid state. Reviewers: aprantl, vsk, efriedma Reviewed By: vsk, efriedma Differential Revision: https://reviews.llvm.org/D66955 llvm-svn: 374068
2019-10-08 23:43:12 +08:00
// Since this instruction is going to be deleted, update call
// site info state if the instruction is call instruction.
if (DI1->isCall(MachineInstr::IgnoreBundle))
MBB1.getParent()->eraseCallSiteInfo(&*DI1);
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
// skip dbg_value instructions
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (!DI1->isDebugInstr())
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
++i;
}
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MBB1.erase(DI1, MBB1.end());
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
DI2 = BBI2->BB->end();
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
// The branches have been checked to match. Skip over the branch in the false
// block so that we don't try to predicate it.
if (RemoveBranch)
Finish renaming remaining analyzeBranch functions llvm-svn: 281535
2016-09-15 04:43:16 +08:00
BBI2->NonPredSize -= TII->removeBranch(*BBI2->BB);
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
else {
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
// Make DI2 point to the end of the range where the common "tail"
// instructions could be found.
while (DI2 != MBB2.begin()) {
MachineBasicBlock::iterator Prev = std::prev(DI2);
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (!Prev->isBranch() && !Prev->isDebugInstr())
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
break;
DI2 = Prev;
}
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
}
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
while (NumDups2 != 0) {
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
// NumDups2 only counted non-dbg_value instructions, so this won't
// run off the head of the list.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
assert(DI2 != MBB2.begin());
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
--DI2;
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
// skip dbg_value instructions
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (!DI2->isDebugInstr())
More dbg_value cleanup so the presence of debug info doesn't affect code-gen. Make sure to skip the dbg_value instructions when moving dups out of the diamond. rdar://7797940 llvm-svn: 105965
2010-06-15 05:30:32 +08:00
--NumDups2;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
// Remember which registers would later be defined by the false block.
// This allows us not to predicate instructions in the true block that would
// later be re-defined. That is, rather than
// subeq r0, r1, #1
// addne r0, r1, #1
// generate:
// sub r0, r1, #1
// addne r0, r1, #1
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SmallSet<MCPhysReg, 4> RedefsByFalse;
SmallSet<MCPhysReg, 4> ExtUses;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (TII->isProfitableToUnpredicate(MBB1, MBB2)) {
for (const MachineInstr &FI : make_range(MBB2.begin(), DI2)) {
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (FI.isDebugInstr())
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
continue;
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SmallVector<MCPhysReg, 4> Defs;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (const MachineOperand &MO : FI.operands()) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!MO.isReg())
continue;
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
Register Reg = MO.getReg();
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!Reg)
continue;
if (MO.isDef()) {
Defs.push_back(Reg);
} else if (!RedefsByFalse.count(Reg)) {
// These are defined before ctrl flow reach the 'false' instructions.
// They cannot be modified by the 'true' instructions.
Simplify logic now that r182490 is in place. No functional change intended. llvm-svn: 182531
2013-05-23 07:17:36 +08:00
for (MCSubRegIterator SubRegs(Reg, TRI, /*IncludeSelf=*/true);
SubRegs.isValid(); ++SubRegs)
Switch all register list clients to the new MC*Iterator interface. No functional change intended. Sorry for the churn. The iterator classes are supposed to help avoid giant commits like this one in the future. The TableGen-produced register lists are getting quite large, and it may be necessary to change the table representation. This makes it possible to do so without changing all clients (again). llvm-svn: 157854
2012-06-02 07:28:30 +08:00
ExtUses.insert(*SubRegs);
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
}
}
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
for (MCPhysReg Reg : Defs) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!ExtUses.count(Reg)) {
Simplify logic now that r182490 is in place. No functional change intended. llvm-svn: 182531
2013-05-23 07:17:36 +08:00
for (MCSubRegIterator SubRegs(Reg, TRI, /*IncludeSelf=*/true);
SubRegs.isValid(); ++SubRegs)
Switch all register list clients to the new MC*Iterator interface. No functional change intended. Sorry for the churn. The iterator classes are supposed to help avoid giant commits like this one in the future. The TableGen-produced register lists are getting quite large, and it may be necessary to change the table representation. This makes it possible to do so without changing all clients (again). llvm-svn: 157854
2012-06-02 07:28:30 +08:00
RedefsByFalse.insert(*SubRegs);
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
}
}
}
}
// Predicate the 'true' block.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
PredicateBlock(*BBI1, MBB1.end(), *Cond1, &RedefsByFalse);
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// After predicating BBI1, if there is a predicated terminator in BBI1 and
// a non-predicated in BBI2, then we don't want to predicate the one from
// BBI2. The reason is that if we merged these blocks, we would end up with
// two predicated terminators in the same block.
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
// Also, if the branches in MBB1 and MBB2 were non-analyzable, then don't
// predicate them either. They were checked to be identical, and so the
// same branch would happen regardless of which path was taken.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (!MBB2.empty() && (DI2 == MBB2.end())) {
MachineBasicBlock::iterator BBI1T = MBB1.getFirstTerminator();
MachineBasicBlock::iterator BBI2T = MBB2.getFirstTerminator();
[if-converter] Handle BBs that terminate in ret during diamond conversion This fixes https://llvm.org/PR36825. Original patch by Valentin Churavy (D45218). Differential Revision: https://reviews.llvm.org/D45731 llvm-svn: 330345
2018-04-20 01:26:46 +08:00
bool BB1Predicated = BBI1T != MBB1.end() && TII->isPredicated(*BBI1T);
bool BB2NonPredicated = BBI2T != MBB2.end() && !TII->isPredicated(*BBI2T);
if (BB2NonPredicated && (BB1Predicated || !BBI2->IsBrAnalyzable))
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
--DI2;
}
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
// Predicate the 'false' block.
Use a SparseSet in LiveRegUnits. Some clients may add block live ins and may track liveness over a large scope. This guarantees an efficient implementation in all cases with no memory allocation/deallocation, independent of the number of target registers. It could be slightly less convenient but is fine in the expected case. llvm-svn: 192622
2013-10-15 04:45:17 +08:00
PredicateBlock(*BBI2, DI2, *Cond2);
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
// Merge the true block into the entry of the diamond.
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
MergeBlocks(BBI, *BBI1, MergeAddEdges);
MergeBlocks(BBI, *BBI2, MergeAddEdges);
return true;
}
/// If convert an almost-diamond sub-CFG where the true
/// and false blocks share a common tail.
bool IfConverter::IfConvertForkedDiamond(
BBInfo &BBI, IfcvtKind Kind,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred) {
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
// Save the debug location for later.
DebugLoc dl;
MachineBasicBlock::iterator TIE = TrueBBI.BB->getFirstTerminator();
if (TIE != TrueBBI.BB->end())
dl = TIE->getDebugLoc();
// Removing branches from both blocks is safe, because we have already
// determined that both blocks have the same branch instructions. The branch
// will be added back at the end, unpredicated.
if (!IfConvertDiamondCommon(
BBI, TrueBBI, FalseBBI,
NumDups1, NumDups2,
TClobbersPred, FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
/* RemoveBranch */ true, /* MergeAddEdges */ true))
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
return false;
// Add back the branch.
// Debug location saved above when removing the branch from BBI2
Make analyzeBranch family of instruction names consistent analyzeBranch was renamed to use lowercase first, rename the related set to match. llvm-svn: 281506
2016-09-15 01:24:15 +08:00
TII->insertBranch(*BBI.BB, TrueBBI.TrueBB, TrueBBI.FalseBB,
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
TrueBBI.BrCond, dl);
// Update block info.
BBI.IsDone = TrueBBI.IsDone = FalseBBI.IsDone = true;
InvalidatePreds(*BBI.BB);
// FIXME: Must maintain LiveIns.
return true;
}
/// If convert a diamond sub-CFG.
bool IfConverter::IfConvertDiamond(BBInfo &BBI, IfcvtKind Kind,
unsigned NumDups1, unsigned NumDups2,
bool TClobbersPred, bool FClobbersPred) {
BBInfo &TrueBBI = BBAnalysis[BBI.TrueBB->getNumber()];
BBInfo &FalseBBI = BBAnalysis[BBI.FalseBB->getNumber()];
MachineBasicBlock *TailBB = TrueBBI.TrueBB;
// True block must fall through or end with an unanalyzable terminator.
if (!TailBB) {
if (blockAlwaysFallThrough(TrueBBI))
TailBB = FalseBBI.TrueBB;
assert((TailBB || !TrueBBI.IsBrAnalyzable) && "Unexpected!");
}
if (!IfConvertDiamondCommon(
BBI, TrueBBI, FalseBBI,
NumDups1, NumDups2,
IfConversion: Fix bug introduced by rescanning diamonds. Passing the wrong values for predicate-clobbering. Simple to miss. Added an assert to make this easier to catch in the future. llvm-svn: 280517
2016-09-03 02:29:26 +08:00
TClobbersPred, FClobbersPred,
IfConversion: Fix branch predication bug. This bug shows up with diamonds that share unpredicable, unanalyzable branches. There's an included test case from Hexagon. What was happening was that we were attempting to predicate the branch instruction despite the fact that it was checked to be the same. Now for unanalyzable branches we skip over the branch instructions when predicating the block. Differential Revision: https://reviews.llvm.org/D23939 llvm-svn: 279985
2016-08-30 02:27:12 +08:00
/* RemoveBranch */ TrueBBI.IsBrAnalyzable,
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
/* MergeAddEdges */ TailBB == nullptr))
return false;
Fix a couple of typos and be smarter about order of blocks when ifcvt a diamond. llvm-svn: 37449
2007-06-06 08:57:55 +08:00
Fix some typos and spelling and grammar, mostly in comments, but also one field name. No functional changes. llvm-svn: 71739
2009-05-14 07:25:24 +08:00
// If the if-converted block falls through or unconditionally branches into
// the tail block, and the tail block does not have other predecessors, then
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
// fold the tail block in as well. Otherwise, unless it falls through to the
// tail, add a unconditional branch to it.
if (TailBB) {
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
// We need to remove the edges to the true and false blocks manually since
// we didn't let IfConvertDiamondCommon update the CFG.
BBI.BB->removeSuccessor(TrueBBI.BB);
BBI.BB->removeSuccessor(FalseBBI.BB, true);
Added missing &. Fixes <rdar://problem/10393723> llvm-svn: 143753
2011-11-05 07:49:14 +08:00
BBInfo &TailBBI = BBAnalysis[TailBB->getNumber()];
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
bool CanMergeTail = !TailBBI.HasFallThrough &&
!TailBBI.BB->hasAddressTaken();
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// The if-converted block can still have a predicated terminator
// (e.g. a predicated return). If that is the case, we cannot merge
// it with the tail block.
MachineBasicBlock::const_iterator TI = BBI.BB->getFirstTerminator();
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (TI != BBI.BB->end() && TII->isPredicated(*TI))
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
CanMergeTail = false;
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
// There may still be a fall-through edge from BBI1 or BBI2 to TailBB;
// check if there are any other predecessors besides those.
unsigned NumPreds = TailBB->pred_size();
if (NumPreds > 1)
CanMergeTail = false;
else if (NumPreds == 1 && CanMergeTail) {
MachineBasicBlock::pred_iterator PI = TailBB->pred_begin();
CodeGen: If Convert blocks that would form a diamond when tail-merged. The following function currently relies on tail-merging for if conversion to succeed. The common tail of cond_true and cond_false is extracted, and this then forms a diamond pattern that can be successfully if converted. If this block does not get extracted, either because tail-merging is disabled or the threshold is higher, we should still recognize this pattern and if-convert it. Fixed a regression in the original commit. Need to un-reverse branches after reversing them, or other conversions go awry. define i32 @t2(i32 %a, i32 %b) nounwind { entry: %tmp1434 = icmp eq i32 %a, %b ; <i1> [#uses=1] br i1 %tmp1434, label %bb17, label %bb.outer bb.outer: ; preds = %cond_false, %entry %b_addr.021.0.ph = phi i32 [ %b, %entry ], [ %tmp10, %cond_false ] %a_addr.026.0.ph = phi i32 [ %a, %entry ], [ %a_addr.026.0, %cond_false ] br label %bb bb: ; preds = %cond_true, %bb.outer %indvar = phi i32 [ 0, %bb.outer ], [ %indvar.next, %cond_true ] %tmp. = sub i32 0, %b_addr.021.0.ph %tmp.40 = mul i32 %indvar, %tmp. %a_addr.026.0 = add i32 %tmp.40, %a_addr.026.0.ph %tmp3 = icmp sgt i32 %a_addr.026.0, %b_addr.021.0.ph br i1 %tmp3, label %cond_true, label %cond_false cond_true: ; preds = %bb %tmp7 = sub i32 %a_addr.026.0, %b_addr.021.0.ph %tmp1437 = icmp eq i32 %tmp7, %b_addr.021.0.ph %indvar.next = add i32 %indvar, 1 br i1 %tmp1437, label %bb17, label %bb cond_false: ; preds = %bb %tmp10 = sub i32 %b_addr.021.0.ph, %a_addr.026.0 %tmp14 = icmp eq i32 %a_addr.026.0, %tmp10 br i1 %tmp14, label %bb17, label %bb.outer bb17: ; preds = %cond_false, %cond_true, %entry %a_addr.026.1 = phi i32 [ %a, %entry ], [ %tmp7, %cond_true ], [ %a_addr.026.0, %cond_false ] ret i32 %a_addr.026.1 } Without tail-merging or diamond-tail if conversion: LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ble LBB1_3 @ BB#2: @ %cond_true @ in Loop: Header=BB1_1 Depth=1 subs r0, r0, r1 cmp r1, r0 it ne cmpne r0, r1 bgt LBB1_4 LBB1_3: @ %cond_false @ in Loop: Header=BB1_1 Depth=1 subs r1, r1, r0 cmp r1, r0 bne LBB1_1 LBB1_4: @ %bb17 bx lr With diamond-tail if conversion, but without tail-merging: @ BB#0: @ %entry cmp r0, r1 it eq bxeq lr LBB1_1: @ %bb @ =>This Inner Loop Header: Depth=1 cmp r0, r1 ite le suble r1, r1, r0 subgt r0, r0, r1 cmp r1, r0 bne LBB1_1 @ BB#2: @ %bb17 bx lr llvm-svn: 279671
2016-08-25 05:34:27 +08:00
if (*PI != TrueBBI.BB && *PI != FalseBBI.BB)
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
CanMergeTail = false;
}
if (CanMergeTail) {
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
MergeBlocks(BBI, TailBBI);
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
TailBBI.IsDone = true;
} else {
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
BBI.BB->addSuccessor(TailBB, BranchProbability::getOne());
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InsertUncondBranch(*BBI.BB, *TailBB, TII);
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
BBI.HasFallThrough = false;
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
}
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
}
Make use of target specific block size limits; bug fixes. llvm-svn: 37195
2007-05-18 08:20:58 +08:00
Fix diamond shape ifcvt bugs. llvm-svn: 37444
2007-06-06 07:46:14 +08:00
// Update block info.
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
BBI.IsDone = TrueBBI.IsDone = FalseBBI.IsDone = true;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
InvalidatePreds(*BBI.BB);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
If-convert early exit blocks (returns, etc.); bug fixes, etc. llvm-svn: 37270
2007-05-22 06:22:58 +08:00
// FIXME: Must maintain LiveIns.
return true;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
static bool MaySpeculate(const MachineInstr &MI,
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SmallSet<MCPhysReg, 4> &LaterRedefs) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
bool SawStore = true;
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
if (!MI.isSafeToMove(nullptr, SawStore))
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
return false;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (const MachineOperand &MO : MI.operands()) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!MO.isReg())
continue;
Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-16 03:22:08 +08:00
Register Reg = MO.getReg();
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (!Reg)
continue;
if (MO.isDef() && !LaterRedefs.count(Reg))
return false;
}
return true;
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Predicate instructions from the start of the block to the specified end with
/// the specified condition.
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
void IfConverter::PredicateBlock(BBInfo &BBI,
Properly remove duplicate instructions as result of diamond if-conversion. Other bug fixes. llvm-svn: 37623
2007-06-18 16:37:25 +08:00
MachineBasicBlock::iterator E,
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
SmallVectorImpl<MachineOperand> &Cond,
LivePhysRegs/IfConversion: Change some types from unsigned to MCPhysReg; NFC Change the type in a couple of lists and sets that only store physical registers from unsigned to MCPhysRegs. The later is only 16bits and saves us a bit of memory. llvm-svn: 346254
2018-11-07 03:00:11 +08:00
SmallSet<MCPhysReg, 4> *LaterRedefs) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
bool AnyUnpred = false;
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
bool MaySpec = LaterRedefs != nullptr;
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineInstr &I : make_range(BBI.BB->begin(), E)) {
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (I.isDebugInstr() || TII->isPredicated(I))
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
continue;
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
// It may be possible not to predicate an instruction if it's the 'true'
// side of a diamond and the 'false' side may re-define the instruction's
// defs.
MachineInstr: Remove unused parameter. llvm-svn: 237726
2015-05-20 05:22:20 +08:00
if (MaySpec && MaySpeculate(I, *LaterRedefs)) {
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
AnyUnpred = true;
continue;
}
// If any instruction is predicated, then every instruction after it must
// be predicated.
MaySpec = false;
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
if (!TII->PredicateInstruction(I, Cond)) {
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#ifndef NDEBUG
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
dbgs() << "Unable to predicate " << I << "!\n";
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#endif
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
llvm_unreachable(nullptr);
isBlockPredicable() always ignore terminal instructions; add comments. llvm-svn: 37126
2007-05-17 05:54:37 +08:00
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
Tidy. llvm-svn: 106383
2010-06-19 13:33:57 +08:00
// If the predicated instruction now redefines a register as the result of
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
// if-conversion, add an implicit kill.
CodeGen: Use MachineInstr& in IfConversion, NFC Switch to a range-based for in IfConverter::PredicateBlock and take MachineInstr& in MaySpeculate to avoid an implicit conversion from MachineBasicBlock::iterator to MachineInstr*. llvm-svn: 274290
2016-07-01 07:04:51 +08:00
UpdatePredRedefs(I, Redefs);
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
Replace std::copy with a back inserter with vector append where feasible All of the cases were just appending from random access iterators to a vector. Using insert/append can grow the vector to the perfect size directly and moves the growing out of the loop. No intended functionalty change. llvm-svn: 230845
2015-02-28 18:11:12 +08:00
BBI.Predicate.append(Cond.begin(), Cond.end());
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
Fix some stupid bugs that have effectively disabled if-conversion. llvm-svn: 37575
2007-06-15 04:28:52 +08:00
BBI.IsAnalyzed = false;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
BBI.NonPredSize = 0;
Correct transfer predicate information. llvm-svn: 37524
2007-06-09 03:17:12 +08:00
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
++NumIfConvBBs;
Add a if-conversion optimization that allows 'true' side of a diamond to be unpredicated. That is, turn subeq r0, r1, #1 addne r0, r1, #1 into sub r0, r1, #1 addne r0, r1, #1 For targets where conditional instructions are always executed, this may be beneficial. It may remove pseudo anti-dependency in out-of-order execution CPUs. e.g. op r1, ... str r1, [r10] ; end-of-life of r1 as div result cmp r0, #65 movne r1, #44 ; raw dependency on previous r1 moveq r1, #12 If movne is unpredicated, then op r1, ... str r1, [r10] cmp r0, #65 mov r1, #44 ; r1 written unconditionally moveq r1, #12 Both mov and moveq are no longer depdendent on the first instruction. This gives the out-of-order execution engine more freedom to reorder them. This has passed entire LLVM test suite. But it has not been enabled for any ARM variant pending more performance evaluation. rdar://8951196 llvm-svn: 146914
2011-12-20 06:01:30 +08:00
if (AnyUnpred)
++NumUnpred;
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Copy and predicate instructions from source BB to the destination block.
/// Skip end of block branches if IgnoreBr is true.
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
void IfConverter::CopyAndPredicateBlock(BBInfo &ToBBI, BBInfo &FromBBI,
Convert uses of std::vector in TargetInstrInfo to SmallVector. This change had to be propoagated down into all the targets and up into all clients of this API. llvm-svn: 54802
2008-08-15 06:49:33 +08:00
SmallVectorImpl<MachineOperand> &Cond,
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
bool IgnoreBr) {
Pool-allocation for MachineInstrs, MachineBasicBlocks, and MachineMemOperands. The pools are owned by MachineFunctions. This drastically reduces the number of calls to malloc/free made during the "Emit" phase of scheduling, as well as later phases in CodeGen. Combined with other changes, this speeds up the "instruction selection" phase of CodeGen by 10% in some cases. llvm-svn: 53212
2008-07-08 07:14:23 +08:00
MachineFunction &MF = *ToBBI.BB->getParent();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &FromMBB = *FromBBI.BB;
for (MachineInstr &I : FromMBB) {
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
// Do not copy the end of the block branches.
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (IgnoreBr && I.isBranch())
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
break;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
MachineInstr *MI = MF.CloneMachineInstr(&I);
[DebugInfo][If-Converter] Update call site info during the optimization During the If-Converter optimization pay attention when copying or deleting call instructions in order to keep call site information in valid state. Reviewers: aprantl, vsk, efriedma Reviewed By: vsk, efriedma Differential Revision: https://reviews.llvm.org/D66955 llvm-svn: 374068
2019-10-08 23:43:12 +08:00
// Make a copy of the call site info.
if (MI->isCall(MachineInstr::IgnoreBundle))
MF.copyCallSiteInfo(&I,MI);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
ToBBI.BB->insert(ToBBI.BB->end(), MI);
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
ToBBI.NonPredSize++;
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
unsigned ExtraPredCost = TII->getPredicationCost(I);
unsigned NumCycles = SchedModel.computeInstrLatency(&I, false);
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
if (NumCycles > 1)
ToBBI.ExtraCost += NumCycles-1;
ToBBI.ExtraCost2 += ExtraPredCost;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
[DebugInfo] Examine all uses of isDebugValue() for debug instructions. Because we create a new kind of debug instruction, DBG_LABEL, we need to check all passes which use isDebugValue() to check MachineInstr is debug instruction or not. When expelling debug instructions, we should expel both DBG_VALUE and DBG_LABEL. So, I create a new function, isDebugInstr(), in MachineInstr to check whether the MachineInstr is debug instruction or not. This patch has no new test case. I have run regression test and there is no difference in regression test. Differential Revision: https://reviews.llvm.org/D45342 Patch by Hsiangkai Wang. llvm-svn: 331844
2018-05-09 10:42:00 +08:00
if (!TII->isPredicated(I) && !MI->isDebugInstr()) {
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
if (!TII->PredicateInstruction(*MI, Cond)) {
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#ifndef NDEBUG
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
dbgs() << "Unable to predicate " << I << "!\n";
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-13 04:07:01 +08:00
#endif
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
llvm_unreachable(nullptr);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
}
Tidy. llvm-svn: 106383
2010-06-19 13:33:57 +08:00
// If the predicated instruction now redefines a register as the result of
Make post-ra scheduling, anti-dep breaking, and register scavenger (conservatively) aware of predicated instructions. This enables ARM to move if-conversion before post-ra scheduler. llvm-svn: 106091
2010-06-16 15:35:02 +08:00
// if-conversion, add an implicit kill.
CodeGen: TII: Take MachineInstr& in predicate API, NFC Change TargetInstrInfo API to take `MachineInstr&` instead of `MachineInstr*` in the functions related to predicated instructions (I'll try to come back later and get some of the rest). All of these functions require non-null parameters already, so references are more clear. As a bonus, this happens to factor away a host of implicit iterator => pointer conversions. No functionality change intended. llvm-svn: 261605
2016-02-23 10:46:52 +08:00
UpdatePredRedefs(*MI, Redefs);
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
if (!IgnoreBr) {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
std::vector<MachineBasicBlock *> Succs(FromMBB.succ_begin(),
FromMBB.succ_end());
MachineBasicBlock *NBB = getNextBlock(FromMBB);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
MachineBasicBlock *FallThrough = FromBBI.HasFallThrough ? NBB : nullptr;
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock *Succ : Succs) {
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
// Fallthrough edge can't be transferred.
if (Succ == FallThrough)
continue;
ToBBI.BB->addSuccessor(Succ);
}
Fix some fragile code wrt CFG edge updating. llvm-svn: 37634
2007-06-19 06:44:57 +08:00
}
Replace std::copy with a back inserter with vector append where feasible All of the cases were just appending from random access iterators to a vector. Using insert/append can grow the vector to the perfect size directly and moves the growing out of the loop. No intended functionalty change. llvm-svn: 230845
2015-02-28 18:11:12 +08:00
ToBBI.Predicate.append(FromBBI.Predicate.begin(), FromBBI.Predicate.end());
ToBBI.Predicate.append(Cond.begin(), Cond.end());
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
ToBBI.ClobbersPred |= FromBBI.ClobbersPred;
ToBBI.IsAnalyzed = false;
Use pre-increment instead of post-increment when the result is not used. llvm-svn: 106542
2010-06-22 23:08:57 +08:00
++NumDupBBs;
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
}
IfConversion: Improve doxygen comments llvm-svn: 278893
2016-08-17 10:51:57 +08:00
/// Move all instructions from FromBB to the end of ToBB. This will leave
/// FromBB as an empty block, so remove all of its successor edges except for
/// the fall-through edge. If AddEdges is true, i.e., when FromBBI's branch is
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
/// being moved, add those successor edges to ToBBI and remove the old edge
/// from ToBBI to FromBBI.
Reapply my if-conversion cleanup from svn r106939 with fixes. There are 2 changes relative to the previous version of the patch: 1) For the "simple" if-conversion case, there's no need to worry about RemoveExtraEdges not handling an unanalyzable branch. Predicated terminators are ignored in this context, so RemoveExtraEdges does the right thing. This might break someday if we ever treat indirect branches (BRIND) as predicable, but for now, I just removed this part of the patch, because in the case where we do not add an unconditional branch, we rely on keeping the fall-through edge to CvtBBI (which is empty after this transformation). The change relative to the previous patch is: @@ -1036,10 +1036,6 @@ IterIfcvt = false; } - // RemoveExtraEdges won't work if the block has an unanalyzable branch, - // which is typically the case for IfConvertSimple, so explicitly remove - // CvtBBI as a successor. - BBI.BB->removeSuccessor(CvtBBI->BB); RemoveExtraEdges(BBI); // Update block info. BB can be iteratively if-converted. 2) My patch exposed a bug in the code for merging the tail of a "diamond", which had previously never been exercised. The code was simply checking that the tail had a single predecessor, but there was a case in MultiSource/Benchmarks/VersaBench/dbms where that single predecessor was neither edge of the diamond. I added the following change to check for that: @@ -1276,7 +1276,18 @@ // tail, add a unconditional branch to it. if (TailBB) { BBInfo TailBBI = BBAnalysis[TailBB->getNumber()]; - if (TailBB->pred_size() == 1 && !TailBBI.HasFallThrough) { + bool CanMergeTail = !TailBBI.HasFallThrough; + // There may still be a fall-through edge from BBI1 or BBI2 to TailBB; + // check if there are any other predecessors besides those. + unsigned NumPreds = TailBB->pred_size(); + if (NumPreds > 1) + CanMergeTail = false; + else if (NumPreds == 1 && CanMergeTail) { + MachineBasicBlock::pred_iterator PI = TailBB->pred_begin(); + if (*PI != BBI1->BB && *PI != BBI2->BB) + CanMergeTail = false; + } + if (CanMergeTail) { MergeBlocks(BBI, TailBBI); TailBBI.IsDone = true; } else { With these fixes, I was able to run all the SingleSource and MultiSource tests successfully. llvm-svn: 107110
2010-06-29 08:55:23 +08:00
void IfConverter::MergeBlocks(BBInfo &ToBBI, BBInfo &FromBBI, bool AddEdges) {
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock &FromMBB = *FromBBI.BB;
assert(!FromMBB.hasAddressTaken() &&
Teach if-converter to avoid removing BBs whose addresses are takne. rdar://13782395 llvm-svn: 181160
2013-05-06 02:03:49 +08:00
"Removing a BB whose address is taken!");
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// In case FromMBB contains terminators (e.g. return instruction),
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// first move the non-terminator instructions, then the terminators.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
MachineBasicBlock::iterator FromTI = FromMBB.getFirstTerminator();
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
MachineBasicBlock::iterator ToTI = ToBBI.BB->getFirstTerminator();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
ToBBI.BB->splice(ToTI, &FromMBB, FromMBB.begin(), FromTI);
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
// If FromBB has non-predicated terminator we should copy it at the end.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (FromTI != FromMBB.end() && !TII->isPredicated(*FromTI))
Proper handling of diamond-like cases in if-conversion If converter was somewhat careless about "diamond" cases, where there was no join block, or in other words, where the true/false blocks did not have analyzable branches. In such cases, it was possible for it to remove (needed) branches, resulting in a loss of entire basic blocks. Differential Revision: http://reviews.llvm.org/D16156 llvm-svn: 258310
2016-01-20 21:14:52 +08:00
ToTI = ToBBI.BB->end();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
ToBBI.BB->splice(ToTI, &FromMBB, FromTI, FromMBB.end());
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
Fix PR25838. This is a quick fix to PR25838. The issue comes from the restriction that we cannot normalize probabilities containing both known and unknown ones. A patch that removes this restriction is under the review now: http://reviews.llvm.org/D15548 llvm-svn: 255867
2015-12-17 09:29:08 +08:00
// Force normalizing the successors' probabilities of ToBBI.BB to convert all
// unknown probabilities into known ones.
// FIXME: This usage is too tricky and in the future we would like to
// eliminate all unknown probabilities in MBB.
[IfConversion] Only renormalize probabilities if branches are analyzable If a block has non-analyzable branches, the listed successors don't need to add up to one. For example, if a block has a conditional tail call, that tail call will not have a corresponding successor in the successor list, but will still be a possible branch. Differential Revision: https://reviews.llvm.org/D30556 llvm-svn: 297054
2017-03-07 03:12:42 +08:00
if (ToBBI.IsBrAnalyzable)
ToBBI.BB->normalizeSuccProbs();
Fix PR25838. This is a quick fix to PR25838. The issue comes from the restriction that we cannot normalize probabilities containing both known and unknown ones. A patch that removes this restriction is under the review now: http://reviews.llvm.org/D15548 llvm-svn: 255867
2015-12-17 09:29:08 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
SmallVector<MachineBasicBlock *, 4> FromSuccs(FromMBB.succ_begin(),
FromMBB.succ_end());
MachineBasicBlock *NBB = getNextBlock(FromMBB);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 08:51:57 +08:00
MachineBasicBlock *FallThrough = FromBBI.HasFallThrough ? NBB : nullptr;
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// The edge probability from ToBBI.BB to FromMBB, which is only needed when
// AddEdges is true and FromMBB is a successor of ToBBI.BB.
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto To2FromProb = BranchProbability::getZero();
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
if (AddEdges && ToBBI.BB->isSuccessor(&FromMBB)) {
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
// Remove the old edge but remember the edge probability so we can calculate
// the correct weights on the new edges being added further down.
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
To2FromProb = MBPI->getEdgeProbability(ToBBI.BB, &FromMBB);
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
ToBBI.BB->removeSuccessor(&FromMBB);
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
}
IfConversion: Use range based for; NFC Also avoid some pointless use of auto! Because that's friendlier to readers and avoids several types accidentally resolving to unnecessary references here (MachineInstr *&, unsigned &). llvm-svn: 278894
2016-08-17 10:51:59 +08:00
for (MachineBasicBlock *Succ : FromSuccs) {
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
// Fallthrough edge can't be transferred.
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
if (Succ == FallThrough)
continue;
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
auto NewProb = BranchProbability::getZero();
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
if (AddEdges) {
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// Calculate the edge probability for the edge from ToBBI.BB to Succ,
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// which is a portion of the edge probability from FromMBB to Succ. The
// portion ratio is the edge probability from ToBBI.BB to FromMBB (if
[NFC] fix trivial typos in comments llvm-svn: 350690
2019-01-09 13:11:10 +08:00
// FromBBI is a successor of ToBBI.BB. See comment below for exception).
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
NewProb = MBPI->getEdgeProbability(&FromMBB, Succ);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// To2FromProb is 0 when FromMBB is not a successor of ToBBI.BB. This
// only happens when if-converting a diamond CFG and FromMBB is the
// tail BB. In this case FromMBB post-dominates ToBBI.BB and hence we
// could just use the probabilities on FromMBB's out-edges when adding
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// new successors.
if (!To2FromProb.isZero())
NewProb *= To2FromProb;
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
}
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
FromMBB.removeSuccessor(Succ);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
if (AddEdges) {
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// If the edge from ToBBI.BB to Succ already exists, update the
Normalize MBB's successors' probabilities in several locations. This patch adds some missing calls to MBB::normalizeSuccProbs() in several locations where it should be called. Those places are found by checking if the sum of successors' probabilities is approximate one in MachineBlockPlacement pass with some instrumented code (not in this patch). Differential revision: http://reviews.llvm.org/D15259 llvm-svn: 255455
2015-12-13 17:26:17 +08:00
// probability of this edge by adding NewProb to it. An example is shown
IfConversion: Use references instead of pointers where possible; NFC Also put some commonly used subexpressions into variables. llvm-svn: 278895
2016-08-17 10:52:01 +08:00
// below, in which A is ToBBI.BB and B is FromMBB. In this case we
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
// don't have to set C as A's successor as it already is. We only need to
// update the edge probability on A->C. Note that B will not be
// immediately removed from A's successors. It is possible that B->D is
// not removed either if D is a fallthrough of B. Later the edge A->D
// (generated here) and B->D will be combined into one edge. To maintain
// correct edge probability of this combined edge, we need to set the edge
// probability of A->B to zero, which is already done above. The edge
// probability on A->D is calculated by scaling the original probability
// on A->B by the probability of B->D.
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
//
// Before ifcvt: After ifcvt (assume B->D is kept):
//
// A A
// /| /|\
// / B / B|
// | /| | ||
// |/ | | |/
// C D C D
//
if (ToBBI.BB->isSuccessor(Succ))
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
ToBBI.BB->setSuccProbability(
Use the range variant of find/find_if instead of unpacking begin/end If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278469
2016-08-12 11:55:06 +08:00
find(ToBBI.BB->successors(), Succ),
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
MBPI->getEdgeProbability(ToBBI.BB, Succ) + NewProb);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
else
Replace all weight-based interfaces in MBB with probability-based interfaces, and update all uses of old interfaces. (This is the second attempt to submit this patch. The first caused two assertion failures and was reverted. See https://llvm.org/bugs/show_bug.cgi?id=25687) The patch in http://reviews.llvm.org/D13745 is broken into four parts: 1. New interfaces without functional changes (http://reviews.llvm.org/D13908). 2. Use new interfaces in SelectionDAG, while in other passes treat probabilities as weights (http://reviews.llvm.org/D14361). 3. Use new interfaces in all other passes. 4. Remove old interfaces. This patch is 3+4 above. In this patch, MBB won't provide weight-based interfaces any more, which are totally replaced by probability-based ones. The interface addSuccessor() is redesigned so that the default probability is unknown. We allow unknown probabilities but don't allow using it together with known probabilities in successor list. That is to say, we either have a list of successors with all known probabilities, or all unknown probabilities. In the latter case, we assume each successor has 1/N probability where N is the number of successors. An assertion checks if the user is attempting to add a successor with the disallowed mixed use as stated above. This can help us catch many misuses. All uses of weight-based interfaces are now updated to use probability-based ones. Differential revision: http://reviews.llvm.org/D14973 llvm-svn: 254377
2015-12-01 13:29:22 +08:00
ToBBI.BB->addSuccessor(Succ, NewProb);
Update edge weights properly when merging blocks in if-conversion. In if-conversion, there is a utility function MergeBlocks() that is used to merge blocks. However, when new edges are built in this function the edge weight is either not provided or not updated properly, leading to a modified CFG with incorrect edge weights. This patch corrects this issue. Differential Revision: http://reviews.llvm.org/D12513 llvm-svn: 248030
2015-09-19 04:22:41 +08:00
}
Lots of bug fixes. Now finally in a reasonable state. llvm-svn: 37485
2007-06-07 10:12:15 +08:00
}
Let IfConverter loose. Allow more aggressive subsumptions; reorder basic blocks to expose more ifcvt opportunities; code clean up and fixes. llvm-svn: 37409
2007-06-04 14:47:22 +08:00
[IfConversion] Maintain the CFG when predicating/merging blocks in IfConvert* Summary: This fixes PR32721 in IfConvertTriangle and possible similar problems in IfConvertSimple, IfConvertDiamond and IfConvertForkedDiamond. In PR32721 we had a triangle EBB | \ | | | TBB | / FBB where FBB didn't have any successors at all since it ended with an unconditional return. Then TBB and FBB were be merged into EBB, but EBB would still keep its successors, and the use of analyzeBranch and CorrectExtraCFGEdges wouldn't help to remove them since the return instruction is not analyzable (at least not on ARM). The edge updating code and branch probability updating code is now pushed into MergeBlocks() which allows us to share the same update logic between more callsites. This lets us remove several dependencies on analyzeBranch and completely eliminate RemoveExtraEdges. One thing that showed up with this patch was that IfConversion sometimes left a successor with 0% probability even if there was no branch or fallthrough to the successor. One such example from the test case ifcvt_bad_zero_prob_succ.mir. The indirect branch tBRIND can only jump to bb.1, but without the patch we got: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000), %bb.2(0x00000000) tBRIND %r1, 1, %cpsr B %bb.1 bb.2: There is no way to jump from bb.1 to bb2, but still there is a 0% edge from bb.1 to bb.2. With the patch applied we instead get the expected: bb.0: successors: %bb.1(0x80000000) bb.1: successors: %bb.1(0x80000000) tBRIND %r1, 1, %cpsr B %bb.1 Since bb.2 had no predecessor at all, it was removed. Several testcases had to be updated due to this since the removed successor made the "Branch Probability Basic Block Placement" pass sometimes place blocks in a different order. Finally added a couple of new test cases: * PR32721_ifcvt_triangle_unanalyzable.mir: Regression test for the original problem dexcribed in PR 32721. * ifcvt_triangleWoCvtToNextEdge.mir: Regression test for problem that caused a revert of my first attempt to solve PR 32721. * ifcvt_simple_bad_zero_prob_succ.mir: Test case showing the problem where a wrong successor with 0% probability was previously left. * ifcvt_[diamond|forked_diamond|simple]_unanalyzable.mir Very simple test cases for the simple and (forked) diamond cases involving unanalyzable branches that can be nice to have as a base if wanting to write more complicated tests. Reviewers: iteratee, MatzeB, grosser, kparzysz Reviewed By: kparzysz Subscribers: kbarton, davide, aemerson, nemanjai, javed.absar, kristof.beyls, llvm-commits Differential Revision: https://reviews.llvm.org/D34099 llvm-svn: 310697
2017-08-11 14:57:08 +08:00
// Move the now empty FromMBB out of the way to the end of the function so
// it doesn't interfere with fallthrough checks done by canFallThroughTo().
MachineBasicBlock *Last = &*FromMBB.getParent()->rbegin();
if (Last != &FromMBB)
FromMBB.moveAfter(Last);
Carefully remove extraneous CFG edges after each ifcvt. llvm-svn: 37529
2007-06-09 06:01:07 +08:00
Normalize MBB's successors' probabilities in several locations. This patch adds some missing calls to MBB::normalizeSuccProbs() in several locations where it should be called. Those places are found by checking if the sum of successors' probabilities is approximate one in MachineBlockPlacement pass with some instrumented code (not in this patch). Differential revision: http://reviews.llvm.org/D15259 llvm-svn: 255455
2015-12-13 17:26:17 +08:00
// Normalize the probabilities of ToBBI.BB's successors with all adjustment
// we've done above.
[IfConversion] Only renormalize probabilities if branches are analyzable If a block has non-analyzable branches, the listed successors don't need to add up to one. For example, if a block has a conditional tail call, that tail call will not have a corresponding successor in the successor list, but will still be a possible branch. Differential Revision: https://reviews.llvm.org/D30556 llvm-svn: 297054
2017-03-07 03:12:42 +08:00
if (ToBBI.IsBrAnalyzable && FromBBI.IsBrAnalyzable)
ToBBI.BB->normalizeSuccProbs();
Normalize MBB's successors' probabilities in several locations. This patch adds some missing calls to MBB::normalizeSuccProbs() in several locations where it should be called. Those places are found by checking if the sum of successors' probabilities is approximate one in MachineBlockPlacement pass with some instrumented code (not in this patch). Differential revision: http://reviews.llvm.org/D15259 llvm-svn: 255455
2015-12-13 17:26:17 +08:00
Replace std::copy with a back inserter with vector append where feasible All of the cases were just appending from random access iterators to a vector. Using insert/append can grow the vector to the perfect size directly and moves the growing out of the loop. No intended functionalty change. llvm-svn: 230845
2015-02-28 18:11:12 +08:00
ToBBI.Predicate.append(FromBBI.Predicate.begin(), FromBBI.Predicate.end());
Allow small blocks to be duplicated to enable if-conversion. llvm-svn: 37590
2007-06-15 15:36:12 +08:00
FromBBI.Predicate.clear();
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
ToBBI.NonPredSize += FromBBI.NonPredSize;
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
ToBBI.ExtraCost += FromBBI.ExtraCost;
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
ToBBI.ExtraCost2 += FromBBI.ExtraCost2;
Preliminary iterative if-conversion support. llvm-svn: 37309
2007-05-23 15:23:16 +08:00
FromBBI.NonPredSize = 0;
Change if-conversion to keep track of the extra cost due to microcoded instructions separately from the count of non-predicated instructions. The instruction count is used in places to determine how many instructions to copy, predicate, etc. and things get confused if that count includes the extra cost for microcoded ops. llvm-svn: 117332
2010-10-26 08:02:21 +08:00
FromBBI.ExtraCost = 0;
Two sets of changes. Sorry they are intermingled. 1. Fix pre-ra scheduler so it doesn't try to push instructions above calls to "optimize for latency". Call instructions don't have the right latency and this is more likely to use introduce spills. 2. Fix if-converter cost function. For ARM, it should use instruction latencies, not # of micro-ops since multi-latency instructions is completely executed even when the predicate is false. Also, some instruction will be "slower" when they are predicated due to the register def becoming implicit input. rdar://8598427 llvm-svn: 118135
2010-11-03 08:45:17 +08:00
FromBBI.ExtraCost2 = 0;
Lots of bug fixes. llvm-svn: 37467
2007-06-06 18:16:17 +08:00
Restructure code to reduce ifcvt compile time cost. llvm-svn: 37543
2007-06-12 06:26:22 +08:00
ToBBI.ClobbersPred |= FromBBI.ClobbersPred;
Don't change CFG during analysis stage. Do so during ifcvt and invalidate predecessors accordingly. llvm-svn: 37531
2007-06-09 09:03:43 +08:00
ToBBI.HasFallThrough = FromBBI.HasFallThrough;
Fix some stupid bugs that have effectively disabled if-conversion. llvm-svn: 37575
2007-06-15 04:28:52 +08:00
ToBBI.IsAnalyzed = false;
FromBBI.IsAnalyzed = false;
Initial commit of (very basic) if converter. llvm-svn: 37092
2007-05-16 10:00:57 +08:00
}
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
FunctionPass *
CodeGen/Passes: Pass MachineFunction as functor arg; NFC Passing a MachineFunction as argument is more natural and avoids an unnecessary round-trip through the logic determining the correct Subtarget because MachineFunction already has a reference anyway. llvm-svn: 285039
2016-10-25 07:23:02 +08:00
llvm::createIfConverter(std::function<bool(const MachineFunction &)> Ftor) {
Move instances of std::function. Or replace with llvm::function_ref if it's never stored. NFC intended. llvm-svn: 272513
2016-06-13 00:13:55 +08:00
return new IfConverter(std::move(Ftor));
[ARM] Pass a callback to FunctionPass constructors to enable skipping execution on a per-function basis. Previously some of the passes were conditionally added to ARM's pass pipeline based on the target machine's subtarget. This patch makes changes to add those passes unconditionally and execute them conditonally based on the predicate functor passed to the pass constructors. This enables running different sets of passes for different functions in the module. rdar://problem/20542263 Differential Revision: http://reviews.llvm.org/D8717 llvm-svn: 239325
2015-06-09 02:50:43 +08:00
}