maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp

1021 lines
40 KiB
C++
Raw Normal View History

Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
//===- PassManagerBuilder.cpp - Build Standard Pass -----------------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
// This file defines the PassManagerBuilder class, which is used to set up a
// "standard" optimization sequence suitable for languages like C and C++.
//
//===----------------------------------------------------------------------===//
#include "llvm/Transforms/IPO/PassManagerBuilder.h"
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
#include "llvm-c/Transforms/PassManagerBuilder.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/ADT/SmallVector.h"
[PM/AA] Hoist the interface for BasicAA into a header file. This is the first mechanical step in preparation for making this and all the other alias analysis passes available to the new pass manager. I'm factoring out all the totally boring changes I can so I'm moving code around here with no other changes. I've even minimized the formatting churn. I'll reformat and freshen comments on the interface now that its located in the right place so that the substantive changes don't triger this. llvm-svn: 244197
2015-08-06 15:33:15 +08:00
#include "llvm/Analysis/BasicAliasAnalysis.h"
[CFLAA] Split into Anders+Steens analysis. StratifiedSets (as implemented) is very fast, but its accuracy is also limited. If we take a more aggressive andersens-like approach, we can be way more accurate, but we'll also end up being slower. So, we've decided to split CFLAA into CFLSteensAA and CFLAndersAA. Long-term, we want to end up in a place where CFLSteens is queried first; if it can provide an answer, great (since queries are basically map lookups). Otherwise, we'll fall back to CFLAnders, BasicAA, etc. This patch splits everything out so we can try to do something like that when we get a reasonable CFLAnders implementation. Patch by Jia Chen. Differential Revision: http://reviews.llvm.org/D21910 llvm-svn: 274589
2016-07-06 08:26:41 +08:00
#include "llvm/Analysis/CFLAndersAliasAnalysis.h"
#include "llvm/Analysis/CFLSteensAliasAnalysis.h"
[PM/AA] Extract the interface for GlobalsModRef into a header along with its creation function. This required shifting a bunch of method definitions to be out-of-line so that we could leave most of the implementation guts in the .cpp file. llvm-svn: 245021
2015-08-14 11:48:20 +08:00
#include "llvm/Analysis/GlobalsModRef.h"
Add a new method to create SimpleInliner instance and make pre-inliner use this. This adds a createFunctionInliningPass pass that takes an InlineParams object and use this to create the pre-inliner pass. This prevents the regular inliner's threshold flag from influencing the preinliner. Differential revision: https://reviews.llvm.org/D23377 llvm-svn: 278377
2016-08-12 02:24:08 +08:00
#include "llvm/Analysis/InlineCost.h"
[attrs] Extract the pure inference of function attributes into a standalone pass. There is no call graph or even interesting analysis for this part of function attributes -- it is literally inferring attributes based on the target library identification. As such, we can do it using a much simpler module pass that just walks the declarations. This can also happen much earlier in the pass pipeline which has benefits for any number of other passes. In the process, I've cleaned up one particular aspect of the logic which was necessary in order to separate the two passes cleanly. It now counts inferred attributes independently rather than just counting all the inferred attributes as one, and the counts are more clearly explained. The two test cases we had for this code path are both ... woefully inadequate and copies of each other. I've kept the superset test and updated it. We need more testing here, but I had to pick somewhere to stop fixing everything broken I saw here. Differential Revision: http://reviews.llvm.org/D15676 llvm-svn: 256466
2015-12-27 16:41:34 +08:00
#include "llvm/Analysis/Passes.h"
[PM/AA] Hoist ScopedNoAliasAA's interface into a header and move the creation function there. Same basic refactoring as the other alias analyses. Nothing special required this time around. llvm-svn: 245012
2015-08-14 10:55:50 +08:00
#include "llvm/Analysis/ScopedNoAliasAA.h"
[PM] Move TargetLibraryInfo into the Analysis library. While the term "Target" is in the name, it doesn't really have to do with the LLVM Target library -- this isn't an abstraction which LLVM targets generally need to implement or extend. It has much more to do with modeling the various runtime libraries on different OSes and with different runtime environments. The "target" in this sense is the more general sense of a target of cross compilation. This is in preparation for porting this analysis to the new pass manager. No functionality changed, and updates inbound for Clang and Polly. llvm-svn: 226078
2015-01-15 10:16:27 +08:00
#include "llvm/Analysis/TargetLibraryInfo.h"
[PM/AA] Hoist the interface to TBAA into a dedicated header along with its creation function. Update the relevant includes accordingly. llvm-svn: 245019
2015-08-14 11:33:48 +08:00
#include "llvm/Analysis/TypeBasedAliasAnalysis.h"
[attrs] Extract the pure inference of function attributes into a standalone pass. There is no call graph or even interesting analysis for this part of function attributes -- it is literally inferring attributes based on the target library identification. As such, we can do it using a much simpler module pass that just walks the declarations. This can also happen much earlier in the pass pipeline which has benefits for any number of other passes. In the process, I've cleaned up one particular aspect of the logic which was necessary in order to separate the two passes cleanly. It now counts inferred attributes independently rather than just counting all the inferred attributes as one, and the counts are more clearly explained. The two test cases we had for this code path are both ... woefully inadequate and copies of each other. I've kept the superset test and updated it. We need more testing here, but I had to pick somewhere to stop fixing everything broken I saw here. Differential Revision: http://reviews.llvm.org/D15676 llvm-svn: 256466
2015-12-27 16:41:34 +08:00
#include "llvm/IR/DataLayout.h"
#include "llvm/IR/LegacyPassManager.h"
[ThinLTO] Renaming of function index to module summary index (NFC) (Resubmitting after fixing missing file issue) With the changes in r263275, there are now more than just functions in the summary. Completed the renaming of data structures (started in r263275) to reflect the wider scope. In particular, changed the FunctionIndex* data structures to ModuleIndex*, and renamed related variables and comments. Also renamed the files to reflect the changes. A companion clang patch will immediately succeed this patch to reflect this renaming. llvm-svn: 263513
2016-03-15 08:04:37 +08:00
#include "llvm/IR/ModuleSummaryIndex.h"
[attrs] Extract the pure inference of function attributes into a standalone pass. There is no call graph or even interesting analysis for this part of function attributes -- it is literally inferring attributes based on the target library identification. As such, we can do it using a much simpler module pass that just walks the declarations. This can also happen much earlier in the pass pipeline which has benefits for any number of other passes. In the process, I've cleaned up one particular aspect of the logic which was necessary in order to separate the two passes cleanly. It now counts inferred attributes independently rather than just counting all the inferred attributes as one, and the counts are more clearly explained. The two test cases we had for this code path are both ... woefully inadequate and copies of each other. I've kept the superset test and updated it. We need more testing here, but I had to pick somewhere to stop fixing everything broken I saw here. Differential Revision: http://reviews.llvm.org/D15676 llvm-svn: 256466
2015-12-27 16:41:34 +08:00
#include "llvm/IR/Verifier.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/ManagedStatic.h"
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
#include "llvm/Target/TargetMachine.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-04 00:50:05 +08:00
#include "llvm/Transforms/IPO.h"
[attrs] Split off the forced attributes utility into its own pass that is (by default) run much earlier than FuncitonAttrs proper. This allows forcing optnone or other widely impactful attributes. It is also a bit simpler as the force attribute behavior needs no specific iteration order. I've added the pass into the default module pass pipeline and LTO pass pipeline which mirrors where function attrs itself was being run. Differential Revision: http://reviews.llvm.org/D15668 llvm-svn: 256465
2015-12-27 16:13:45 +08:00
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
[PM] Port the PostOrderFunctionAttrs pass to the new pass manager and convert one test to use this. This is a particularly significant milestone because it required a working per-function AA framework which can be queried over each function from within a CGSCC transform pass (and additionally a module analysis to be accessible). This is essentially *the* point of the entire pass manager rewrite. A CGSCC transform is able to query for multiple different function's analysis results. It works. The whole thing appears to actually work and accomplish the original goal. While we were able to hack function attrs and basic-aa to "work" in the old pass manager, this port doesn't use any of that, it directly leverages the new fundamental functionality. For this to work, the CGSCC framework also has to support SCC-based behavior analysis, etc. The only part of the CGSCC pass infrastructure not sorted out at this point are the updates in the face of inlining and running function passes that mutate the call graph. The changes are pretty boring and boiler-plate. Most of the work was factored into more focused preperatory patches. But this is what wires it all together. llvm-svn: 261203
2016-02-18 19:03:11 +08:00
#include "llvm/Transforms/IPO/FunctionAttrs.h"
[attrs] Extract the pure inference of function attributes into a standalone pass. There is no call graph or even interesting analysis for this part of function attributes -- it is literally inferring attributes based on the target library identification. As such, we can do it using a much simpler module pass that just walks the declarations. This can also happen much earlier in the pass pipeline which has benefits for any number of other passes. In the process, I've cleaned up one particular aspect of the logic which was necessary in order to separate the two passes cleanly. It now counts inferred attributes independently rather than just counting all the inferred attributes as one, and the counts are more clearly explained. The two test cases we had for this code path are both ... woefully inadequate and copies of each other. I've kept the superset test and updated it. We need more testing here, but I had to pick somewhere to stop fixing everything broken I saw here. Differential Revision: http://reviews.llvm.org/D15676 llvm-svn: 256466
2015-12-27 16:41:34 +08:00
#include "llvm/Transforms/IPO/InferFunctionAttrs.h"
[ThinLTO] Renaming of function index to module summary index (NFC) (Resubmitting after fixing missing file issue) With the changes in r263275, there are now more than just functions in the summary. Completed the renaming of data structures (started in r263275) to reflect the wider scope. In particular, changed the FunctionIndex* data structures to ModuleIndex*, and renamed related variables and comments. Also renamed the files to reflect the changes. A companion clang patch will immediately succeed this patch to reflect this renaming. llvm-svn: 263513
2016-03-15 08:04:37 +08:00
#include "llvm/Transforms/Instrumentation.h"
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
#include "llvm/Transforms/Scalar.h"
[PM] Port GVN to the new pass manager, wire it up, and teach a couple of tests to run GVN in both modes. This is mostly the boring refactoring just like SROA and other complex transformation passes. There is some trickiness in that GVN's ValueNumber class requires hand holding to get to compile cleanly. I'm open to suggestions about a better pattern there, but I tried several before settling on this. I was trying to balance my desire to sink as much implementation detail into the source file as possible without introducing overly many layers of abstraction. Much like with SROA, the design of this system is made somewhat more cumbersome by the need to support both pass managers without duplicating the significant state and logic of the pass. The same compromise is struck here. I've also left a FIXME in a doxygen comment as the GVN pass seems to have pretty woeful documentation within it. I'd like to submit this with the FIXME and let those more deeply familiar backfill the information here now that we have a nice place in an interface to put that kind of documentaiton. Differential Revision: http://reviews.llvm.org/D18019 llvm-svn: 263208
2016-03-11 16:50:55 +08:00
#include "llvm/Transforms/Scalar/GVN.h"
[PM/LoopUnswitch] Introduce a new, simpler loop unswitch pass. Currently, this pass only focuses on *trivial* loop unswitching. At that reduced problem it remains significantly better than the current loop unswitch: - Old pass is worse than cubic complexity. New pass is (I think) linear. - New pass is much simpler in its design by focusing on full unswitching. (See below for details on this). - New pass doesn't carry state for thresholds between pass iterations. - New pass doesn't carry state for correctness (both miscompile and infloop) between pass iterations. - New pass produces substantially better code after unswitching. - New pass can handle more trivial unswitch cases. - New pass doesn't recompute the dominator tree for the entire function and instead incrementally updates it. I've ported all of the trivial unswitching test cases from the old pass to the new one to make sure that major functionality isn't lost in the process. For several of the test cases I've worked to improve the precision and rigor of the CHECKs, but for many I've just updated them to handle the new IR produced. My initial motivation was the fact that the old pass carried state in very unreliable ways between pass iterations, and these mechansims were incompatible with the new pass manager. However, I discovered many more improvements to make along the way. This pass makes two very significant assumptions that enable most of these improvements: 1) Focus on *full* unswitching -- that is, completely removing whatever control flow construct is being unswitched from the loop. In the case of trivial unswitching, this means removing the trivial (exiting) edge. In non-trivial unswitching, this means removing the branch or switch itself. This is in opposition to *partial* unswitching where some part of the unswitched control flow remains in the loop. Partial unswitching only really applies to switches and to folded branches. These are very similar to full unrolling and partial unrolling. The full form is an effective canonicalization, the partial form needs a complex cost model, cannot be iterated, isn't canonicalizing, and should be a separate pass that runs very late (much like unrolling). 2) Leverage LLVM's Loop machinery to the fullest. The original unswitch dates from a time when a great deal of LLVM's loop infrastructure was missing, ineffective, and/or unreliable. As a consequence, a lot of complexity was added which we no longer need. With these two overarching principles, I think we can build a fast and effective unswitcher that fits in well in the new PM and in the canonicalization pipeline. Some of the remaining functionality around partial unswitching may not be relevant today (not many test cases or benchmarks I can find) but if they are I'd like to add support for them as a separate layer that runs very late in the pipeline. Purely to make reviewing and introducing this code more manageable, I've split this into first a trivial-unswitch-only pass and in the next patch I'll add support for full non-trivial unswitching against a *fixed* threshold, exactly like full unrolling. I even plan to re-use the unrolling thresholds, as these are incredibly similar cost tradeoffs: we're cloning a loop body in order to end up with simplified control flow. We should only do that when the total growth is reasonably small. One of the biggest changes with this pass compared to the previous one is that previously, each individual trivial exiting edge from a switch was unswitched separately as a branch. Now, we unswitch the entire switch at once, with cases going to the various destinations. This lets us unswitch multiple exiting edges in a single operation and also avoids numerous extremely bad behaviors, where we would introduce 1000s of branches to test for thousands of possible values, all of which would take the exact same exit path bypassing the loop. Now we will use a switch with 1000s of cases that can be efficiently lowered into a jumptable. This avoids relying on somehow forming a switch out of the branches or getting horrible code if that fails for any reason. Another significant change is that this pass actively updates the CFG based on unswitching. For trivial unswitching, this is actually very easy because of the definition of loop simplified form. Doing this makes the code coming out of loop unswitch dramatically more friendly. We still should run loop-simplifycfg (at the least) after this to clean up, but it will have to do a lot less work. Finally, this pass makes much fewer attempts to simplify instructions based on the unswitch. Something like loop-instsimplify, instcombine, or GVN can be used to do increasingly powerful simplifications based on the now dominating predicate. The old simplifications are things that something like loop-instsimplify should get today or a very, very basic loop-instcombine could get. Keeping that logic separate is a big simplifying technique. Most of the code in this pass that isn't in the old one has to do with achieving specific goals: - Updating the dominator tree as we go - Unswitching all cases in a switch in a single step. I think it is still shorter than just the trivial unswitching code in the old pass despite having this functionality. Differential Revision: https://reviews.llvm.org/D32409 llvm-svn: 301576
2017-04-28 02:45:20 +08:00
#include "llvm/Transforms/Scalar/SimpleLoopUnswitch.h"
Add a basic-block autovectorization pass. This is the initial checkin of the basic-block autovectorization pass along with some supporting vectorization infrastructure. Special thanks to everyone who helped review this code over the last several months (especially Tobias Grosser). llvm-svn: 149468
2012-02-01 11:51:43 +08:00
#include "llvm/Transforms/Vectorize.h"
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
using namespace llvm;
Add a basic-block autovectorization pass. This is the initial checkin of the basic-block autovectorization pass along with some supporting vectorization infrastructure. Special thanks to everyone who helped review this code over the last several months (especially Tobias Grosser). llvm-svn: 149468
2012-02-01 11:51:43 +08:00
static cl::opt<bool>
[PartialInlining] Add internal options to enable partial inlining in pass pipeline (off by default) 1. Legacy: -mllvm -enable-partial-inlining 2. New: -mllvm -enable-npm-partial-inlining -fexperimental-new-pass-manager Differential Revision: http://reviews.llvm.org/D33382 llvm-svn: 303567
2017-05-23 00:41:57 +08:00
RunPartialInlining("enable-partial-inlining", cl::init(false), cl::Hidden,
cl::ZeroOrMore, cl::desc("Run Partial inlinining pass"));
static cl::opt<bool>
RunLoopVectorization("vectorize-loops", cl::Hidden,
cl::desc("Run the Loop vectorization passes"));
Change the PassManagerBuilder (used by -O3) loop vectorizer flag from -vectorize to -vectorize-loops because we dont want to share the same flag as the bb-vectorizer. llvm-svn: 166937
2012-10-30 00:36:25 +08:00
static cl::opt<bool>
Mark some command line flags as hidden llvm-svn: 193013
2013-10-19 07:38:13 +08:00
RunSLPVectorization("vectorize-slp", cl::Hidden,
Add an option -vectorize-slp-aggressive for running the BB vectorizer. Make -fslp-vectorize run the slp-vectorizer. llvm-svn: 179508
2013-04-15 13:39:58 +08:00
cl::desc("Run the SLP vectorization passes"));
static cl::opt<bool>
Mark some command line flags as hidden llvm-svn: 193013
2013-10-19 07:38:13 +08:00
RunBBVectorization("vectorize-slp-aggressive", cl::Hidden,
Add an option -vectorize-slp-aggressive for running the BB vectorizer. Make -fslp-vectorize run the slp-vectorizer. llvm-svn: 179508
2013-04-15 13:39:58 +08:00
cl::desc("Run the BB vectorization passes"));
Add a basic-block autovectorization pass. This is the initial checkin of the basic-block autovectorization pass along with some supporting vectorization infrastructure. Special thanks to everyone who helped review this code over the last several months (especially Tobias Grosser). llvm-svn: 149468
2012-02-01 11:51:43 +08:00
By default, use Early-CSE instead of GVN for vectorization cleanup. As has been suggested by Duncan and others, Early-CSE and GVN should do similar redundancy elimination, but Early-CSE is much less expensive. Most of my autovectorization benchmarks show a performance regresion, but all of these are < 0.1%, and so I think that it is still worth using the less expensive pass. llvm-svn: 154673
2012-04-14 01:15:33 +08:00
static cl::opt<bool>
UseGVNAfterVectorization("use-gvn-after-vectorization",
cl::init(false), cl::Hidden,
cl::desc("Run GVN instead of Early CSE after vectorization passes"));
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
static cl::opt<bool> ExtraVectorizerPasses(
"extra-vectorizer-passes", cl::init(false), cl::Hidden,
cl::desc("Run cleanup optimization passes after vectorization."));
Add a loop rerolling pass This adds a loop rerolling pass: the opposite of (partial) loop unrolling. The transformation aims to take loops like this: for (int i = 0; i < 3200; i += 5) { a[i] += alpha * b[i]; a[i + 1] += alpha * b[i + 1]; a[i + 2] += alpha * b[i + 2]; a[i + 3] += alpha * b[i + 3]; a[i + 4] += alpha * b[i + 4]; } and turn them into this: for (int i = 0; i < 3200; ++i) { a[i] += alpha * b[i]; } and loops like this: for (int i = 0; i < 500; ++i) { x[3*i] = foo(0); x[3*i+1] = foo(0); x[3*i+2] = foo(0); } and turn them into this: for (int i = 0; i < 1500; ++i) { x[i] = foo(0); } There are two motivations for this transformation: 1. Code-size reduction (especially relevant, obviously, when compiling for code size). 2. Providing greater choice to the loop vectorizer (and generic unroller) to choose the unrolling factor (and a better ability to vectorize). The loop vectorizer can take vector lengths and register pressure into account when choosing an unrolling factor, for example, and a pre-unrolled loop limits that choice. This is especially problematic if the manual unrolling was optimized for a machine different from the current target. The current implementation is limited to single basic-block loops only. The rerolling recognition should work regardless of how the loop iterations are intermixed within the loop body (subject to dependency and side-effect constraints), but the significant restriction is that the order of the instructions in each iteration must be identical. This seems sufficient to capture all current use cases. This pass is not currently enabled by default at any optimization level. llvm-svn: 194939
2013-11-17 07:59:05 +08:00
static cl::opt<bool>
RunLoopRerolling("reroll-loops", cl::Hidden,
cl::desc("Run the loop rerolling pass"));
[NewGVN] Add a flag to enable the pass via `-mllvm`. NewGVN can be tested passing `-mllvm -enable-newgvn` to clang. Differential Revision: https://reviews.llvm.org/D28059 llvm-svn: 290548
2016-12-27 02:26:19 +08:00
static cl::opt<bool> RunNewGVN("enable-newgvn", cl::init(false), cl::Hidden,
cl::desc("Run the NewGVN pass"));
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
static cl::opt<bool>
RunSLPAfterLoopVectorization("run-slp-after-loop-vectorization",
Enable noalias metadata by default and swap the order of the SLP and Loop vectorizers by default. After some time maturing, hopefully the flags themselves will be removed. llvm-svn: 217144
2014-09-04 21:23:08 +08:00
cl::init(true), cl::Hidden,
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
cl::desc("Run the SLP vectorizer (and BB vectorizer) after the Loop "
"vectorizer instead of before"));
[CFLAA] Split into Anders+Steens analysis. StratifiedSets (as implemented) is very fast, but its accuracy is also limited. If we take a more aggressive andersens-like approach, we can be way more accurate, but we'll also end up being slower. So, we've decided to split CFLAA into CFLSteensAA and CFLAndersAA. Long-term, we want to end up in a place where CFLSteens is queried first; if it can provide an answer, great (since queries are basically map lookups). Otherwise, we'll fall back to CFLAnders, BasicAA, etc. This patch splits everything out so we can try to do something like that when we get a reasonable CFLAnders implementation. Patch by Jia Chen. Differential Revision: http://reviews.llvm.org/D21910 llvm-svn: 274589
2016-07-06 08:26:41 +08:00
// Experimental option to use CFL-AA
enum class CFLAAType { None, Steensgaard, Andersen, Both };
static cl::opt<CFLAAType>
UseCFLAA("use-cfl-aa", cl::init(CFLAAType::None), cl::Hidden,
cl::desc("Enable the new, experimental CFL alias analysis"),
cl::values(clEnumValN(CFLAAType::None, "none", "Disable CFL-AA"),
clEnumValN(CFLAAType::Steensgaard, "steens",
"Enable unification-based CFL-AA"),
clEnumValN(CFLAAType::Andersen, "anders",
"Enable inclusion-based CFL-AA"),
clEnumValN(CFLAAType::Both, "both",
Turn cl::values() (for enum) from a vararg function to using C++ variadic template The core of the change is supposed to be NFC, however it also fixes what I believe was an undefined behavior when calling: va_start(ValueArgs, Desc); with Desc being a StringRef. Differential Revision: https://reviews.llvm.org/D25342 llvm-svn: 283671
2016-10-09 03:41:06 +08:00
"Enable both variants of CFL-AA")));
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
Add a new pass "Loop Interchange" This pass interchanges loops to provide a more cache-friendly memory access. For e.g. given a loop like - for(int i=0;i<N;i++) for(int j=0;j<N;j++) A[j][i] = A[j][i]+B[j][i]; is interchanged to - for(int j=0;j<N;j++) for(int i=0;i<N;i++) A[j][i] = A[j][i]+B[j][i]; This pass is currently disabled by default. To give a brief introduction it consists of 3 stages- LoopInterchangeLegality : Checks the legality of loop interchange based on Dependency matrix. LoopInterchangeProfitability: A very basic heuristic has been added to check for profitibility. This will evolve over time. LoopInterchangeTransform : Which does the actual transform. LNT Performance tests shows improvement in Polybench/linear-algebra/kernels/mvt and Polybench/linear-algebra/kernels/gemver becnmarks. TODO: 1) Add support for reductions and lcssa phi. 2) Improve profitability model. 3) Improve loop selection algorithm to select best loop for interchange. Currently the innermost loop is selected for interchange. 4) Improve compile time regression found in llvm lnt due to this pass. 5) Fix issues in Dependency Analysis module. A special thanks to Hal for reviewing this code. Review: http://reviews.llvm.org/D7499 llvm-svn: 231458
2015-03-06 18:11:25 +08:00
static cl::opt<bool> EnableLoopInterchange(
"enable-loopinterchange", cl::init(false), cl::Hidden,
cl::desc("Enable the new, experimental LoopInterchange Pass"));
[GMR] Add a flag to enable GlobalsModRef in the normal compilation pipeline. Even before I started improving its runtime, it was already crazy fast once the call graph exists, and if we can get it to be conservatively correct, will still likely catch a lot of interesting and useful cases. So it may well be useful to enable by default. But more importantly for me, this should make it easier for me to test that changes aren't breaking it in fundamental ways by enabling it for normal builds. llvm-svn: 242895
2015-07-22 19:57:28 +08:00
static cl::opt<bool> EnableNonLTOGlobalsModRef(
[GlobalsAA] Turn GlobalsAA on again by default Now that all the known faults with GlobalsAA have been fixed, flip the big switch on -enable-non-lto-gmr again. Feel free to pester me with any more bugs found, and don't hesitate to flip the switch back off. llvm-svn: 250157
2015-10-13 18:43:57 +08:00
"enable-non-lto-gmr", cl::init(true), cl::Hidden,
[GMR] Add a flag to enable GlobalsModRef in the normal compilation pipeline. Even before I started improving its runtime, it was already crazy fast once the call graph exists, and if we can get it to be conservatively correct, will still likely catch a lot of interesting and useful cases. So it may well be useful to enable by default. But more importantly for me, this should make it easier for me to test that changes aren't breaking it in fundamental ways by enabling it for normal builds. llvm-svn: 242895
2015-07-22 19:57:28 +08:00
cl::desc(
"Enable the GlobalsModRef AliasAnalysis outside of the LTO pipeline."));
LLE 6/6: Add LoopLoadElimination pass Summary: The goal of this pass is to perform store-to-load forwarding across the backedge of a loop. E.g.: for (i) A[i + 1] = A[i] + B[i] => T = A[0] for (i) T = T + B[i] A[i + 1] = T The pass relies on loop dependence analysis via LoopAccessAnalisys to find opportunities of loop-carried dependences with a distance of one between a store and a load. Since it's using LoopAccessAnalysis, it was easy to also add support for versioning away may-aliasing intervening stores that would otherwise prevent this transformation. This optimization is also performed by Load-PRE in GVN without the option of multi-versioning. As was discussed with Daniel Berlin in http://reviews.llvm.org/D9548, this is inferior to a more loop-aware solution applied here. Hopefully, we will be able to remove some complexity from GVN/MemorySSA as a consequence. In the long run, we may want to extend this pass (or create a new one if there is little overlap) to also eliminate loop-indepedent redundant loads and store that *require* versioning due to may-aliasing intervening stores/loads. I have some motivating cases for store elimination. My plan right now is to wait for MemorySSA to come online first rather than using memdep for this. The main motiviation for this pass is the 456.hmmer loop in SPECint2006 where after distributing the original loop and vectorizing the top part, we are left with the critical path exposed in the bottom loop. Being able to promote the memory dependence into a register depedence (even though the HW does perform store-to-load fowarding as well) results in a major gain (~20%). This gain also transfers over to x86: it's around 8-10%. Right now the pass is off by default and can be enabled with -enable-loop-load-elim. On the LNT testsuite, there are two performance changes (negative number -> improvement): 1. -28% in Polybench/linear-algebra/solvers/dynprog: the length of the critical paths is reduced 2. +2% in Polybench/stencils/adi: Unfortunately, I couldn't reproduce this outside of LNT The pass is scheduled after the loop vectorizer (which is after loop distribution). The rational is to try to reuse LAA state, rather than recomputing it. The order between LV and LLE is not critical because normally LV does not touch scalar st->ld forwarding cases where vectorizing would inhibit the CPU's st->ld forwarding to kick in. LoopLoadElimination requires LAA to provide the full set of dependences (including forward dependences). LAA is known to omit loop-independent dependences in certain situations. The big comment before removeDependencesFromMultipleStores explains why this should not occur for the cases that we're interested in. Reviewers: dberlin, hfinkel Subscribers: junbuml, dberlin, mssimpso, rengolin, sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D13259 llvm-svn: 252017
2015-11-04 07:50:08 +08:00
static cl::opt<bool> EnableLoopLoadElim(
Turn LoopLoadElimination on again The latent bug that LLE exposed in the LoopVectorizer was resolved (PR26952). The pass can be disabled with -mllvm -enable-loop-load-elim=0 llvm-svn: 263595
2016-03-16 06:26:12 +08:00
"enable-loop-load-elim", cl::init(true), cl::Hidden,
cl::desc("Enable the LoopLoadElimination Pass"));
LLE 6/6: Add LoopLoadElimination pass Summary: The goal of this pass is to perform store-to-load forwarding across the backedge of a loop. E.g.: for (i) A[i + 1] = A[i] + B[i] => T = A[0] for (i) T = T + B[i] A[i + 1] = T The pass relies on loop dependence analysis via LoopAccessAnalisys to find opportunities of loop-carried dependences with a distance of one between a store and a load. Since it's using LoopAccessAnalysis, it was easy to also add support for versioning away may-aliasing intervening stores that would otherwise prevent this transformation. This optimization is also performed by Load-PRE in GVN without the option of multi-versioning. As was discussed with Daniel Berlin in http://reviews.llvm.org/D9548, this is inferior to a more loop-aware solution applied here. Hopefully, we will be able to remove some complexity from GVN/MemorySSA as a consequence. In the long run, we may want to extend this pass (or create a new one if there is little overlap) to also eliminate loop-indepedent redundant loads and store that *require* versioning due to may-aliasing intervening stores/loads. I have some motivating cases for store elimination. My plan right now is to wait for MemorySSA to come online first rather than using memdep for this. The main motiviation for this pass is the 456.hmmer loop in SPECint2006 where after distributing the original loop and vectorizing the top part, we are left with the critical path exposed in the bottom loop. Being able to promote the memory dependence into a register depedence (even though the HW does perform store-to-load fowarding as well) results in a major gain (~20%). This gain also transfers over to x86: it's around 8-10%. Right now the pass is off by default and can be enabled with -enable-loop-load-elim. On the LNT testsuite, there are two performance changes (negative number -> improvement): 1. -28% in Polybench/linear-algebra/solvers/dynprog: the length of the critical paths is reduced 2. +2% in Polybench/stencils/adi: Unfortunately, I couldn't reproduce this outside of LNT The pass is scheduled after the loop vectorizer (which is after loop distribution). The rational is to try to reuse LAA state, rather than recomputing it. The order between LV and LLE is not critical because normally LV does not touch scalar st->ld forwarding cases where vectorizing would inhibit the CPU's st->ld forwarding to kick in. LoopLoadElimination requires LAA to provide the full set of dependences (including forward dependences). LAA is known to omit loop-independent dependences in certain situations. The big comment before removeDependencesFromMultipleStores explains why this should not occur for the cases that we're interested in. Reviewers: dberlin, hfinkel Subscribers: junbuml, dberlin, mssimpso, rengolin, sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D13259 llvm-svn: 252017
2015-11-04 07:50:08 +08:00
[ThinLTO] Ensure anonymous globals renamed even at -O0 Summary: This fixes an issue when files are compiled with -flto=thin at default -O0. We need to rename anonymous globals before attempting to write the module summary because all values need names for the summary. This was happening at -O1 and above, but not before the early exit when constructing the pipeline for -O0. Also add an internal -prepare-for-thinlto option to enable this to be tested via opt. Fixes PR30419. Reviewers: mehdi_amini Subscribers: probinson, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D24701 llvm-svn: 281840
2016-09-18 04:40:16 +08:00
static cl::opt<bool>
EnablePrepareForThinLTO("prepare-for-thinlto", cl::init(false), cl::Hidden,
cl::desc("Enable preparation for ThinLTO."));
[Profile] Use explicit flag to enable IR PGO Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22607 llvm-svn: 276516
2016-07-23 12:28:52 +08:00
static cl::opt<bool> RunPGOInstrGen(
"profile-generate", cl::init(false), cl::Hidden,
cl::desc("Enable PGO instrumentation."));
static cl::opt<std::string>
PGOOutputFile("profile-generate-file", cl::init(""), cl::Hidden,
cl::desc("Specify the path of profile data file."));
[PGO] Passmanagerbuilder change that enable IR level PGO instrumentation This patch includes the passmanagerbuilder change that enables IR level PGO instrumentation. It adds two passmanagerbuilder options: -profile-generate=<profile_filename> and -profile-use=<profile_filename>. The new options are primarily for debug purpose. Reviewers: davidxl, silvas Differential Revision: http://reviews.llvm.org/D15828 llvm-svn: 258420
2016-01-22 02:28:59 +08:00
static cl::opt<std::string> RunPGOInstrUse(
"profile-use", cl::init(""), cl::Hidden, cl::value_desc("filename"),
cl::desc("Enable use phase of PGO instrumentation and specify the path "
"of profile data file"));
New Loop Versioning LICM Pass Summary: When alias analysis is uncertain about the aliasing between any two accesses, it will return MayAlias. This uncertainty from alias analysis restricts LICM from proceeding further. In cases where alias analysis is uncertain we might use loop versioning as an alternative. Loop Versioning will create a version of the loop with aggressive aliasing assumptions in addition to the original with conservative (default) aliasing assumptions. The version of the loop making aggressive aliasing assumptions will have all the memory accesses marked as no-alias. These two versions of loop will be preceded by a memory runtime check. This runtime check consists of bound checks for all unique memory accessed in loop, and it ensures the lack of memory aliasing. The result of the runtime check determines which of the loop versions is executed: If the runtime check detects any memory aliasing, then the original loop is executed. Otherwise, the version with aggressive aliasing assumptions is used. The pass is off by default and can be enabled with command line option -enable-loop-versioning-licm. Reviewers: hfinkel, anemet, chatur01, reames Subscribers: MatzeB, grosser, joker.eph, sanjoy, javed.absar, sbaranga, llvm-commits Differential Revision: http://reviews.llvm.org/D9151 llvm-svn: 259986
2016-02-06 15:47:48 +08:00
static cl::opt<bool> UseLoopVersioningLICM(
"enable-loop-versioning-licm", cl::init(false), cl::Hidden,
cl::desc("Enable the experimental Loop Versioning LICM pass"));
[PGO] IRPGO pre-cleanup pass changes This patch adds a selected set of cleanup passes including a pre-inline pass before LLVM IR PGO instrumentation. The inline is only intended to apply those obvious/trivial ones before instrumentation so that much less instrumentation is needed to get better profiling information. This will drastically improve the instrumented code performance for large C++ applications. Another benefit is the context sensitive counts that can potentially improve the PGO optimization. Differential Revision: http://reviews.llvm.org/D21405 llvm-svn: 275588
2016-07-16 02:10:49 +08:00
static cl::opt<bool>
DisablePreInliner("disable-preinline", cl::init(false), cl::Hidden,
cl::desc("Disable pre-instrumentation inliner"));
static cl::opt<int> PreInlineThreshold(
"preinline-threshold", cl::Hidden, cl::init(75), cl::ZeroOrMore,
cl::desc("Control the amount of inlining in pre-instrumentation inliner "
"(default = 75)"));
[EarlyCSE] Add option to use MemorySSA for function simplification run of EarlyCSE (off by default). Summary: Use MemorySSA for memory dependency checking in the EarlyCSE pass at the start of the function simplification portion of the pipeline. We rely on the fact that GVNHoist runs just after this pass of EarlyCSE to amortize the MemorySSA construction cost since GVNHoist uses MemorySSA and EarlyCSE preserves it. This is turned off by default. A follow-up change will turn it on to allow for easier reversion in case it breaks something. llvm-svn: 305146
2017-06-10 23:20:03 +08:00
static cl::opt<bool> EnableEarlyCSEMemSSA(
"enable-earlycse-memssa", cl::init(false), cl::Hidden,
cl::desc("Enable the EarlyCSE w/ MemorySSA pass (default = off)"));
Add flag to PassManagerBuilder to disable GVN Hoist Pass. Summary: Adding a flag to diable GVN Hoisting by default. Note: The GVN Hoist Pass causes some Halide tests to hang. Halide will disable the pass while investigating. Reviewers: llvm-commits, chandlerc, spop, dberlin Subscribers: mehdi_amini Differential Revision: https://reviews.llvm.org/D22639 llvm-svn: 276479
2016-07-23 06:02:19 +08:00
static cl::opt<bool> EnableGVNHoist(
Disable GVN Hoist due to still more bugs being found in it. There is also a discussion about exactly what we should do prior to re-enabling it. The current bug is http://llvm.org/PR32821 and the discussion about this is in the review thread for r300200. llvm-svn: 301505
2017-04-27 08:28:03 +08:00
"enable-gvn-hoist", cl::init(false), cl::Hidden,
cl::desc("Enable the GVN hoisting pass (default = off)"));
Add flag to PassManagerBuilder to disable GVN Hoist Pass. Summary: Adding a flag to diable GVN Hoisting by default. Note: The GVN Hoist Pass causes some Halide tests to hang. Halide will disable the pass while investigating. Reviewers: llvm-commits, chandlerc, spop, dberlin Subscribers: mehdi_amini Differential Revision: https://reviews.llvm.org/D22639 llvm-svn: 276479
2016-07-23 06:02:19 +08:00
Conditionally eliminate library calls where the result value is not used Summary: This pass shrink-wraps a condition to some library calls where the call result is not used. For example: sqrt(val); is transformed to if (val < 0) sqrt(val); Even if the result of library call is not being used, the compiler cannot safely delete the call because the function can set errno on error conditions. Note in many functions, the error condition solely depends on the incoming parameter. In this optimization, we can generate the condition can lead to the errno to shrink-wrap the call. Since the chances of hitting the error condition is low, the runtime call is effectively eliminated. These partially dead calls are usually results of C++ abstraction penalty exposed by inlining. This optimization hits 108 times in 19 C/C++ programs in SPEC2006. Reviewers: hfinkel, mehdi_amini, davidxl Subscribers: modocache, mgorny, mehdi_amini, xur, llvm-commits, beanz Differential Revision: https://reviews.llvm.org/D24414 llvm-svn: 284542
2016-10-19 05:36:27 +08:00
static cl::opt<bool>
DisableLibCallsShrinkWrap("disable-libcalls-shrinkwrap", cl::init(false),
cl::Hidden,
cl::desc("Disable shrink-wrap library calls"));
[PM/LoopUnswitch] Introduce a new, simpler loop unswitch pass. Currently, this pass only focuses on *trivial* loop unswitching. At that reduced problem it remains significantly better than the current loop unswitch: - Old pass is worse than cubic complexity. New pass is (I think) linear. - New pass is much simpler in its design by focusing on full unswitching. (See below for details on this). - New pass doesn't carry state for thresholds between pass iterations. - New pass doesn't carry state for correctness (both miscompile and infloop) between pass iterations. - New pass produces substantially better code after unswitching. - New pass can handle more trivial unswitch cases. - New pass doesn't recompute the dominator tree for the entire function and instead incrementally updates it. I've ported all of the trivial unswitching test cases from the old pass to the new one to make sure that major functionality isn't lost in the process. For several of the test cases I've worked to improve the precision and rigor of the CHECKs, but for many I've just updated them to handle the new IR produced. My initial motivation was the fact that the old pass carried state in very unreliable ways between pass iterations, and these mechansims were incompatible with the new pass manager. However, I discovered many more improvements to make along the way. This pass makes two very significant assumptions that enable most of these improvements: 1) Focus on *full* unswitching -- that is, completely removing whatever control flow construct is being unswitched from the loop. In the case of trivial unswitching, this means removing the trivial (exiting) edge. In non-trivial unswitching, this means removing the branch or switch itself. This is in opposition to *partial* unswitching where some part of the unswitched control flow remains in the loop. Partial unswitching only really applies to switches and to folded branches. These are very similar to full unrolling and partial unrolling. The full form is an effective canonicalization, the partial form needs a complex cost model, cannot be iterated, isn't canonicalizing, and should be a separate pass that runs very late (much like unrolling). 2) Leverage LLVM's Loop machinery to the fullest. The original unswitch dates from a time when a great deal of LLVM's loop infrastructure was missing, ineffective, and/or unreliable. As a consequence, a lot of complexity was added which we no longer need. With these two overarching principles, I think we can build a fast and effective unswitcher that fits in well in the new PM and in the canonicalization pipeline. Some of the remaining functionality around partial unswitching may not be relevant today (not many test cases or benchmarks I can find) but if they are I'd like to add support for them as a separate layer that runs very late in the pipeline. Purely to make reviewing and introducing this code more manageable, I've split this into first a trivial-unswitch-only pass and in the next patch I'll add support for full non-trivial unswitching against a *fixed* threshold, exactly like full unrolling. I even plan to re-use the unrolling thresholds, as these are incredibly similar cost tradeoffs: we're cloning a loop body in order to end up with simplified control flow. We should only do that when the total growth is reasonably small. One of the biggest changes with this pass compared to the previous one is that previously, each individual trivial exiting edge from a switch was unswitched separately as a branch. Now, we unswitch the entire switch at once, with cases going to the various destinations. This lets us unswitch multiple exiting edges in a single operation and also avoids numerous extremely bad behaviors, where we would introduce 1000s of branches to test for thousands of possible values, all of which would take the exact same exit path bypassing the loop. Now we will use a switch with 1000s of cases that can be efficiently lowered into a jumptable. This avoids relying on somehow forming a switch out of the branches or getting horrible code if that fails for any reason. Another significant change is that this pass actively updates the CFG based on unswitching. For trivial unswitching, this is actually very easy because of the definition of loop simplified form. Doing this makes the code coming out of loop unswitch dramatically more friendly. We still should run loop-simplifycfg (at the least) after this to clean up, but it will have to do a lot less work. Finally, this pass makes much fewer attempts to simplify instructions based on the unswitch. Something like loop-instsimplify, instcombine, or GVN can be used to do increasingly powerful simplifications based on the now dominating predicate. The old simplifications are things that something like loop-instsimplify should get today or a very, very basic loop-instcombine could get. Keeping that logic separate is a big simplifying technique. Most of the code in this pass that isn't in the old one has to do with achieving specific goals: - Updating the dominator tree as we go - Unswitching all cases in a switch in a single step. I think it is still shorter than just the trivial unswitching code in the old pass despite having this functionality. Differential Revision: https://reviews.llvm.org/D32409 llvm-svn: 301576
2017-04-28 02:45:20 +08:00
static cl::opt<bool>
EnableSimpleLoopUnswitch("enable-simple-loop-unswitch", cl::init(false),
cl::Hidden,
cl::desc("Enable the simple loop unswitch pass."));
[GVNSink] GVNSink pass This patch provides an initial prototype for a pass that sinks instructions based on GVN information, similar to GVNHoist. It is not yet ready for commiting but I've uploaded it to gather some initial thoughts. This pass attempts to sink instructions into successors, reducing static instruction count and enabling if-conversion. We use a variant of global value numbering to decide what can be sunk. Consider: [ %a1 = add i32 %b, 1 ] [ %c1 = add i32 %d, 1 ] [ %a2 = xor i32 %a1, 1 ] [ %c2 = xor i32 %c1, 1 ] \ / [ %e = phi i32 %a2, %c2 ] [ add i32 %e, 4 ] GVN would number %a1 and %c1 differently because they compute different results - the VN of an instruction is a function of its opcode and the transitive closure of its operands. This is the key property for hoisting and CSE. What we want when sinking however is for a numbering that is a function of the *uses* of an instruction, which allows us to answer the question "if I replace %a1 with %c1, will it contribute in an equivalent way to all successive instructions?". The (new) PostValueTable class in GVN provides this mapping. This pass has some shown really impressive improvements especially for codesize already on internal benchmarks, so I have high hopes it can replace all the sinking logic in SimplifyCFG. Differential revision: https://reviews.llvm.org/D24805 llvm-svn: 303850
2017-05-25 20:51:11 +08:00
static cl::opt<bool> EnableGVNSink(
"enable-gvn-sink", cl::init(false), cl::Hidden,
[PM] GVNSink is off by default, fix an obvious typo. llvm-svn: 304497
2017-06-02 07:47:53 +08:00
cl::desc("Enable the GVN sinking pass (default = off)"));
[GVNSink] GVNSink pass This patch provides an initial prototype for a pass that sinks instructions based on GVN information, similar to GVNHoist. It is not yet ready for commiting but I've uploaded it to gather some initial thoughts. This pass attempts to sink instructions into successors, reducing static instruction count and enabling if-conversion. We use a variant of global value numbering to decide what can be sunk. Consider: [ %a1 = add i32 %b, 1 ] [ %c1 = add i32 %d, 1 ] [ %a2 = xor i32 %a1, 1 ] [ %c2 = xor i32 %c1, 1 ] \ / [ %e = phi i32 %a2, %c2 ] [ add i32 %e, 4 ] GVN would number %a1 and %c1 differently because they compute different results - the VN of an instruction is a function of its opcode and the transitive closure of its operands. This is the key property for hoisting and CSE. What we want when sinking however is for a numbering that is a function of the *uses* of an instruction, which allows us to answer the question "if I replace %a1 with %c1, will it contribute in an equivalent way to all successive instructions?". The (new) PostValueTable class in GVN provides this mapping. This pass has some shown really impressive improvements especially for codesize already on internal benchmarks, so I have high hopes it can replace all the sinking logic in SimplifyCFG. Differential revision: https://reviews.llvm.org/D24805 llvm-svn: 303850
2017-05-25 20:51:11 +08:00
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
PassManagerBuilder::PassManagerBuilder() {
OptLevel = 2;
SizeLevel = 0;
[C++] Use 'nullptr'. Transforms edition. llvm-svn: 207196
2014-04-25 13:29:35 +08:00
LibraryInfo = nullptr;
Inliner = nullptr;
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
DisableUnitAtATime = false;
DisableUnrollLoops = false;
Add an option -vectorize-slp-aggressive for running the BB vectorizer. Make -fslp-vectorize run the slp-vectorizer. llvm-svn: 179508
2013-04-15 13:39:58 +08:00
BBVectorize = RunBBVectorization;
Rename the slp-vectorizer clang/llvm flags. No functionality change. llvm-svn: 179505
2013-04-15 12:54:42 +08:00
SLPVectorize = RunSLPVectorization;
Change the PassManagerBuilder (used by -O3) loop vectorizer flag from -vectorize to -vectorize-loops because we dont want to share the same flag as the bb-vectorizer. llvm-svn: 166937
2012-10-30 00:36:25 +08:00
LoopVectorize = RunLoopVectorization;
Add a loop rerolling flag to the PassManagerBuilder This adds a boolean member variable to the PassManagerBuilder to control loop rerolling (just like we have for unrolling and the various vectorization options). This is necessary for control by the frontend. Loop rerolling remains disabled by default at all optimization levels. llvm-svn: 194966
2013-11-18 00:02:50 +08:00
RerollLoops = RunLoopRerolling;
[NewGVN] Add a flag to enable the pass via `-mllvm`. NewGVN can be tested passing `-mllvm -enable-newgvn` to clang. Differential Revision: https://reviews.llvm.org/D28059 llvm-svn: 290548
2016-12-27 02:26:19 +08:00
NewGVN = RunNewGVN;
Move DisableGVNLoadPRE from populateLTOPassManager to PassManagerBuilder. llvm-svn: 216174
2014-08-21 21:13:17 +08:00
DisableGVNLoadPRE = false;
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
VerifyInput = false;
VerifyOutput = false;
Add control of function merging to the PMBuilder. llvm-svn: 217731
2014-09-14 05:46:00 +08:00
MergeFunctions = false;
Resubmit "Add new EliminateAvailableExternally module pass" (r239480) This change includes a fix for https://code.google.com/p/chromium/issues/detail?id=499508#c3, which required updating the visibility for symbols with eliminated definitions. --Original Commit Message-- Add new EliminateAvailableExternally module pass, which is performed in O2 compiles just before GlobalDCE, unless we are preparing for LTO. This pass eliminates available externally globals (turning them into declarations), regardless of whether they are dead/unreferenced, since we are guaranteed to have a copy available elsewhere at link time. This enables additional opportunities for GlobalDCE. If we are preparing for LTO (e.g. a -flto -c compile), the pass is not included as we want to preserve available externally functions for possible link time inlining. The FE indicates whether we are doing an -flto compile via the new PrepareForLTO flag on the PassManagerBuilder. llvm-svn: 241466
2015-07-07 00:22:42 +08:00
PrepareForLTO = false;
[Profile] Use explicit flag to enable IR PGO Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22607 llvm-svn: 276516
2016-07-23 12:28:52 +08:00
EnablePGOInstrGen = RunPGOInstrGen;
PGOInstrGen = PGOOutputFile;
[PGO] Passmanagerbuilder change that enable IR level PGO instrumentation This patch includes the passmanagerbuilder change that enables IR level PGO instrumentation. It adds two passmanagerbuilder options: -profile-generate=<profile_filename> and -profile-use=<profile_filename>. The new options are primarily for debug purpose. Reviewers: davidxl, silvas Differential Revision: http://reviews.llvm.org/D15828 llvm-svn: 258420
2016-01-22 02:28:59 +08:00
PGOInstrUse = RunPGOInstrUse;
[ThinLTO] Ensure anonymous globals renamed even at -O0 Summary: This fixes an issue when files are compiled with -flto=thin at default -O0. We need to rename anonymous globals before attempting to write the module summary because all values need names for the summary. This was happening at -O1 and above, but not before the early exit when constructing the pipeline for -O0. Also add an internal -prepare-for-thinlto option to enable this to be tested via opt. Fixes PR30419. Reviewers: mehdi_amini Subscribers: probinson, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D24701 llvm-svn: 281840
2016-09-18 04:40:16 +08:00
PrepareForThinLTO = EnablePrepareForThinLTO;
Define the ThinLTO Pipeline (experimental) Summary: On the contrary to Full LTO, ThinLTO can afford to shift compile time from the frontend to the linker: both phases are parallel (even if it is not totally "free": projects like clang are reusing product from the "compile phase" for multiple link, think about libLLVMSupport reused for opt, llc, etc.). This pipeline is based on the proposal in D13443 for full LTO. We didn't move forward on this proposal because the LTO link was far too long after that. We believe that we can afford it with ThinLTO. The ThinLTO pipeline integrates in the regular O2/O3 flow: - The compile phase perform the inliner with a somehow lighter function simplification. (TODO: tune the inliner thresholds here) This is intendend to simplify the IR and get rid of obvious things like linkonce_odr that will be inlined. - The link phase will run the pipeline from the start, extended with some specific passes that leverage the augmented knowledge we have during LTO. Especially after the inliner is done, a sequence of globalDCE/globalOpt is performed, followed by another run of the "function simplification" passes. It is not clear if this part of the pipeline will stay as is, as the split model of ThinLTO does not allow the same benefit as FullLTO without added tricks. The measurements on the public test suite as well as on our internal suite show an overall net improvement. The binary size for the clang executable is reduced by 5%. We're still tuning it with the bringup of ThinLTO and it will evolve, but this should provide a good starting point. Reviewers: tejohnson Differential Revision: http://reviews.llvm.org/D17115 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261029
2016-02-17 07:02:29 +08:00
PerformThinLTO = false;
Only unswitch loops with uniform conditions Loop unswitching can be extremely harmful for a SIMT target. In case if hoisted condition is not uniform a SIMT machine will execute both clones of a loop sequentially. Therefor LoopUnswitch checks if the condition is non-divergent. Since DivergenceAnalysis adds an expensive PostDominatorTree analysis not needed for non-SIMT targets a new option is added to avoid unneded analysis initialization. The method getAnalysisUsage is called when TargetTransformInfo is not yet available and we cannot use it here. For that reason a new field DivergentTarget is added to PassManagerBuilder to control the behavior and set this field from a target. Differential Revision: https://reviews.llvm.org/D30796 llvm-svn: 298104
2017-03-18 01:13:41 +08:00
DivergentTarget = false;
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
}
PassManagerBuilder::~PassManagerBuilder() {
delete LibraryInfo;
delete Inliner;
}
Add a mechanism for optimisation plugins to register passes that all front ends can use without needing to be aware of the plugin (or the plugin be aware of the front end). Before 3.0, I'd like to add a mechanism for automatically loading a set of plugins from a config file. API suggestions welcome... llvm-svn: 137717
2011-08-16 21:58:41 +08:00
/// Set of global extensions, automatically added as part of the standard set.
static ManagedStatic<SmallVector<std::pair<PassManagerBuilder::ExtensionPointTy,
PassManagerBuilder::ExtensionFn>, 8> > GlobalExtensions;
Revert r305313 & r305303, self-hosting build-bot isn’t liking it. llvm-svn: 305318
2017-06-14 03:05:24 +08:00
void PassManagerBuilder::addGlobalExtension(
PassManagerBuilder::ExtensionPointTy Ty,
PassManagerBuilder::ExtensionFn Fn) {
GlobalExtensions->push_back(std::make_pair(Ty, std::move(Fn)));
Add a mechanism for optimisation plugins to register passes that all front ends can use without needing to be aware of the plugin (or the plugin be aware of the front end). Before 3.0, I'd like to add a mechanism for automatically loading a set of plugins from a config file. API suggestions welcome... llvm-svn: 137717
2011-08-16 21:58:41 +08:00
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
void PassManagerBuilder::addExtension(ExtensionPointTy Ty, ExtensionFn Fn) {
[PassManager] Make PassManagerBuilder::addExtension take an std::function, rather than a function pointer. Summary: This gives callers flexibility to pass lambdas with captures, which lets callers avoid the C-style void*-ptr closure style. (Currently, callers in clang store state in the PassManagerBuilderBase arg.) No functional change, and the new API is backwards-compatible. Reviewers: chandlerc Subscribers: joker.eph, cfe-commits Differential Revision: http://reviews.llvm.org/D18613 llvm-svn: 264918
2016-03-31 04:39:29 +08:00
Extensions.push_back(std::make_pair(Ty, std::move(Fn)));
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
}
void PassManagerBuilder::addExtensionsToPM(ExtensionPointTy ETy,
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
legacy::PassManagerBase &PM) const {
Add a mechanism for optimisation plugins to register passes that all front ends can use without needing to be aware of the plugin (or the plugin be aware of the front end). Before 3.0, I'd like to add a mechanism for automatically loading a set of plugins from a config file. API suggestions welcome... llvm-svn: 137717
2011-08-16 21:58:41 +08:00
for (unsigned i = 0, e = GlobalExtensions->size(); i != e; ++i)
if ((*GlobalExtensions)[i].first == ETy)
(*GlobalExtensions)[i].second(*this, PM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
for (unsigned i = 0, e = Extensions.size(); i != e; ++i)
if (Extensions[i].first == ETy)
Extensions[i].second(*this, PM);
}
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
void PassManagerBuilder::addInitialAliasAnalysisPasses(
legacy::PassManagerBase &PM) const {
[CFLAA] Split into Anders+Steens analysis. StratifiedSets (as implemented) is very fast, but its accuracy is also limited. If we take a more aggressive andersens-like approach, we can be way more accurate, but we'll also end up being slower. So, we've decided to split CFLAA into CFLSteensAA and CFLAndersAA. Long-term, we want to end up in a place where CFLSteens is queried first; if it can provide an answer, great (since queries are basically map lookups). Otherwise, we'll fall back to CFLAnders, BasicAA, etc. This patch splits everything out so we can try to do something like that when we get a reasonable CFLAnders implementation. Patch by Jia Chen. Differential Revision: http://reviews.llvm.org/D21910 llvm-svn: 274589
2016-07-06 08:26:41 +08:00
switch (UseCFLAA) {
case CFLAAType::Steensgaard:
PM.add(createCFLSteensAAWrapperPass());
break;
case CFLAAType::Andersen:
PM.add(createCFLAndersAAWrapperPass());
break;
case CFLAAType::Both:
PM.add(createCFLSteensAAWrapperPass());
PM.add(createCFLAndersAAWrapperPass());
break;
default:
break;
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Add TypeBasedAliasAnalysis before BasicAliasAnalysis so that
// BasicAliasAnalysis wins if they disagree. This is intended to help
// support "obvious" type-punning idioms.
[PM/AA] Rebuild LLVM's alias analysis infrastructure in a way compatible with the new pass manager, and no longer relying on analysis groups. This builds essentially a ground-up new AA infrastructure stack for LLVM. The core ideas are the same that are used throughout the new pass manager: type erased polymorphism and direct composition. The design is as follows: - FunctionAAResults is a type-erasing alias analysis results aggregation interface to walk a single query across a range of results from different alias analyses. Currently this is function-specific as we always assume that aliasing queries are *within* a function. - AAResultBase is a CRTP utility providing stub implementations of various parts of the alias analysis result concept, notably in several cases in terms of other more general parts of the interface. This can be used to implement only a narrow part of the interface rather than the entire interface. This isn't really ideal, this logic should be hoisted into FunctionAAResults as currently it will cause a significant amount of redundant work, but it faithfully models the behavior of the prior infrastructure. - All the alias analysis passes are ported to be wrapper passes for the legacy PM and new-style analysis passes for the new PM with a shared result object. In some cases (most notably CFL), this is an extremely naive approach that we should revisit when we can specialize for the new pass manager. - BasicAA has been restructured to reflect that it is much more fundamentally a function analysis because it uses dominator trees and loop info that need to be constructed for each function. All of the references to getting alias analysis results have been updated to use the new aggregation interface. All the preservation and other pass management code has been updated accordingly. The way the FunctionAAResultsWrapperPass works is to detect the available alias analyses when run, and add them to the results object. This means that we should be able to continue to respect when various passes are added to the pipeline, for example adding CFL or adding TBAA passes should just cause their results to be available and to get folded into this. The exception to this rule is BasicAA which really needs to be a function pass due to using dominator trees and loop info. As a consequence, the FunctionAAResultsWrapperPass directly depends on BasicAA and always includes it in the aggregation. This has significant implications for preserving analyses. Generally, most passes shouldn't bother preserving FunctionAAResultsWrapperPass because rebuilding the results just updates the set of known AA passes. The exception to this rule are LoopPass instances which need to preserve all the function analyses that the loop pass manager will end up needing. This means preserving both BasicAAWrapperPass and the aggregating FunctionAAResultsWrapperPass. Now, when preserving an alias analysis, you do so by directly preserving that analysis. This is only necessary for non-immutable-pass-provided alias analyses though, and there are only three of interest: BasicAA, GlobalsAA (formerly GlobalsModRef), and SCEVAA. Usually BasicAA is preserved when needed because it (like DominatorTree and LoopInfo) is marked as a CFG-only pass. I've expanded GlobalsAA into the preserved set everywhere we previously were preserving all of AliasAnalysis, and I've added SCEVAA in the intersection of that with where we preserve SCEV itself. One significant challenge to all of this is that the CGSCC passes were actually using the alias analysis implementations by taking advantage of a pretty amazing set of loop holes in the old pass manager's analysis management code which allowed analysis groups to slide through in many cases. Moving away from analysis groups makes this problem much more obvious. To fix it, I've leveraged the flexibility the design of the new PM components provides to just directly construct the relevant alias analyses for the relevant functions in the IPO passes that need them. This is a bit hacky, but should go away with the new pass manager, and is already in many ways cleaner than the prior state. Another significant challenge is that various facilities of the old alias analysis infrastructure just don't fit any more. The most significant of these is the alias analysis 'counter' pass. That pass relied on the ability to snoop on AA queries at different points in the analysis group chain. Instead, I'm planning to build printing functionality directly into the aggregation layer. I've not included that in this patch merely to keep it smaller. Note that all of this needs a nearly complete rewrite of the AA documentation. I'm planning to do that, but I'd like to make sure the new design settles, and to flesh out a bit more of what it looks like in the new pass manager first. Differential Revision: http://reviews.llvm.org/D12080 llvm-svn: 247167
2015-09-10 01:55:00 +08:00
PM.add(createTypeBasedAAWrapperPass());
PM.add(createScopedNoAliasAAWrapperPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
}
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
void PassManagerBuilder::addInstructionCombiningPass(
legacy::PassManagerBase &PM) const {
bool ExpensiveCombines = OptLevel > 2;
PM.add(createInstructionCombiningPass(ExpensiveCombines));
}
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
void PassManagerBuilder::populateFunctionPassManager(
legacy::FunctionPassManager &FPM) {
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
addExtensionsToPM(EP_EarlyAsPossible, FPM);
// Add LibraryInfo if we have some.
[PM] Separate the TargetLibraryInfo object from the immutable pass. The pass is really just a means of accessing a cached instance of the TargetLibraryInfo object, and this way we can re-use that object for the new pass manager as its result. Lots of delta, but nothing interesting happening here. This is the common pattern that is developing to allow analyses to live in both the old and new pass manager -- a wrapper pass in the old pass manager emulates the separation intrinsic to the new pass manager between the result and pass for analyses. llvm-svn: 226157
2015-01-15 18:41:28 +08:00
if (LibraryInfo)
FPM.add(new TargetLibraryInfoWrapperPass(*LibraryInfo));
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
if (OptLevel == 0) return;
addInitialAliasAnalysisPasses(FPM);
FPM.add(createCFGSimplificationPass());
Remove the ScalarReplAggregates pass Nearly all the changes to this pass have been done while maintaining and updating other parts of LLVM. LLVM has had another pass, SROA, which has superseded ScalarReplAggregates for quite some time. Differential Revision: http://reviews.llvm.org/D21316 llvm-svn: 272737
2016-06-15 08:19:09 +08:00
FPM.add(createSROAPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
FPM.add(createEarlyCSEPass());
FPM.add(createLowerExpectIntrinsicPass());
}
[PGO] Passmanagerbuilder change that enable IR level PGO instrumentation This patch includes the passmanagerbuilder change that enables IR level PGO instrumentation. It adds two passmanagerbuilder options: -profile-generate=<profile_filename> and -profile-use=<profile_filename>. The new options are primarily for debug purpose. Reviewers: davidxl, silvas Differential Revision: http://reviews.llvm.org/D15828 llvm-svn: 258420
2016-01-22 02:28:59 +08:00
// Do PGO instrumentation generation or use pass as the option specified.
void PassManagerBuilder::addPGOInstrPasses(legacy::PassManagerBase &MPM) {
[Profile] Use explicit flag to enable IR PGO Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22607 llvm-svn: 276516
2016-07-23 12:28:52 +08:00
if (!EnablePGOInstrGen && PGOInstrUse.empty())
[PGO] IRPGO pre-cleanup pass changes This patch adds a selected set of cleanup passes including a pre-inline pass before LLVM IR PGO instrumentation. The inline is only intended to apply those obvious/trivial ones before instrumentation so that much less instrumentation is needed to get better profiling information. This will drastically improve the instrumented code performance for large C++ applications. Another benefit is the context sensitive counts that can potentially improve the PGO optimization. Differential Revision: http://reviews.llvm.org/D21405 llvm-svn: 275588
2016-07-16 02:10:49 +08:00
return;
// Perform the preinline and cleanup passes for O1 and above.
// And avoid doing them if optimizing for size.
if (OptLevel > 0 && SizeLevel == 0 && !DisablePreInliner) {
Add a new method to create SimpleInliner instance and make pre-inliner use this. This adds a createFunctionInliningPass pass that takes an InlineParams object and use this to create the pre-inliner pass. This prevents the regular inliner's threshold flag from influencing the preinliner. Differential revision: https://reviews.llvm.org/D23377 llvm-svn: 278377
2016-08-12 02:24:08 +08:00
// Create preinline pass. We construct an InlineParams object and specify
// the threshold here to avoid the command line options of the regular
// inliner to influence pre-inlining. The only fields of InlineParams we
// care about are DefaultThreshold and HintThreshold.
InlineParams IP;
IP.DefaultThreshold = PreInlineThreshold;
// FIXME: The hint threshold has the same value used by the regular inliner.
// This should probably be lowered after performance testing.
IP.HintThreshold = 325;
MPM.add(createFunctionInliningPass(IP));
[PGO] IRPGO pre-cleanup pass changes This patch adds a selected set of cleanup passes including a pre-inline pass before LLVM IR PGO instrumentation. The inline is only intended to apply those obvious/trivial ones before instrumentation so that much less instrumentation is needed to get better profiling information. This will drastically improve the instrumented code performance for large C++ applications. Another benefit is the context sensitive counts that can potentially improve the PGO optimization. Differential Revision: http://reviews.llvm.org/D21405 llvm-svn: 275588
2016-07-16 02:10:49 +08:00
MPM.add(createSROAPass());
MPM.add(createEarlyCSEPass()); // Catch trivial redundancies
MPM.add(createCFGSimplificationPass()); // Merge & remove BBs
MPM.add(createInstructionCombiningPass()); // Combine silly seq's
addExtensionsToPM(EP_Peephole, MPM);
}
[Profile] Use explicit flag to enable IR PGO Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22607 llvm-svn: 276516
2016-07-23 12:28:52 +08:00
if (EnablePGOInstrGen) {
[PM] port IR based PGO prof-gen pass to new pass manager llvm-svn: 268710
2016-05-06 13:49:19 +08:00
MPM.add(createPGOInstrumentationGenLegacyPass());
[PGO] Passmanagerbuilder change that enable IR level PGO instrumentation This patch includes the passmanagerbuilder change that enables IR level PGO instrumentation. It adds two passmanagerbuilder options: -profile-generate=<profile_filename> and -profile-use=<profile_filename>. The new options are primarily for debug purpose. Reviewers: davidxl, silvas Differential Revision: http://reviews.llvm.org/D15828 llvm-svn: 258420
2016-01-22 02:28:59 +08:00
// Add the profile lowering pass.
InstrProfOptions Options;
[Profile] Use explicit flag to enable IR PGO Patch by Jake VanAdrighem Differential Revision: http://reviews.llvm.org/D22607 llvm-svn: 276516
2016-07-23 12:28:52 +08:00
if (!PGOInstrGen.empty())
Options.InstrProfileOutput = PGOInstrGen;
Port InstrProfiling pass to the new pass manager Differential Revision: http://reviews.llvm.org/D18126 llvm-svn: 266637
2016-04-19 01:47:38 +08:00
MPM.add(createInstrProfilingLegacyPass(Options));
[PGO] Passmanagerbuilder change that enable IR level PGO instrumentation This patch includes the passmanagerbuilder change that enables IR level PGO instrumentation. It adds two passmanagerbuilder options: -profile-generate=<profile_filename> and -profile-use=<profile_filename>. The new options are primarily for debug purpose. Reviewers: davidxl, silvas Differential Revision: http://reviews.llvm.org/D15828 llvm-svn: 258420
2016-01-22 02:28:59 +08:00
}
if (!PGOInstrUse.empty())
[PM] code refactoring -- preparation for new PM porting /NFC llvm-svn: 268851
2016-05-07 13:39:12 +08:00
MPM.add(createPGOInstrumentationUseLegacyPass(PGOInstrUse));
[PM] Run IndirectCallPromotion only when PGO is enabled. Differential Revision: https://reviews.llvm.org/D32465 llvm-svn: 301327
2017-04-26 00:54:45 +08:00
// Indirect call promotion that promotes intra-module targets only.
// For ThinLTO this is done earlier due to interactions with globalopt
// for imported functions. We don't run this at -O0.
if (OptLevel > 0)
MPM.add(
createPGOIndirectCallPromotionLegacyPass(false, !PGOSampleUse.empty()));
[PGO] Passmanagerbuilder change that enable IR level PGO instrumentation This patch includes the passmanagerbuilder change that enables IR level PGO instrumentation. It adds two passmanagerbuilder options: -profile-generate=<profile_filename> and -profile-use=<profile_filename>. The new options are primarily for debug purpose. Reviewers: davidxl, silvas Differential Revision: http://reviews.llvm.org/D15828 llvm-svn: 258420
2016-01-22 02:28:59 +08:00
}
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
void PassManagerBuilder::addFunctionSimplificationPasses(
Revert "Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()"" This reverts commit r260603. I didn't intend to push it :( From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 260607
2016-02-12 06:09:11 +08:00
legacy::PassManagerBase &MPM) {
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Start of function pass.
// Break up aggregate allocas, using SSAUpdater.
Remove the ScalarReplAggregates pass Nearly all the changes to this pass have been done while maintaining and updating other parts of LLVM. LLVM has had another pass, SROA, which has superseded ScalarReplAggregates for quite some time. Differential Revision: http://reviews.llvm.org/D21316 llvm-svn: 272737
2016-06-15 08:19:09 +08:00
MPM.add(createSROAPass());
[EarlyCSE] Add option to use MemorySSA for function simplification run of EarlyCSE (off by default). Summary: Use MemorySSA for memory dependency checking in the EarlyCSE pass at the start of the function simplification portion of the pipeline. We rely on the fact that GVNHoist runs just after this pass of EarlyCSE to amortize the MemorySSA construction cost since GVNHoist uses MemorySSA and EarlyCSE preserves it. This is turned off by default. A follow-up change will turn it on to allow for easier reversion in case it breaks something. llvm-svn: 305146
2017-06-10 23:20:03 +08:00
MPM.add(createEarlyCSEPass(EnableEarlyCSEMemSSA)); // Catch trivial redundancies
Re-apply "[GVNHoist] Move GVNHoist to function simplification part of pipeline." This reverts commit r296872 now that PR32153 has been fixed. llvm-svn: 300200
2017-04-13 23:36:25 +08:00
if (EnableGVNHoist)
MPM.add(createGVNHoistPass());
[GVNSink] GVNSink pass This patch provides an initial prototype for a pass that sinks instructions based on GVN information, similar to GVNHoist. It is not yet ready for commiting but I've uploaded it to gather some initial thoughts. This pass attempts to sink instructions into successors, reducing static instruction count and enabling if-conversion. We use a variant of global value numbering to decide what can be sunk. Consider: [ %a1 = add i32 %b, 1 ] [ %c1 = add i32 %d, 1 ] [ %a2 = xor i32 %a1, 1 ] [ %c2 = xor i32 %c1, 1 ] \ / [ %e = phi i32 %a2, %c2 ] [ add i32 %e, 4 ] GVN would number %a1 and %c1 differently because they compute different results - the VN of an instruction is a function of its opcode and the transitive closure of its operands. This is the key property for hoisting and CSE. What we want when sinking however is for a numbering that is a function of the *uses* of an instruction, which allows us to answer the question "if I replace %a1 with %c1, will it contribute in an equivalent way to all successive instructions?". The (new) PostValueTable class in GVN provides this mapping. This pass has some shown really impressive improvements especially for codesize already on internal benchmarks, so I have high hopes it can replace all the sinking logic in SimplifyCFG. Differential revision: https://reviews.llvm.org/D24805 llvm-svn: 303850
2017-05-25 20:51:11 +08:00
if (EnableGVNSink) {
MPM.add(createGVNSinkPass());
MPM.add(createCFGSimplificationPass());
}
[PM] Add a SpeculativeExecution pass for targets with divergent branches. Summary: This IR pass is helpful for GPUs, and other targets with divergent branches. It's a nop on targets without divergent branches. Reviewers: chandlerc Subscribers: llvm-commits, jingyue, rnk, joker.eph, tra Differential Revision: http://reviews.llvm.org/D18626 llvm-svn: 266399
2016-04-15 08:32:12 +08:00
// Speculative execution if the target has divergent branches; otherwise nop.
MPM.add(createSpeculativeExecutionIfHasBranchDivergencePass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createJumpThreadingPass()); // Thread jumps.
MPM.add(createCorrelatedValuePropagationPass()); // Propagate conditionals
MPM.add(createCFGSimplificationPass()); // Merge & remove BBs
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
// Combine silly seq's
addInstructionCombiningPass(MPM);
Conditionally eliminate library calls where the result value is not used Summary: This pass shrink-wraps a condition to some library calls where the call result is not used. For example: sqrt(val); is transformed to if (val < 0) sqrt(val); Even if the result of library call is not being used, the compiler cannot safely delete the call because the function can set errno on error conditions. Note in many functions, the error condition solely depends on the incoming parameter. In this optimization, we can generate the condition can lead to the errno to shrink-wrap the call. Since the chances of hitting the error condition is low, the runtime call is effectively eliminated. These partially dead calls are usually results of C++ abstraction penalty exposed by inlining. This optimization hits 108 times in 19 C/C++ programs in SPEC2006. Reviewers: hfinkel, mehdi_amini, davidxl Subscribers: modocache, mgorny, mehdi_amini, xur, llvm-commits, beanz Differential Revision: https://reviews.llvm.org/D24414 llvm-svn: 284542
2016-10-19 05:36:27 +08:00
if (SizeLevel == 0 && !DisableLibCallsShrinkWrap)
MPM.add(createLibCallsShrinkWrapPass());
Add an extension point for peephole optimizers. This extension point allows adding passes that perform peephole optimizations similar to the instruction combiner. These passes will be inserted after each instance of the instruction combiner pass. Differential Revision: http://reviews.llvm.org/D3905 llvm-svn: 209595
2014-05-25 18:27:02 +08:00
addExtensionsToPM(EP_Peephole, MPM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
[PGO] Memory intrinsic calls optimization based on profiled size This patch optimizes two memory intrinsic operations: memset and memcpy based on the profiled size of the operation. The high level transformation is like: mem_op(..., size) ==> switch (size) { case s1: mem_op(..., s1); goto merge_bb; case s2: mem_op(..., s2); goto merge_bb; ... default: mem_op(..., size); goto merge_bb; } merge_bb: Differential Revision: http://reviews.llvm.org/D28966 llvm-svn: 299446
2017-04-05 00:42:20 +08:00
// Optimize memory intrinsic calls based on the profiled size information.
if (SizeLevel == 0)
MPM.add(createPGOMemOPSizeOptLegacyPass());
Remove DisableTailCalls from TargetOptions and the code in resetTargetOptions that was resetting it. Remove the uses of DisableTailCalls in subclasses of TargetLowering and use the value of function attribute "disable-tail-calls" instead. Also, unconditionally add pass TailCallElim to the pipeline and check the function attribute at the start of runOnFunction to disable the pass on a per-function basis. This is part of the work to remove TargetMachine::resetTargetOptions, and since DisableTailCalls was the last non-fast-math option that was being reset in that function, we should be able to remove the function entirely after the work to propagate IR-level fast-math flags to DAG nodes is completed. Out-of-tree users should remove the uses of DisableTailCalls and make changes to attach attribute "disable-tail-calls"="true" or "false" to the functions in the IR. rdar://problem/13752163 Differential Revision: http://reviews.llvm.org/D10099 llvm-svn: 239427
2015-06-10 03:07:19 +08:00
MPM.add(createTailCallEliminationPass()); // Eliminate tail calls
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createCFGSimplificationPass()); // Merge & remove BBs
MPM.add(createReassociatePass()); // Reassociate expressions
Disable header duplication at -Oz in loop-rotate pass. llvm-svn: 222562
2014-11-22 03:53:24 +08:00
// Rotate Loop - disable header duplication at -Oz
MPM.add(createLoopRotatePass(SizeLevel == 2 ? 0 : -1));
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createLICMPass()); // Hoist loop invariants
[PM/LoopUnswitch] Introduce a new, simpler loop unswitch pass. Currently, this pass only focuses on *trivial* loop unswitching. At that reduced problem it remains significantly better than the current loop unswitch: - Old pass is worse than cubic complexity. New pass is (I think) linear. - New pass is much simpler in its design by focusing on full unswitching. (See below for details on this). - New pass doesn't carry state for thresholds between pass iterations. - New pass doesn't carry state for correctness (both miscompile and infloop) between pass iterations. - New pass produces substantially better code after unswitching. - New pass can handle more trivial unswitch cases. - New pass doesn't recompute the dominator tree for the entire function and instead incrementally updates it. I've ported all of the trivial unswitching test cases from the old pass to the new one to make sure that major functionality isn't lost in the process. For several of the test cases I've worked to improve the precision and rigor of the CHECKs, but for many I've just updated them to handle the new IR produced. My initial motivation was the fact that the old pass carried state in very unreliable ways between pass iterations, and these mechansims were incompatible with the new pass manager. However, I discovered many more improvements to make along the way. This pass makes two very significant assumptions that enable most of these improvements: 1) Focus on *full* unswitching -- that is, completely removing whatever control flow construct is being unswitched from the loop. In the case of trivial unswitching, this means removing the trivial (exiting) edge. In non-trivial unswitching, this means removing the branch or switch itself. This is in opposition to *partial* unswitching where some part of the unswitched control flow remains in the loop. Partial unswitching only really applies to switches and to folded branches. These are very similar to full unrolling and partial unrolling. The full form is an effective canonicalization, the partial form needs a complex cost model, cannot be iterated, isn't canonicalizing, and should be a separate pass that runs very late (much like unrolling). 2) Leverage LLVM's Loop machinery to the fullest. The original unswitch dates from a time when a great deal of LLVM's loop infrastructure was missing, ineffective, and/or unreliable. As a consequence, a lot of complexity was added which we no longer need. With these two overarching principles, I think we can build a fast and effective unswitcher that fits in well in the new PM and in the canonicalization pipeline. Some of the remaining functionality around partial unswitching may not be relevant today (not many test cases or benchmarks I can find) but if they are I'd like to add support for them as a separate layer that runs very late in the pipeline. Purely to make reviewing and introducing this code more manageable, I've split this into first a trivial-unswitch-only pass and in the next patch I'll add support for full non-trivial unswitching against a *fixed* threshold, exactly like full unrolling. I even plan to re-use the unrolling thresholds, as these are incredibly similar cost tradeoffs: we're cloning a loop body in order to end up with simplified control flow. We should only do that when the total growth is reasonably small. One of the biggest changes with this pass compared to the previous one is that previously, each individual trivial exiting edge from a switch was unswitched separately as a branch. Now, we unswitch the entire switch at once, with cases going to the various destinations. This lets us unswitch multiple exiting edges in a single operation and also avoids numerous extremely bad behaviors, where we would introduce 1000s of branches to test for thousands of possible values, all of which would take the exact same exit path bypassing the loop. Now we will use a switch with 1000s of cases that can be efficiently lowered into a jumptable. This avoids relying on somehow forming a switch out of the branches or getting horrible code if that fails for any reason. Another significant change is that this pass actively updates the CFG based on unswitching. For trivial unswitching, this is actually very easy because of the definition of loop simplified form. Doing this makes the code coming out of loop unswitch dramatically more friendly. We still should run loop-simplifycfg (at the least) after this to clean up, but it will have to do a lot less work. Finally, this pass makes much fewer attempts to simplify instructions based on the unswitch. Something like loop-instsimplify, instcombine, or GVN can be used to do increasingly powerful simplifications based on the now dominating predicate. The old simplifications are things that something like loop-instsimplify should get today or a very, very basic loop-instcombine could get. Keeping that logic separate is a big simplifying technique. Most of the code in this pass that isn't in the old one has to do with achieving specific goals: - Updating the dominator tree as we go - Unswitching all cases in a switch in a single step. I think it is still shorter than just the trivial unswitching code in the old pass despite having this functionality. Differential Revision: https://reviews.llvm.org/D32409 llvm-svn: 301576
2017-04-28 02:45:20 +08:00
if (EnableSimpleLoopUnswitch)
MPM.add(createSimpleLoopUnswitchLegacyPass());
else
MPM.add(createLoopUnswitchPass(SizeLevel || OptLevel < 3, DivergentTarget));
Add CFG Simplification pass after Loop Unswitching. Loop unswitching produces conditional branches with constant condition, and it's beneficial for later passes to clean this up with simplify-cfg. We do this after the second invocation of loop-unswitch, but not after the first one. Not doing so might cause problem for passes like LoopUnroll, whose estimate of loop body size would be less accurate. Reviewers: hfinkel Differential Revision: http://reviews.llvm.org/D13064 llvm-svn: 248460
2015-09-24 11:50:17 +08:00
MPM.add(createCFGSimplificationPass());
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createIndVarSimplifyPass()); // Canonicalize indvars
MPM.add(createLoopIdiomPass()); // Recognize idioms like memset.
Add loop pass insertion point EP_LateLoopOptimizations Differential Revision: https://reviews.llvm.org/D28694 llvm-svn: 293067
2017-01-26 00:12:25 +08:00
addExtensionsToPM(EP_LateLoopOptimizations, MPM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createLoopDeletionPass()); // Delete dead loops
Add loop pass insertion point EP_LateLoopOptimizations Differential Revision: https://reviews.llvm.org/D28694 llvm-svn: 293067
2017-01-26 00:12:25 +08:00
Add support to interchange loops with reductions. This patch enables interchanging of tightly nested loops with reductions. Differential Revision: http://reviews.llvm.org/D8314 llvm-svn: 235571
2015-04-23 12:51:44 +08:00
if (EnableLoopInterchange) {
Add a new pass "Loop Interchange" This pass interchanges loops to provide a more cache-friendly memory access. For e.g. given a loop like - for(int i=0;i<N;i++) for(int j=0;j<N;j++) A[j][i] = A[j][i]+B[j][i]; is interchanged to - for(int j=0;j<N;j++) for(int i=0;i<N;i++) A[j][i] = A[j][i]+B[j][i]; This pass is currently disabled by default. To give a brief introduction it consists of 3 stages- LoopInterchangeLegality : Checks the legality of loop interchange based on Dependency matrix. LoopInterchangeProfitability: A very basic heuristic has been added to check for profitibility. This will evolve over time. LoopInterchangeTransform : Which does the actual transform. LNT Performance tests shows improvement in Polybench/linear-algebra/kernels/mvt and Polybench/linear-algebra/kernels/gemver becnmarks. TODO: 1) Add support for reductions and lcssa phi. 2) Improve profitability model. 3) Improve loop selection algorithm to select best loop for interchange. Currently the innermost loop is selected for interchange. 4) Improve compile time regression found in llvm lnt due to this pass. 5) Fix issues in Dependency Analysis module. A special thanks to Hal for reviewing this code. Review: http://reviews.llvm.org/D7499 llvm-svn: 231458
2015-03-06 18:11:25 +08:00
MPM.add(createLoopInterchangePass()); // Interchange loops
Add support to interchange loops with reductions. This patch enables interchanging of tightly nested loops with reductions. Differential Revision: http://reviews.llvm.org/D8314 llvm-svn: 235571
2015-04-23 12:51:44 +08:00
MPM.add(createCFGSimplificationPass());
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
if (!DisableUnrollLoops)
Increases full-unroll threshold. Summary: The default threshold for fully unroll is too conservative. This patch doubles the full-unroll threshold This change will affect the following speccpu2006 benchmarks (performance numbers were collected from Intel Sandybridge): Performance: 403 0.11% 433 0.51% 445 0.48% 447 3.50% 453 1.49% 464 0.75% Code size: 403 0.56% 433 0.96% 445 2.16% 447 2.96% 453 0.94% 464 8.02% The compiler time overhead is similar with code size. Reviewers: davidxl, mkuper, mzolotukhin, hfinkel, chandlerc Reviewed By: hfinkel, chandlerc Subscribers: mehdi_amini, zzheng, efriedma, haicheng, hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D28368 llvm-svn: 295538
2017-02-18 11:46:51 +08:00
MPM.add(createSimpleLoopUnrollPass(OptLevel)); // Unroll small loops
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
addExtensionsToPM(EP_LoopOptimizerEnd, MPM);
MergedLoadStoreMotion pass Merges equivalent loads on both sides of a hammock/diamond and hoists into into the header. Merges equivalent stores on both sides of a hammock/diamond and sinks it to the footer. Can enable if conversion and tolerate better load misses and store operand latencies. llvm-svn: 213396
2014-07-19 03:13:09 +08:00
if (OptLevel > 1) {
[PM] MLSM has been enabled for a way. Reclaim a cl::opt. llvm-svn: 293401
2017-01-29 07:45:37 +08:00
MPM.add(createMergedLoadStoreMotionPass()); // Merge ld/st in diamonds
[NewGVN] Add a flag to enable the pass via `-mllvm`. NewGVN can be tested passing `-mllvm -enable-newgvn` to clang. Differential Revision: https://reviews.llvm.org/D28059 llvm-svn: 290548
2016-12-27 02:26:19 +08:00
MPM.add(NewGVN ? createNewGVNPass()
: createGVNPass(DisableGVNLoadPRE)); // Remove redundancies
MergedLoadStoreMotion pass Merges equivalent loads on both sides of a hammock/diamond and hoists into into the header. Merges equivalent stores on both sides of a hammock/diamond and sinks it to the footer. Can enable if conversion and tolerate better load misses and store operand latencies. llvm-svn: 213396
2014-07-19 03:13:09 +08:00
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createMemCpyOptPass()); // Remove memcpy / form memset
MPM.add(createSCCPPass()); // Constant prop with SCCP
[BDCE] Add a bit-tracking DCE pass BDCE is a bit-tracking dead code elimination pass. It is based on ADCE (the "aggressive DCE" pass), with the added capability to track dead bits of integer valued instructions and remove those instructions when all of the bits are dead. Currently, it does not actually do this all-bits-dead removal, but rather replaces the instruction's uses with a constant zero, and lets instcombine (and the later run of ADCE) do the rest. Because we essentially get a run of ADCE "for free" while tracking the dead bits, we also do what ADCE does and removes actually-dead instructions as well (this includes instructions newly trivially dead because all bits were dead, but not all such instructions can be removed). The motivation for this is a case like: int __attribute__((const)) foo(int i); int bar(int x) { x |= (4 & foo(5)); x |= (8 & foo(3)); x |= (16 & foo(2)); x |= (32 & foo(1)); x |= (64 & foo(0)); x |= (128& foo(4)); return x >> 4; } As it turns out, if you order the bit-field insertions so that all of the dead ones come last, then instcombine will remove them. However, if you pick some other order (such as the one above), the fact that some of the calls to foo() are useless is not locally obvious, and we don't remove them (without this pass). I did a quick compile-time overhead check using sqlite from the test suite (Release+Asserts). BDCE took ~0.4% of the compilation time (making it about twice as expensive as ADCE). I've not looked at why yet, but we eliminate instructions due to having all-dead bits in: External/SPEC/CFP2006/447.dealII/447.dealII External/SPEC/CINT2006/400.perlbench/400.perlbench External/SPEC/CINT2006/403.gcc/403.gcc MultiSource/Applications/ClamAV/clamscan MultiSource/Benchmarks/7zip/7zip-benchmark llvm-svn: 229462
2015-02-17 09:36:59 +08:00
// Delete dead bit computations (instcombine runs after to fold away the dead
// computations, and then ADCE will run later to exploit any new DCE
// opportunities that creates).
MPM.add(createBitTrackingDCEPass()); // Delete dead bit computations
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Run instcombine after redundancy elimination to exploit opportunities
// opened up by them.
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add an extension point for peephole optimizers. This extension point allows adding passes that perform peephole optimizations similar to the instruction combiner. These passes will be inserted after each instance of the instruction combiner pass. Differential Revision: http://reviews.llvm.org/D3905 llvm-svn: 209595
2014-05-25 18:27:02 +08:00
addExtensionsToPM(EP_Peephole, MPM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createJumpThreadingPass()); // Thread jumps
MPM.add(createCorrelatedValuePropagationPass());
MPM.add(createDeadStoreEliminationPass()); // Delete dead stores
Run LICM as part of the cleanup phase from the scalar optimizer. Things like LoopUnrolling can produce loop invariant values - make sure we pick them up. llvm-svn: 229419
2015-02-17 02:59:54 +08:00
MPM.add(createLICMPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
addExtensionsToPM(EP_ScalarOptimizerLate, MPM);
Add a loop rerolling flag to the PassManagerBuilder This adds a boolean member variable to the PassManagerBuilder to control loop rerolling (just like we have for unrolling and the various vectorization options). This is necessary for control by the frontend. Loop rerolling remains disabled by default at all optimization levels. llvm-svn: 194966
2013-11-18 00:02:50 +08:00
if (RerollLoops)
Add a loop rerolling pass This adds a loop rerolling pass: the opposite of (partial) loop unrolling. The transformation aims to take loops like this: for (int i = 0; i < 3200; i += 5) { a[i] += alpha * b[i]; a[i + 1] += alpha * b[i + 1]; a[i + 2] += alpha * b[i + 2]; a[i + 3] += alpha * b[i + 3]; a[i + 4] += alpha * b[i + 4]; } and turn them into this: for (int i = 0; i < 3200; ++i) { a[i] += alpha * b[i]; } and loops like this: for (int i = 0; i < 500; ++i) { x[3*i] = foo(0); x[3*i+1] = foo(0); x[3*i+2] = foo(0); } and turn them into this: for (int i = 0; i < 1500; ++i) { x[i] = foo(0); } There are two motivations for this transformation: 1. Code-size reduction (especially relevant, obviously, when compiling for code size). 2. Providing greater choice to the loop vectorizer (and generic unroller) to choose the unrolling factor (and a better ability to vectorize). The loop vectorizer can take vector lengths and register pressure into account when choosing an unrolling factor, for example, and a pre-unrolled loop limits that choice. This is especially problematic if the manual unrolling was optimized for a machine different from the current target. The current implementation is limited to single basic-block loops only. The rerolling recognition should work regardless of how the loop iterations are intermixed within the loop body (subject to dependency and side-effect constraints), but the significant restriction is that the order of the instructions in each iteration must be identical. This seems sufficient to capture all current use cases. This pass is not currently enabled by default at any optimization level. llvm-svn: 194939
2013-11-17 07:59:05 +08:00
MPM.add(createLoopRerollPass());
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
if (!RunSLPAfterLoopVectorization) {
if (SLPVectorize)
MPM.add(createSLPVectorizerPass()); // Vectorize parallel scalar chains.
if (BBVectorize) {
MPM.add(createBBVectorizePass());
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
addExtensionsToPM(EP_Peephole, MPM);
if (OptLevel > 1 && UseGVNAfterVectorization)
[NewGVN] Add a flag to enable the pass via `-mllvm`. NewGVN can be tested passing `-mllvm -enable-newgvn` to clang. Differential Revision: https://reviews.llvm.org/D28059 llvm-svn: 290548
2016-12-27 02:26:19 +08:00
MPM.add(NewGVN
? createNewGVNPass()
: createGVNPass(DisableGVNLoadPRE)); // Remove redundancies
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
else
MPM.add(createEarlyCSEPass()); // Catch trivial redundancies
// BBVectorize may have significantly shortened a loop body; unroll again.
if (!DisableUnrollLoops)
Increases full-unroll threshold. Summary: The default threshold for fully unroll is too conservative. This patch doubles the full-unroll threshold This change will affect the following speccpu2006 benchmarks (performance numbers were collected from Intel Sandybridge): Performance: 403 0.11% 433 0.51% 445 0.48% 447 3.50% 453 1.49% 464 0.75% Code size: 403 0.56% 433 0.96% 445 2.16% 447 2.96% 453 0.94% 464 8.02% The compiler time overhead is similar with code size. Reviewers: davidxl, mkuper, mzolotukhin, hfinkel, chandlerc Reviewed By: hfinkel, chandlerc Subscribers: mehdi_amini, zzheng, efriedma, haicheng, hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D28368 llvm-svn: 295538
2017-02-18 11:46:51 +08:00
MPM.add(createLoopUnrollPass(OptLevel));
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
}
Add a basic-block autovectorization pass. This is the initial checkin of the basic-block autovectorization pass along with some supporting vectorization infrastructure. Special thanks to everyone who helped review this code over the last several months (especially Tobias Grosser). llvm-svn: 149468
2012-02-01 11:51:43 +08:00
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createAggressiveDCEPass()); // Delete dead instructions
Factor FlattenCFG out from SimplifyCFG Patch by: Mei Ye llvm-svn: 187764
2013-08-06 10:43:45 +08:00
MPM.add(createCFGSimplificationPass()); // Merge & remove BBs
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
// Clean up after everything.
addInstructionCombiningPass(MPM);
Add an extension point for peephole optimizers. This extension point allows adding passes that perform peephole optimizations similar to the instruction combiner. These passes will be inserted after each instance of the instruction combiner pass. Differential Revision: http://reviews.llvm.org/D3905 llvm-svn: 209595
2014-05-25 18:27:02 +08:00
addExtensionsToPM(EP_Peephole, MPM);
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
}
void PassManagerBuilder::populateModulePassManager(
legacy::PassManagerBase &MPM) {
Create SampleProfileLoader pass in llvm instead of clang Summary: We used to create SampleProfileLoader pass in clang. This makes LTO/ThinLTO unable to add this pass in the linker plugin. This patch moves the SampleProfileLoader pass creation from clang to llvm pass manager builder. Reviewers: tejohnson, davidxl, dnovillo Subscribers: llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D27743 llvm-svn: 289714
2016-12-15 05:40:47 +08:00
if (!PGOSampleUse.empty()) {
MPM.add(createPruneEHPass());
MPM.add(createSampleProfileLoaderPass(PGOSampleUse));
}
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
// Allow forcing function attributes as a debugging and tuning aid.
MPM.add(createForceFunctionAttrsLegacyPass());
// If all optimizations are disabled, just run the always-inline pass and,
// if enabled, the function merging pass.
if (OptLevel == 0) {
addPGOInstrPasses(MPM);
if (Inliner) {
MPM.add(Inliner);
Inliner = nullptr;
}
// FIXME: The BarrierNoopPass is a HACK! The inliner pass above implicitly
// creates a CGSCC pass manager, but we don't want to add extensions into
// that pass manager. To prevent this we insert a no-op module pass to reset
// the pass manager to get the same behavior as EP_OptimizerLast in non-O0
// builds. The function merging pass is
if (MergeFunctions)
MPM.add(createMergeFunctionsPass());
else if (!GlobalExtensions->empty() || !Extensions.empty())
MPM.add(createBarrierNoopPass());
[ThinLTO/Summary] Rename anonymous globals as last action ... ... in the per-TU -O0 pipeline. The problem is that there could be passes registered using `addExtensionsToPM()` introducing unnamed globals. Asan is an example, but there may be others. Building cppcheck with `-flto=thin` and `-fsanitize=address` triggers an assertion while we're reading bitcode (in lib/LTO), as the BitcodeReader assumes there are no unnamed globals (because the namer has run). Unfortunately I wasn't able to find an easy way to test this. I added a comment in the hope nobody moves this again. llvm-svn: 301102
2017-04-23 12:49:34 +08:00
addExtensionsToPM(EP_EnabledOnOptLevel0, MPM);
// Rename anon globals to be able to export them in the summary.
// This has to be done after we add the extensions to the pass manager
// as there could be passes (e.g. Adddress sanitizer) which introduce
// new unnamed globals.
[ThinLTO] Ensure anonymous globals renamed even at -O0 Summary: This fixes an issue when files are compiled with -flto=thin at default -O0. We need to rename anonymous globals before attempting to write the module summary because all values need names for the summary. This was happening at -O1 and above, but not before the early exit when constructing the pipeline for -O0. Also add an internal -prepare-for-thinlto option to enable this to be tested via opt. Fixes PR30419. Reviewers: mehdi_amini Subscribers: probinson, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D24701 llvm-svn: 281840
2016-09-18 04:40:16 +08:00
if (PrepareForThinLTO)
MPM.add(createNameAnonGlobalPass());
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
return;
}
// Add LibraryInfo if we have some.
if (LibraryInfo)
MPM.add(new TargetLibraryInfoWrapperPass(*LibraryInfo));
addInitialAliasAnalysisPasses(MPM);
[ThinLTO] Indirect call promotion fixes for promoted local functions Summary: Fix a couple issues limiting the application of indirect call promotion in ThinLTO mode: - Invoke indirect call promotion before globalopt, since it may eliminate imported functions which appear unreferenced. - Invoke indirect call promotion with InLTO=true so that the PGOFuncName metadata is used to get the name for locals which would have been renamed during promotion. Reviewers: davidxl, mehdi_amini Subscribers: Prazek, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D24004 llvm-svn: 280024
2016-08-30 06:46:56 +08:00
// For ThinLTO there are two passes of indirect call promotion. The
// first is during the compile phase when PerformThinLTO=false and
// intra-module indirect call targets are promoted. The second is during
// the ThinLTO backend when PerformThinLTO=true, when we promote imported
// inter-module indirect calls. For that we perform indirect call promotion
// earlier in the pass pipeline, here before globalopt. Otherwise imported
// available_externally functions look unreferenced and are removed.
if (PerformThinLTO)
Add call branch annotation for ICP promoted direct call in SamplePGO mode. Summary: SamplePGO uses branch_weight annotation to represent callsite hotness. When ICP promotes an indirect call to direct call, we need to make sure the direct call is annotated with branch_weight in SamplePGO mode, so that downstream function inliner can use hot callsite heuristic. Reviewers: davidxl, eraman, xur Reviewed By: davidxl, xur Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D30282 llvm-svn: 296028
2017-02-24 06:15:18 +08:00
MPM.add(createPGOIndirectCallPromotionLegacyPass(/*InLTO = */ true,
!PGOSampleUse.empty()));
[ThinLTO] Indirect call promotion fixes for promoted local functions Summary: Fix a couple issues limiting the application of indirect call promotion in ThinLTO mode: - Invoke indirect call promotion before globalopt, since it may eliminate imported functions which appear unreferenced. - Invoke indirect call promotion with InLTO=true so that the PGOFuncName metadata is used to get the name for locals which would have been renamed during promotion. Reviewers: davidxl, mehdi_amini Subscribers: Prazek, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D24004 llvm-svn: 280024
2016-08-30 06:46:56 +08:00
Disable loop unrolling and icp in SamplePGO ThinLTO compile phase Summary: loop unrolling and icp will make the sample profile annotation much harder in the backend. So disable these 2 optimization in the ThinLTO compile phase. Will add a test in cfe in a separate patch. Reviewers: tejohnson Reviewed By: tejohnson Subscribers: mehdi_amini, llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D31217 llvm-svn: 298646
2017-03-24 05:20:05 +08:00
// For SamplePGO in ThinLTO compile phase, we do not want to unroll loops
// as it will change the CFG too much to make the 2nd profile annotation
// in backend more difficult.
bool PrepareForThinLTOUsingPGOSampleProfile =
PrepareForThinLTO && !PGOSampleUse.empty();
if (PrepareForThinLTOUsingPGOSampleProfile)
DisableUnrollLoops = true;
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
if (!DisableUnitAtATime) {
// Infer attributes about declarations if possible.
MPM.add(createInferFunctionAttrsLegacyPass());
addExtensionsToPM(EP_ModuleOptimizerEarly, MPM);
MPM.add(createIPSCCPPass()); // IP SCCP
MPM.add(createGlobalOptimizerPass()); // Optimize out global vars
// Promote any localized global vars.
MPM.add(createPromoteMemoryToRegisterPass());
MPM.add(createDeadArgEliminationPass()); // Dead argument elimination
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM); // Clean up after IPCP & DAE
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
addExtensionsToPM(EP_Peephole, MPM);
MPM.add(createCFGSimplificationPass()); // Clean up after IPCP & DAE
}
Disable loop unrolling and icp in SamplePGO ThinLTO compile phase Summary: loop unrolling and icp will make the sample profile annotation much harder in the backend. So disable these 2 optimization in the ThinLTO compile phase. Will add a test in cfe in a separate patch. Reviewers: tejohnson Reviewed By: tejohnson Subscribers: mehdi_amini, llvm-commits, Prazek Differential Revision: https://reviews.llvm.org/D31217 llvm-svn: 298646
2017-03-24 05:20:05 +08:00
// For SamplePGO in ThinLTO compile phase, we do not want to do indirect
// call promotion as it will change the CFG too much to make the 2nd
// profile annotation in backend more difficult.
[PM] Run IndirectCallPromotion only when PGO is enabled. Differential Revision: https://reviews.llvm.org/D32465 llvm-svn: 301327
2017-04-26 00:54:45 +08:00
// PGO instrumentation is added during the compile phase for ThinLTO, do
// not run it a second time
if (!PerformThinLTO && !PrepareForThinLTOUsingPGOSampleProfile)
Define the ThinLTO Pipeline (experimental) Summary: On the contrary to Full LTO, ThinLTO can afford to shift compile time from the frontend to the linker: both phases are parallel (even if it is not totally "free": projects like clang are reusing product from the "compile phase" for multiple link, think about libLLVMSupport reused for opt, llc, etc.). This pipeline is based on the proposal in D13443 for full LTO. We didn't move forward on this proposal because the LTO link was far too long after that. We believe that we can afford it with ThinLTO. The ThinLTO pipeline integrates in the regular O2/O3 flow: - The compile phase perform the inliner with a somehow lighter function simplification. (TODO: tune the inliner thresholds here) This is intendend to simplify the IR and get rid of obvious things like linkonce_odr that will be inlined. - The link phase will run the pipeline from the start, extended with some specific passes that leverage the augmented knowledge we have during LTO. Especially after the inliner is done, a sequence of globalDCE/globalOpt is performed, followed by another run of the "function simplification" passes. It is not clear if this part of the pipeline will stay as is, as the split model of ThinLTO does not allow the same benefit as FullLTO without added tricks. The measurements on the public test suite as well as on our internal suite show an overall net improvement. The binary size for the clang executable is reduced by 5%. We're still tuning it with the bringup of ThinLTO and it will evolve, but this should provide a good starting point. Reviewers: tejohnson Differential Revision: http://reviews.llvm.org/D17115 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261029
2016-02-17 07:02:29 +08:00
addPGOInstrPasses(MPM);
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
if (EnableNonLTOGlobalsModRef)
// We add a module alias analysis pass here. In part due to bugs in the
// analysis infrastructure this "works" in that the analysis stays alive
// for the entire SCC pass run below.
MPM.add(createGlobalsAAWrapperPass());
// Start of CallGraph SCC passes.
if (!DisableUnitAtATime)
MPM.add(createPruneEHPass()); // Remove dead EH info
if (Inliner) {
MPM.add(Inliner);
Inliner = nullptr;
}
if (!DisableUnitAtATime)
[PM] Port the PostOrderFunctionAttrs pass to the new pass manager and convert one test to use this. This is a particularly significant milestone because it required a working per-function AA framework which can be queried over each function from within a CGSCC transform pass (and additionally a module analysis to be accessible). This is essentially *the* point of the entire pass manager rewrite. A CGSCC transform is able to query for multiple different function's analysis results. It works. The whole thing appears to actually work and accomplish the original goal. While we were able to hack function attrs and basic-aa to "work" in the old pass manager, this port doesn't use any of that, it directly leverages the new fundamental functionality. For this to work, the CGSCC framework also has to support SCC-based behavior analysis, etc. The only part of the CGSCC pass infrastructure not sorted out at this point are the updates in the face of inlining and running function passes that mutate the call graph. The changes are pretty boring and boiler-plate. Most of the work was factored into more focused preperatory patches. But this is what wires it all together. llvm-svn: 261203
2016-02-18 19:03:11 +08:00
MPM.add(createPostOrderFunctionAttrsLegacyPass());
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
if (OptLevel > 2)
MPM.add(createArgumentPromotionPass()); // Scalarize uninlined fn args
Add EP_CGSCCOptimizerLate extension point to PassManagerBuilder The EP_CGSCCOptimizerLate extension point allows adding CallGraphSCC passes at the end of the main CallGraphSCC passes and before any function simplification passes run by CGPassManager. Patch by Gor Nishanov! Differential Revision: https://reviews.llvm.org/D22897 llvm-svn: 276953
2016-07-28 11:28:43 +08:00
addExtensionsToPM(EP_CGSCCOptimizerLate, MPM);
Refactor the PassManagerBuilder: extract a "addFunctionSimplificationPasses()" (NFC) It is intended to contains the passes run over a function after the inliner is done with a function and before it moves to its callers. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261028
2016-02-17 06:54:27 +08:00
addFunctionSimplificationPasses(MPM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
Move createReversePostOrderFunctionAttrsPass right after the inliner is done This is where it was originally, until LoopVersioningLICM was inserted before in r259986, I don't believe it was on purpose. Differential Revision: http://reviews.llvm.org/D19809 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268252
2016-05-03 00:53:16 +08:00
// FIXME: This is a HACK! The inliner pass above implicitly creates a CGSCC
// pass manager that we are specifically trying to avoid. To prevent this
// we must insert a no-op module pass to reset the pass manager.
MPM.add(createBarrierNoopPass());
[PartialInlining] Add internal options to enable partial inlining in pass pipeline (off by default) 1. Legacy: -mllvm -enable-partial-inlining 2. New: -mllvm -enable-npm-partial-inlining -fexperimental-new-pass-manager Differential Revision: http://reviews.llvm.org/D33382 llvm-svn: 303567
2017-05-23 00:41:57 +08:00
if (RunPartialInlining)
MPM.add(createPartialInliningPass());
Move createReversePostOrderFunctionAttrsPass right after the inliner is done This is where it was originally, until LoopVersioningLICM was inserted before in r259986, I don't believe it was on purpose. Differential Revision: http://reviews.llvm.org/D19809 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268252
2016-05-03 00:53:16 +08:00
Move "Eliminate Available Externally" immediately after the inliner This pass is supposed to reduce the size of the IR for compile time purpose. We should run it ASAP, except when we prepare for LTO or ThinLTO, and we want to keep them available for link-time inline. Differential Revision: http://reviews.llvm.org/D19813 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268394
2016-05-03 23:46:00 +08:00
if (!DisableUnitAtATime && OptLevel > 1 && !PrepareForLTO &&
!PrepareForThinLTO)
// Remove avail extern fns and globals definitions if we aren't
// compiling an object file for later LTO. For LTO we want to preserve
// these so they are eligible for inlining at link-time. Note if they
// are unreferenced they will be removed by GlobalDCE later, so
// this only impacts referenced available externally globals.
// Eventually they will be suppressed during codegen, but eliminating
// here enables more opportunity for GlobalDCE as it may make
// globals referenced by available external functions dead
// and saves running remaining passes on the eliminated functions.
MPM.add(createEliminateAvailableExternallyPass());
Move createReversePostOrderFunctionAttrsPass right after the inliner is done This is where it was originally, until LoopVersioningLICM was inserted before in r259986, I don't believe it was on purpose. Differential Revision: http://reviews.llvm.org/D19809 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268252
2016-05-03 00:53:16 +08:00
if (!DisableUnitAtATime)
MPM.add(createReversePostOrderFunctionAttrsPass());
Define the ThinLTO Pipeline (experimental) Summary: On the contrary to Full LTO, ThinLTO can afford to shift compile time from the frontend to the linker: both phases are parallel (even if it is not totally "free": projects like clang are reusing product from the "compile phase" for multiple link, think about libLLVMSupport reused for opt, llc, etc.). This pipeline is based on the proposal in D13443 for full LTO. We didn't move forward on this proposal because the LTO link was far too long after that. We believe that we can afford it with ThinLTO. The ThinLTO pipeline integrates in the regular O2/O3 flow: - The compile phase perform the inliner with a somehow lighter function simplification. (TODO: tune the inliner thresholds here) This is intendend to simplify the IR and get rid of obvious things like linkonce_odr that will be inlined. - The link phase will run the pipeline from the start, extended with some specific passes that leverage the augmented knowledge we have during LTO. Especially after the inliner is done, a sequence of globalDCE/globalOpt is performed, followed by another run of the "function simplification" passes. It is not clear if this part of the pipeline will stay as is, as the split model of ThinLTO does not allow the same benefit as FullLTO without added tricks. The measurements on the public test suite as well as on our internal suite show an overall net improvement. The binary size for the clang executable is reduced by 5%. We're still tuning it with the bringup of ThinLTO and it will evolve, but this should provide a good starting point. Reviewers: tejohnson Differential Revision: http://reviews.llvm.org/D17115 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261029
2016-02-17 07:02:29 +08:00
// If we are planning to perform ThinLTO later, let's not bloat the code with
// unrolling/vectorization/... now. We'll first run the inliner + CGSCC passes
// during ThinLTO and perform the rest of the optimizations afterward.
ThinLTO: Move createNameAnonFunctionPass insertion in PassManagerBuilder (NFC) It is just code motion, but makes more sense this way. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 267384
2016-04-25 16:47:37 +08:00
if (PrepareForThinLTO) {
Run GlobalOpt before emitting the bitcode for ThinLTO This is motivated by reducing the size of the IR and thus reduce compile time. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 267385
2016-04-25 16:47:49 +08:00
// Reduce the size of the IR as much as possible.
MPM.add(createGlobalOptimizerPass());
Rename NameAnonFunctions to NameAnonGlobals to match what it is doing (NFC) llvm-svn: 281745
2016-09-17 00:56:30 +08:00
// Rename anon globals to be able to export them in the summary.
MPM.add(createNameAnonGlobalPass());
Define the ThinLTO Pipeline (experimental) Summary: On the contrary to Full LTO, ThinLTO can afford to shift compile time from the frontend to the linker: both phases are parallel (even if it is not totally "free": projects like clang are reusing product from the "compile phase" for multiple link, think about libLLVMSupport reused for opt, llc, etc.). This pipeline is based on the proposal in D13443 for full LTO. We didn't move forward on this proposal because the LTO link was far too long after that. We believe that we can afford it with ThinLTO. The ThinLTO pipeline integrates in the regular O2/O3 flow: - The compile phase perform the inliner with a somehow lighter function simplification. (TODO: tune the inliner thresholds here) This is intendend to simplify the IR and get rid of obvious things like linkonce_odr that will be inlined. - The link phase will run the pipeline from the start, extended with some specific passes that leverage the augmented knowledge we have during LTO. Especially after the inliner is done, a sequence of globalDCE/globalOpt is performed, followed by another run of the "function simplification" passes. It is not clear if this part of the pipeline will stay as is, as the split model of ThinLTO does not allow the same benefit as FullLTO without added tricks. The measurements on the public test suite as well as on our internal suite show an overall net improvement. The binary size for the clang executable is reduced by 5%. We're still tuning it with the bringup of ThinLTO and it will evolve, but this should provide a good starting point. Reviewers: tejohnson Differential Revision: http://reviews.llvm.org/D17115 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261029
2016-02-17 07:02:29 +08:00
return;
ThinLTO: Move createNameAnonFunctionPass insertion in PassManagerBuilder (NFC) It is just code motion, but makes more sense this way. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 267384
2016-04-25 16:47:37 +08:00
}
Define the ThinLTO Pipeline (experimental) Summary: On the contrary to Full LTO, ThinLTO can afford to shift compile time from the frontend to the linker: both phases are parallel (even if it is not totally "free": projects like clang are reusing product from the "compile phase" for multiple link, think about libLLVMSupport reused for opt, llc, etc.). This pipeline is based on the proposal in D13443 for full LTO. We didn't move forward on this proposal because the LTO link was far too long after that. We believe that we can afford it with ThinLTO. The ThinLTO pipeline integrates in the regular O2/O3 flow: - The compile phase perform the inliner with a somehow lighter function simplification. (TODO: tune the inliner thresholds here) This is intendend to simplify the IR and get rid of obvious things like linkonce_odr that will be inlined. - The link phase will run the pipeline from the start, extended with some specific passes that leverage the augmented knowledge we have during LTO. Especially after the inliner is done, a sequence of globalDCE/globalOpt is performed, followed by another run of the "function simplification" passes. It is not clear if this part of the pipeline will stay as is, as the split model of ThinLTO does not allow the same benefit as FullLTO without added tricks. The measurements on the public test suite as well as on our internal suite show an overall net improvement. The binary size for the clang executable is reduced by 5%. We're still tuning it with the bringup of ThinLTO and it will evolve, but this should provide a good starting point. Reviewers: tejohnson Differential Revision: http://reviews.llvm.org/D17115 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261029
2016-02-17 07:02:29 +08:00
Tweak the ThinLTO pass pipeline Summary: The original ThinLTO pipeline was derived from some work I did tuning FullLTO on the test suite and SPEC. This patch reduces the amount of work done in the "linker phase" of the build, and extend the function simplifications passes performed during the "compile phase". This helps the build time by reducing the IR as much as possible during the compile phase and limiting the work to be performed during the "link phase", while keeping the performance "on par" with the existing pipeline. Reviewers: tejohnson Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D19773 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268769
2016-05-07 02:17:03 +08:00
if (PerformThinLTO)
// Optimize globals now when performing ThinLTO, this enables more
// optimizations later.
MPM.add(createGlobalOptimizerPass());
Fixed typo in comment & coding style for LoopVersioningLICM. llvm-svn: 260504
2016-02-11 17:23:53 +08:00
// Scheduling LoopVersioningLICM when inlining is over, because after that
New Loop Versioning LICM Pass Summary: When alias analysis is uncertain about the aliasing between any two accesses, it will return MayAlias. This uncertainty from alias analysis restricts LICM from proceeding further. In cases where alias analysis is uncertain we might use loop versioning as an alternative. Loop Versioning will create a version of the loop with aggressive aliasing assumptions in addition to the original with conservative (default) aliasing assumptions. The version of the loop making aggressive aliasing assumptions will have all the memory accesses marked as no-alias. These two versions of loop will be preceded by a memory runtime check. This runtime check consists of bound checks for all unique memory accessed in loop, and it ensures the lack of memory aliasing. The result of the runtime check determines which of the loop versions is executed: If the runtime check detects any memory aliasing, then the original loop is executed. Otherwise, the version with aggressive aliasing assumptions is used. The pass is off by default and can be enabled with command line option -enable-loop-versioning-licm. Reviewers: hfinkel, anemet, chatur01, reames Subscribers: MatzeB, grosser, joker.eph, sanjoy, javed.absar, sbaranga, llvm-commits Differential Revision: http://reviews.llvm.org/D9151 llvm-svn: 259986
2016-02-06 15:47:48 +08:00
// we may see more accurate aliasing. Reason to run this late is that too
// early versioning may prevent further inlining due to increase of code
Tweak the ThinLTO pass pipeline Summary: The original ThinLTO pipeline was derived from some work I did tuning FullLTO on the test suite and SPEC. This patch reduces the amount of work done in the "linker phase" of the build, and extend the function simplifications passes performed during the "compile phase". This helps the build time by reducing the IR as much as possible during the compile phase and limiting the work to be performed during the "link phase", while keeping the performance "on par" with the existing pipeline. Reviewers: tejohnson Subscribers: llvm-commits, joker.eph Differential Revision: http://reviews.llvm.org/D19773 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 268769
2016-05-07 02:17:03 +08:00
// size. By placing it just after inlining other optimizations which runs
// later might get benefit of no-alias assumption in clone loop.
New Loop Versioning LICM Pass Summary: When alias analysis is uncertain about the aliasing between any two accesses, it will return MayAlias. This uncertainty from alias analysis restricts LICM from proceeding further. In cases where alias analysis is uncertain we might use loop versioning as an alternative. Loop Versioning will create a version of the loop with aggressive aliasing assumptions in addition to the original with conservative (default) aliasing assumptions. The version of the loop making aggressive aliasing assumptions will have all the memory accesses marked as no-alias. These two versions of loop will be preceded by a memory runtime check. This runtime check consists of bound checks for all unique memory accessed in loop, and it ensures the lack of memory aliasing. The result of the runtime check determines which of the loop versions is executed: If the runtime check detects any memory aliasing, then the original loop is executed. Otherwise, the version with aggressive aliasing assumptions is used. The pass is off by default and can be enabled with command line option -enable-loop-versioning-licm. Reviewers: hfinkel, anemet, chatur01, reames Subscribers: MatzeB, grosser, joker.eph, sanjoy, javed.absar, sbaranga, llvm-commits Differential Revision: http://reviews.llvm.org/D9151 llvm-svn: 259986
2016-02-06 15:47:48 +08:00
if (UseLoopVersioningLICM) {
MPM.add(createLoopVersioningLICMPass()); // Do LoopVersioningLICM
MPM.add(createLICMPass()); // Hoist loop invariants
}
[GMR] Add a late run of GlobalsModRef to the main pass pipeline behind the general GMR-in-non-LTO flag. Without this, we have the global information during the CGSCC pipeline for GVN and such, but don't have it available during the late loop optimizations such as the vectorizer. Moreover, after the CGSCC pipeline has finished we have substantially more accurate and refined call graph information, function annotations, etc, which will make GMR even more powerful than it is early in the pipelien. Note that we have to play silly games with preserving AliasAnalysis (which is now trivially preserved) in order to let a module analysis magically be preserved into the entire function pass pipeline. Simultaneously we have to not make GMR an immutable pass in order to be able to re-run it and collect fresh data on the final call graph. llvm-svn: 242999
2015-07-23 17:34:01 +08:00
if (EnableNonLTOGlobalsModRef)
// We add a fresh GlobalsModRef run at this point. This is particularly
// useful as the above will have inlined, DCE'ed, and function-attr
// propagated everything. We should at this point have a reasonably minimal
// and richly annotated call graph. By computing aliasing and mod/ref
// information for all local globals here, the late loop passes and notably
// the vectorizer will be able to use them to help recognize vectorizable
// memory operations.
//
// Note that this relies on a bug in the pass manager which preserves
// a module analysis into a function pass pipeline (and throughout it) so
// long as the first function pass doesn't invalidate the module analysis.
// Thus both Float2Int and LoopRotate have to preserve AliasAnalysis for
// this to work. Fortunately, it is trivial to preserve AliasAnalysis
// (doing nothing preserves it as it is required to be conservatively
// correct in the face of IR changes).
[PM/AA] Rebuild LLVM's alias analysis infrastructure in a way compatible with the new pass manager, and no longer relying on analysis groups. This builds essentially a ground-up new AA infrastructure stack for LLVM. The core ideas are the same that are used throughout the new pass manager: type erased polymorphism and direct composition. The design is as follows: - FunctionAAResults is a type-erasing alias analysis results aggregation interface to walk a single query across a range of results from different alias analyses. Currently this is function-specific as we always assume that aliasing queries are *within* a function. - AAResultBase is a CRTP utility providing stub implementations of various parts of the alias analysis result concept, notably in several cases in terms of other more general parts of the interface. This can be used to implement only a narrow part of the interface rather than the entire interface. This isn't really ideal, this logic should be hoisted into FunctionAAResults as currently it will cause a significant amount of redundant work, but it faithfully models the behavior of the prior infrastructure. - All the alias analysis passes are ported to be wrapper passes for the legacy PM and new-style analysis passes for the new PM with a shared result object. In some cases (most notably CFL), this is an extremely naive approach that we should revisit when we can specialize for the new pass manager. - BasicAA has been restructured to reflect that it is much more fundamentally a function analysis because it uses dominator trees and loop info that need to be constructed for each function. All of the references to getting alias analysis results have been updated to use the new aggregation interface. All the preservation and other pass management code has been updated accordingly. The way the FunctionAAResultsWrapperPass works is to detect the available alias analyses when run, and add them to the results object. This means that we should be able to continue to respect when various passes are added to the pipeline, for example adding CFL or adding TBAA passes should just cause their results to be available and to get folded into this. The exception to this rule is BasicAA which really needs to be a function pass due to using dominator trees and loop info. As a consequence, the FunctionAAResultsWrapperPass directly depends on BasicAA and always includes it in the aggregation. This has significant implications for preserving analyses. Generally, most passes shouldn't bother preserving FunctionAAResultsWrapperPass because rebuilding the results just updates the set of known AA passes. The exception to this rule are LoopPass instances which need to preserve all the function analyses that the loop pass manager will end up needing. This means preserving both BasicAAWrapperPass and the aggregating FunctionAAResultsWrapperPass. Now, when preserving an alias analysis, you do so by directly preserving that analysis. This is only necessary for non-immutable-pass-provided alias analyses though, and there are only three of interest: BasicAA, GlobalsAA (formerly GlobalsModRef), and SCEVAA. Usually BasicAA is preserved when needed because it (like DominatorTree and LoopInfo) is marked as a CFG-only pass. I've expanded GlobalsAA into the preserved set everywhere we previously were preserving all of AliasAnalysis, and I've added SCEVAA in the intersection of that with where we preserve SCEV itself. One significant challenge to all of this is that the CGSCC passes were actually using the alias analysis implementations by taking advantage of a pretty amazing set of loop holes in the old pass manager's analysis management code which allowed analysis groups to slide through in many cases. Moving away from analysis groups makes this problem much more obvious. To fix it, I've leveraged the flexibility the design of the new PM components provides to just directly construct the relevant alias analyses for the relevant functions in the IPO passes that need them. This is a bit hacky, but should go away with the new pass manager, and is already in many ways cleaner than the prior state. Another significant challenge is that various facilities of the old alias analysis infrastructure just don't fit any more. The most significant of these is the alias analysis 'counter' pass. That pass relied on the ability to snoop on AA queries at different points in the analysis group chain. Instead, I'm planning to build printing functionality directly into the aggregation layer. I've not included that in this patch merely to keep it smaller. Note that all of this needs a nearly complete rewrite of the AA documentation. I'm planning to do that, but I'd like to make sure the new design settles, and to flesh out a bit more of what it looks like in the new pass manager first. Differential Revision: http://reviews.llvm.org/D12080 llvm-svn: 247167
2015-09-10 01:55:00 +08:00
MPM.add(createGlobalsAAWrapperPass());
[GMR] Add a late run of GlobalsModRef to the main pass pipeline behind the general GMR-in-non-LTO flag. Without this, we have the global information during the CGSCC pipeline for GVN and such, but don't have it available during the late loop optimizations such as the vectorizer. Moreover, after the CGSCC pipeline has finished we have substantially more accurate and refined call graph information, function annotations, etc, which will make GMR even more powerful than it is early in the pipelien. Note that we have to play silly games with preserving AliasAnalysis (which is now trivially preserved) in order to let a module analysis magically be preserved into the entire function pass pipeline. Simultaneously we have to not make GMR an immutable pass in order to be able to re-run it and collect fresh data on the final call graph. llvm-svn: 242999
2015-07-23 17:34:01 +08:00
[PMBuilder] Remove RunFloat2Int cl::opt. The pass has been on by default for a long time without problems. llvm-svn: 290814
2017-01-03 01:49:18 +08:00
MPM.add(createFloat2IntPass());
Reapply r233175 and r233183: float2int. This re-adds float2int to the tree, after fixing PR23038. It turns out the argument to APSInt() is true-if-unsigned, rather than true-if-signed :(. Added testcase and explanatory comment. llvm-svn: 233370
2015-03-27 18:36:57 +08:00
Add PM extension point EP_VectorizerStart This extension point allows passes to be executed right before the vectorizer and other highly target specific optimizations are run. llvm-svn: 242389
2015-07-16 16:20:37 +08:00
addExtensionsToPM(EP_VectorizerStart, MPM);
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
// Re-rotate loops in all our loop nests. These may have fallout out of
// rotated form due to GVN or other transformations, and the vectorizer relies
Disable loop re-rotation for -Oz (patch by Andrey Turetsky) After changes in rL231820 loop re-rotation is performed even in -Oz mode. Since loop rotation is disabled for -Oz, it seems loop re-rotation should be disabled too. Differential Revision: http://reviews.llvm.org/D10961 llvm-svn: 241897
2015-07-10 18:37:09 +08:00
// on the rotated form. Disable header duplication at -Oz.
MPM.add(createLoopRotatePass(SizeLevel == 2 ? 0 : -1));
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
New Loop Distribution pass Summary: This implements the initial version as was proposed earlier this year (http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-January/080462.html). Since then Loop Access Analysis was split out from the Loop Vectorizer and was made into a separate analysis pass. Loop Distribution becomes the second user of this analysis. The pass is off by default and can be enabled with -enable-loop-distribution. There is currently no notion of profitability; if there is a loop with dependence cycles, the pass will try to split them off from other memory operations into a separate loop. I decided to remove the control-dependence calculation from this first version. This and the issues with the PDT are actively discussed so it probably makes sense to treat it separately. Right now I just mark all terminator instruction required which keeps identical CFGs for each distributed loop. This seems to be working pretty well for 456.hmmer where even though there is an empty if-then block in the distributed loop initially, it gets completely removed. The pass keeps DominatorTree and LoopInfo updated. I've tested this with -loop-distribute-verify with the testsuite where we distribute ~90 loops. SimplifyLoop is violated in some cases and I have a FIXME covering this. Reviewers: hfinkel, nadav, aschwaighofer Reviewed By: aschwaighofer Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D8831 llvm-svn: 237358
2015-05-14 20:05:18 +08:00
// Distribute loops to allow partial vectorization. I.e. isolate dependences
[LoopDist] Add llvm.loop.distribute.enable loop metadata Summary: D19403 adds a new pragma for loop distribution. This change adds support for the corresponding metadata that the pragma is translated to by the FE. As part of this I had to rethink the flag -enable-loop-distribute. My goal was to be backward compatible with the existing behavior: A1. pass is off by default from the optimization pipeline unless -enable-loop-distribute is specified A2. pass is on when invoked directly from opt (e.g. for unit-testing) The new pragma/metadata overrides these defaults so the new behavior is: B1. A1 + enable distribution for individual loop with the pragma/metadata B2. A2 + disable distribution for individual loop with the pragma/metadata The default value whether the pass is on or off comes from the initiator of the pass. From the PassManagerBuilder the default is off, from opt it's on. I moved -enable-loop-distribute under the pass. If the flag is specified it overrides the default from above. Then the pragma/metadata can further modifies this per loop. As a side-effect, we can now also use -enable-loop-distribute=0 from opt to emulate the default from the optimization pipeline. So to be precise this is the new behavior: C1. pass is off by default from the optimization pipeline unless -enable-loop-distribute or the pragma/metadata enables it C2. pass is on when invoked directly from opt unless -enable-loop-distribute=0 or the pragma/metadata disables it Reviewers: hfinkel Subscribers: joker.eph, mzolotukhin, llvm-commits Differential Revision: http://reviews.llvm.org/D19431 llvm-svn: 267672
2016-04-27 13:28:18 +08:00
// into separate loop that would otherwise inhibit vectorization. This is
// currently only performed for loops marked with the metadata
// llvm.loop.distribute=true or when -enable-loop-distribute is specified.
[LDist] Match behavior between invoking via optimization pipeline or opt -loop-distribute In r267672, where the loop distribution pragma was introduced, I tried it hard to keep the old behavior for opt: when opt is invoked with -loop-distribute, it should distribute the loop (it's off by default when ran via the optimization pipeline). As MichaelZ has discovered this has the unintended consequence of breaking a very common developer work-flow to reproduce compilations using opt: First you print the pass pipeline of clang with -debug-pass=Arguments and then invoking opt with the returned arguments. clang -debug-pass will include -loop-distribute but the pass is invoked with default=off so nothing happens unless the loop carries the pragma. While through opt (default=on) we will try to distribute all loops. This changes opt's default to off as well to match clang. The tests are modified to explicitly enable the transformation. llvm-svn: 290235
2016-12-21 12:07:40 +08:00
MPM.add(createLoopDistributePass());
New Loop Distribution pass Summary: This implements the initial version as was proposed earlier this year (http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-January/080462.html). Since then Loop Access Analysis was split out from the Loop Vectorizer and was made into a separate analysis pass. Loop Distribution becomes the second user of this analysis. The pass is off by default and can be enabled with -enable-loop-distribution. There is currently no notion of profitability; if there is a loop with dependence cycles, the pass will try to split them off from other memory operations into a separate loop. I decided to remove the control-dependence calculation from this first version. This and the issues with the PDT are actively discussed so it probably makes sense to treat it separately. Right now I just mark all terminator instruction required which keeps identical CFGs for each distributed loop. This seems to be working pretty well for 456.hmmer where even though there is an empty if-then block in the distributed loop initially, it gets completely removed. The pass keeps DominatorTree and LoopInfo updated. I've tested this with -loop-distribute-verify with the testsuite where we distribute ~90 loops. SimplifyLoop is violated in some cases and I have a FIXME covering this. Reviewers: hfinkel, nadav, aschwaighofer Reviewed By: aschwaighofer Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D8831 llvm-svn: 237358
2015-05-14 20:05:18 +08:00
Add #pragma vectorize enable/disable to LLVM The intended behaviour is to force vectorization on the presence of the flag (either turn on or off), and to continue the behaviour as expected in its absence. Tests were added to make sure the all cases are covered in opt. No tests were added in other tools with the assumption that they should use the PassManagerBuilder in the same way. This patch also removes the outdated -late-vectorize flag, which was on by default and not helping much. The pragma metadata is being attached to the same place as other loop metadata, but nothing forbids one from attaching it to a function (to enable #pragma optimize) or basic blocks (to hint the basic-block vectorizers), etc. The logic should be the same all around. Patches to Clang to produce the metadata will be produced after the initial implementation is agreed upon and committed. Patches to other vectorizers (such as SLP and BB) will be added once we're happy with the pass manager changes. llvm-svn: 196537
2013-12-06 05:20:02 +08:00
MPM.add(createLoopVectorizePass(DisableUnrollLoops, LoopVectorize));
LLE 6/6: Add LoopLoadElimination pass Summary: The goal of this pass is to perform store-to-load forwarding across the backedge of a loop. E.g.: for (i) A[i + 1] = A[i] + B[i] => T = A[0] for (i) T = T + B[i] A[i + 1] = T The pass relies on loop dependence analysis via LoopAccessAnalisys to find opportunities of loop-carried dependences with a distance of one between a store and a load. Since it's using LoopAccessAnalysis, it was easy to also add support for versioning away may-aliasing intervening stores that would otherwise prevent this transformation. This optimization is also performed by Load-PRE in GVN without the option of multi-versioning. As was discussed with Daniel Berlin in http://reviews.llvm.org/D9548, this is inferior to a more loop-aware solution applied here. Hopefully, we will be able to remove some complexity from GVN/MemorySSA as a consequence. In the long run, we may want to extend this pass (or create a new one if there is little overlap) to also eliminate loop-indepedent redundant loads and store that *require* versioning due to may-aliasing intervening stores/loads. I have some motivating cases for store elimination. My plan right now is to wait for MemorySSA to come online first rather than using memdep for this. The main motiviation for this pass is the 456.hmmer loop in SPECint2006 where after distributing the original loop and vectorizing the top part, we are left with the critical path exposed in the bottom loop. Being able to promote the memory dependence into a register depedence (even though the HW does perform store-to-load fowarding as well) results in a major gain (~20%). This gain also transfers over to x86: it's around 8-10%. Right now the pass is off by default and can be enabled with -enable-loop-load-elim. On the LNT testsuite, there are two performance changes (negative number -> improvement): 1. -28% in Polybench/linear-algebra/solvers/dynprog: the length of the critical paths is reduced 2. +2% in Polybench/stencils/adi: Unfortunately, I couldn't reproduce this outside of LNT The pass is scheduled after the loop vectorizer (which is after loop distribution). The rational is to try to reuse LAA state, rather than recomputing it. The order between LV and LLE is not critical because normally LV does not touch scalar st->ld forwarding cases where vectorizing would inhibit the CPU's st->ld forwarding to kick in. LoopLoadElimination requires LAA to provide the full set of dependences (including forward dependences). LAA is known to omit loop-independent dependences in certain situations. The big comment before removeDependencesFromMultipleStores explains why this should not occur for the cases that we're interested in. Reviewers: dberlin, hfinkel Subscribers: junbuml, dberlin, mssimpso, rengolin, sanjoy, llvm-commits Differential Revision: http://reviews.llvm.org/D13259 llvm-svn: 252017
2015-11-04 07:50:08 +08:00
// Eliminate loads by forwarding stores from the previous iteration to loads
// of the current iteration.
if (EnableLoopLoadElim)
MPM.add(createLoopLoadEliminationPass());
Add #pragma vectorize enable/disable to LLVM The intended behaviour is to force vectorization on the presence of the flag (either turn on or off), and to continue the behaviour as expected in its absence. Tests were added to make sure the all cases are covered in opt. No tests were added in other tools with the assumption that they should use the PassManagerBuilder in the same way. This patch also removes the outdated -late-vectorize flag, which was on by default and not helping much. The pragma metadata is being attached to the same place as other loop metadata, but nothing forbids one from attaching it to a function (to enable #pragma optimize) or basic blocks (to hint the basic-block vectorizers), etc. The logic should be the same all around. Patches to Clang to produce the metadata will be produced after the initial implementation is agreed upon and committed. Patches to other vectorizers (such as SLP and BB) will be added once we're happy with the pass manager changes. llvm-svn: 196537
2013-12-06 05:20:02 +08:00
// FIXME: Because of #pragma vectorize enable, the passes below are always
// inserted in the pipeline, even when the vectorizer doesn't run (ex. when
// on -O1 and no #pragma is found). Would be good to have these two passes
// as function calls, so that we can only pass them when the vectorizer
// changed the code.
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
if (OptLevel > 1 && ExtraVectorizerPasses) {
// At higher optimization levels, try to clean up any runtime overlap and
// alignment checks inserted by the vectorizer. We want to track correllated
// runtime checks for two inner loops in the same outer loop, fold any
// common computations, hoist loop-invariant aspects out of any outer loop,
// and unswitch the runtime checks if possible. Once hoisted, we may have
// dead (or speculatable) control flows or more combining opportunities.
MPM.add(createEarlyCSEPass());
MPM.add(createCorrelatedValuePropagationPass());
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
MPM.add(createLICMPass());
Only unswitch loops with uniform conditions Loop unswitching can be extremely harmful for a SIMT target. In case if hoisted condition is not uniform a SIMT machine will execute both clones of a loop sequentially. Therefor LoopUnswitch checks if the condition is non-divergent. Since DivergenceAnalysis adds an expensive PostDominatorTree analysis not needed for non-SIMT targets a new option is added to avoid unneded analysis initialization. The method getAnalysisUsage is called when TargetTransformInfo is not yet available and we cannot use it here. For that reason a new field DivergentTarget is added to PassManagerBuilder to control the behavior and set this field from a target. Differential Revision: https://reviews.llvm.org/D30796 llvm-svn: 298104
2017-03-18 01:13:41 +08:00
MPM.add(createLoopUnswitchPass(SizeLevel || OptLevel < 3, DivergentTarget));
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
MPM.add(createCFGSimplificationPass());
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
}
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
if (RunSLPAfterLoopVectorization) {
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
if (SLPVectorize) {
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
MPM.add(createSLPVectorizerPass()); // Vectorize parallel scalar chains.
Add some optional passes around the vectorizer to both better prepare the IR going into it and to clean up the IR produced by the vectorizers. Note that these are *off by default* right now while folks collect data on whether the performance tradeoff is reasonable. In a build of the 'opt' binary, I see about 2% compile time regression due to this change on average. This is in my mind essentially the worst expected case: very little of the opt binary is going to *benefit* from these extra passes. I've seen several benchmarks improve in performance my small amounts due to running these passes, and there are certain (rare) cases where these passes make a huge difference by either enabling the vectorizer at all or by hoisting runtime checks out of the outer loop. My primary motivation is to prevent people from seeing runtime check overhead in benchmarks where the existing passes and optimizers would be able to eliminate that. I've chosen the sequence of passes based on the kinds of things that seem likely to be relevant for the code at each stage: rotaing loops for the vectorizer, finding correlated values, loop invariants, and unswitching opportunities from any runtime checks, and cleaning up commonalities exposed by the SLP vectorizer. I'll be pinging existing threads where some of these issues have come up and will start new threads to get folks to benchmark and collect data on whether this is the right tradeoff or we should do something else. llvm-svn: 219644
2014-10-14 08:31:29 +08:00
if (OptLevel > 1 && ExtraVectorizerPasses) {
MPM.add(createEarlyCSEPass());
}
}
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
if (BBVectorize) {
MPM.add(createBBVectorizePass());
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
addExtensionsToPM(EP_Peephole, MPM);
if (OptLevel > 1 && UseGVNAfterVectorization)
[NewGVN] Add a flag to enable the pass via `-mllvm`. NewGVN can be tested passing `-mllvm -enable-newgvn` to clang. Differential Revision: https://reviews.llvm.org/D28059 llvm-svn: 290548
2016-12-27 02:26:19 +08:00
MPM.add(NewGVN
? createNewGVNPass()
: createGVNPass(DisableGVNLoadPRE)); // Remove redundancies
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
else
MPM.add(createEarlyCSEPass()); // Catch trivial redundancies
// BBVectorize may have significantly shortened a loop body; unroll again.
if (!DisableUnrollLoops)
Increases full-unroll threshold. Summary: The default threshold for fully unroll is too conservative. This patch doubles the full-unroll threshold This change will affect the following speccpu2006 benchmarks (performance numbers were collected from Intel Sandybridge): Performance: 403 0.11% 433 0.51% 445 0.48% 447 3.50% 453 1.49% 464 0.75% Code size: 403 0.56% 433 0.96% 445 2.16% 447 2.96% 453 0.94% 464 8.02% The compiler time overhead is similar with code size. Reviewers: davidxl, mkuper, mzolotukhin, hfinkel, chandlerc Reviewed By: hfinkel, chandlerc Subscribers: mehdi_amini, zzheng, efriedma, haicheng, hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D28368 llvm-svn: 295538
2017-02-18 11:46:51 +08:00
MPM.add(createLoopUnrollPass(OptLevel));
Add a new option -run-slp-after-loop-vectorization. This swaps the order of the loop vectorizer and the SLP/BB vectorizers. It is disabled by default so we can do performance testing - ideally we want to change to having the loop vectorizer running first, and the SLP vectorizer using its leftovers instead of the other way around. llvm-svn: 214963
2014-08-06 20:56:19 +08:00
}
}
Add an extension point for peephole optimizers. This extension point allows adding passes that perform peephole optimizations similar to the instruction combiner. These passes will be inserted after each instance of the instruction combiner pass. Differential Revision: http://reviews.llvm.org/D3905 llvm-svn: 209595
2014-05-25 18:27:02 +08:00
addExtensionsToPM(EP_Peephole, MPM);
Split the SimplifyCFG pass into two variants. The first variant contains all current transformations except transforming switches into lookup tables. The second variant contains all current transformations. The switch-to-lookup-table conversion results in code that is more difficult to analyze and optimize by other passes. Most importantly, it can inhibit Dead Code Elimination. As such it is often beneficial to only apply this transformation very late. A common example is inlining, which can often result in range restrictions for the switch expression. Changes in execution time according to LNT: SingleSource/Benchmarks/Misc/fp-convert +3.03% MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk -11.20% MultiSource/Benchmarks/Olden/perimeter/perimeter -10.43% and a couple of smaller changes. For perimeter it also results 2.6% a smaller binary. Differential Revision: https://reviews.llvm.org/D30333 llvm-svn: 298799
2017-03-26 14:44:08 +08:00
MPM.add(createLateCFGSimplificationPass()); // Switches to lookup tables
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add a flag to defer vectorization into a phase after the inliner and its CGSCC pass manager. This should insulate the inlining decisions from the vectorization decisions, however it may have both compile time and code size problems so it is just an experimental option right now. Adding this based on a discussion with Arnold and it seems at least worth having this flag for us to both run some experiments to see if this strategy is workable. It may solve some of the regressions seen with the loop vectorizer. llvm-svn: 184698
2013-06-24 15:21:47 +08:00
Reapply 'Run LICM pass after loop unrolling pass.' It's firstly committed at r231630, and reverted at r231635. Function pass InstructionSimplifier is inserted as barrier to make sure loop unroll pass won't affect on LICM pass. llvm-svn: 232011
2015-03-12 13:36:01 +08:00
if (!DisableUnrollLoops) {
Increases full-unroll threshold. Summary: The default threshold for fully unroll is too conservative. This patch doubles the full-unroll threshold This change will affect the following speccpu2006 benchmarks (performance numbers were collected from Intel Sandybridge): Performance: 403 0.11% 433 0.51% 445 0.48% 447 3.50% 453 1.49% 464 0.75% Code size: 403 0.56% 433 0.96% 445 2.16% 447 2.96% 453 0.94% 464 8.02% The compiler time overhead is similar with code size. Reviewers: davidxl, mkuper, mzolotukhin, hfinkel, chandlerc Reviewed By: hfinkel, chandlerc Subscribers: mehdi_amini, zzheng, efriedma, haicheng, hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D28368 llvm-svn: 295538
2017-02-18 11:46:51 +08:00
MPM.add(createLoopUnrollPass(OptLevel)); // Unroll small loops
Move partial/runtime unrolling late in the pipeline The generic (concatenation) loop unroller is currently placed early in the standard optimization pipeline. This is a good place to perform full unrolling, but not the right place to perform partial/runtime unrolling. However, most targets don't enable partial/runtime unrolling, so this never mattered. However, even some x86 cores benefit from partial/runtime unrolling of very small loops, and follow-up commits will enable this. First, we need to move partial/runtime unrolling late in the optimization pipeline (importantly, this is after SLP and loop vectorization, as vectorization can drastically change the size of a loop), while keeping the full unrolling where it is now. This change does just that. llvm-svn: 205264
2014-04-01 07:23:51 +08:00
Add another InstCombine pass after LoopUnroll. This is to cleanup some redundency generated by LoopUnroll pass. Such redundency may not be cleaned up by existing passes after LoopUnroll. Differential Revision: http://reviews.llvm.org/D9777 llvm-svn: 237395
2015-05-15 06:02:54 +08:00
// LoopUnroll may generate some redundency to cleanup.
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(MPM);
Add another InstCombine pass after LoopUnroll. This is to cleanup some redundency generated by LoopUnroll pass. Such redundency may not be cleaned up by existing passes after LoopUnroll. Differential Revision: http://reviews.llvm.org/D9777 llvm-svn: 237395
2015-05-15 06:02:54 +08:00
Reapply 'Run LICM pass after loop unrolling pass.' It's firstly committed at r231630, and reverted at r231635. Function pass InstructionSimplifier is inserted as barrier to make sure loop unroll pass won't affect on LICM pass. llvm-svn: 232011
2015-03-12 13:36:01 +08:00
// Runtime unrolling will introduce runtime check in loop prologue. If the
// unrolled loop is a inner loop, then the prologue will be inside the
// outer loop. LICM pass can help to promote the runtime check out if the
// checked value is loop invariant.
MPM.add(createLICMPass());
Enable Loop Sink pass for functions that has profile. Summary: For functions with profile data, we are confident that loop sink will be optimal in sinking code. Reviewers: davidxl, hfinkel Subscribers: mehdi_amini, mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D26155 llvm-svn: 286325
2016-11-09 08:58:19 +08:00
}
Reapply 'Run LICM pass after loop unrolling pass.' It's firstly committed at r231630, and reverted at r231635. Function pass InstructionSimplifier is inserted as barrier to make sure loop unroll pass won't affect on LICM pass. llvm-svn: 232011
2015-03-12 13:36:01 +08:00
Add an AlignmentFromAssumptions Pass This adds a ScalarEvolution-powered transformation that updates load, store and memory intrinsic pointer alignments based on invariant((a+q) & b == 0) expressions. Many of the simple cases we can get with ValueTracking, but we still need something like this for the more complicated cases (such as those with an offset) that require some algebra. Note that gcc's __builtin_assume_aligned's optional third argument provides exactly for this kind of 'misalignment' offset for which this kind of logic is necessary. The primary motivation is to fixup alignments for vector loads/stores after vectorization (and unrolling). This pass is added to the optimization pipeline just after the SLP vectorizer runs (which, admittedly, does not preserve SE, although I imagine it could). Regardless, I actually don't think that the preservation matters too much in this case: SE computes lazily, and this pass won't issue any SE queries unless there are any assume intrinsics, so there should be no real additional cost in the common case (SLP does preserve DT and LoopInfo). llvm-svn: 217344
2014-09-08 04:05:11 +08:00
// After vectorization and unrolling, assume intrinsics may tell us more
// about pointer alignments.
MPM.add(createAlignmentFromAssumptionsPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
if (!DisableUnitAtATime) {
// FIXME: We shouldn't bother with this anymore.
MPM.add(createStripDeadPrototypesPass()); // Get rid of dead prototypes
GlobalDCE should be run at -O2 / -Os to eliminate unused dtor, etc. rdar://9142819 llvm-svn: 164850
2012-09-29 05:23:26 +08:00
// GlobalOpt already deletes dead functions and globals, at -O2 try a
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// late pass of GlobalDCE. It is capable of deleting dead cycles.
GlobalDCE should be run at -O2 / -Os to eliminate unused dtor, etc. rdar://9142819 llvm-svn: 164850
2012-09-29 05:23:26 +08:00
if (OptLevel > 1) {
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
MPM.add(createGlobalDCEPass()); // Remove dead fns and globals.
MPM.add(createConstantMergePass()); // Merge dup global constants
GlobalDCE should be run at -O2 / -Os to eliminate unused dtor, etc. rdar://9142819 llvm-svn: 164850
2012-09-29 05:23:26 +08:00
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
}
Add control of function merging to the PMBuilder. llvm-svn: 217731
2014-09-14 05:46:00 +08:00
if (MergeFunctions)
MPM.add(createMergeFunctionsPass());
Add comments about why we put LoopSink pass at the very late stage. llvm-svn: 286480
2016-11-11 01:42:18 +08:00
// LoopSink pass sinks instructions hoisted by LICM, which serves as a
// canonicalization pass that enables other optimizations. As a result,
// LoopSink pass needs to be a very late IR pass to avoid undoing LICM
// result too early.
Enable Loop Sink pass for functions that has profile. Summary: For functions with profile data, we are confident that loop sink will be optimal in sinking code. Reviewers: davidxl, hfinkel Subscribers: mehdi_amini, mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D26155 llvm-svn: 286325
2016-11-09 08:58:19 +08:00
MPM.add(createLoopSinkPass());
// Get rid of LCSSA nodes.
MPM.add(createInstructionSimplifierPass());
Simplify the CFG after loop pass cleanup. Summary: Otherwise we might end up with some empty basic blocks or single-entry-single-exit basic blocks. This fixes PR32085 Reviewers: chandlerc, danielcdh Subscribers: mehdi_amini, RKSimon, llvm-commits Differential Revision: https://reviews.llvm.org/D30468 llvm-svn: 301395
2017-04-26 20:02:41 +08:00
// LoopSink (and other loop passes since the last simplifyCFG) might have
// resulted in single-entry-single-exit or empty blocks. Clean up the CFG.
MPM.add(createCFGSimplificationPass());
add EP_OptimizerLast extension point llvm-svn: 153353
2012-03-24 07:22:59 +08:00
addExtensionsToPM(EP_OptimizerLast, MPM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
}
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
void PassManagerBuilder::addLTOOptimizationPasses(legacy::PassManagerBase &PM) {
Move whole-program virtual call optimization pass after function attribute inference in LTO pipeline. As a result of D18634 we no longer infer certain attributes on linkonce_odr functions at compile time, and may only infer them at LTO time. The readnone attribute in particular is required for virtual constant propagation (part of whole-program virtual call optimization) to work correctly. This change moves the whole-program virtual call optimization pass after the function attribute inference passes, and enables the attribute inference passes at opt level 1, so that virtual constant propagation has a chance to work correctly for linkonce_odr functions. Differential Revision: http://reviews.llvm.org/D20643 llvm-svn: 270765
2016-05-26 05:26:14 +08:00
// Remove unused virtual tables to improve the quality of code generated by
// whole-program devirtualization and bitset lowering.
PM.add(createGlobalDCEPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Provide AliasAnalysis services for optimizations.
addInitialAliasAnalysisPasses(PM);
[attrs] Split off the forced attributes utility into its own pass that is (by default) run much earlier than FuncitonAttrs proper. This allows forcing optnone or other widely impactful attributes. It is also a bit simpler as the force attribute behavior needs no specific iteration order. I've added the pass into the default module pass pipeline and LTO pass pipeline which mirrors where function attrs itself was being run. Differential Revision: http://reviews.llvm.org/D15668 llvm-svn: 256465
2015-12-27 16:13:45 +08:00
// Allow forcing function attributes as a debugging and tuning aid.
PM.add(createForceFunctionAttrsLegacyPass());
[attrs] Extract the pure inference of function attributes into a standalone pass. There is no call graph or even interesting analysis for this part of function attributes -- it is literally inferring attributes based on the target library identification. As such, we can do it using a much simpler module pass that just walks the declarations. This can also happen much earlier in the pass pipeline which has benefits for any number of other passes. In the process, I've cleaned up one particular aspect of the logic which was necessary in order to separate the two passes cleanly. It now counts inferred attributes independently rather than just counting all the inferred attributes as one, and the counts are more clearly explained. The two test cases we had for this code path are both ... woefully inadequate and copies of each other. I've kept the superset test and updated it. We need more testing here, but I had to pick somewhere to stop fixing everything broken I saw here. Differential Revision: http://reviews.llvm.org/D15676 llvm-svn: 256466
2015-12-27 16:41:34 +08:00
// Infer attributes about declarations if possible.
PM.add(createInferFunctionAttrsLegacyPass());
Move whole-program virtual call optimization pass after function attribute inference in LTO pipeline. As a result of D18634 we no longer infer certain attributes on linkonce_odr functions at compile time, and may only infer them at LTO time. The readnone attribute in particular is required for virtual constant propagation (part of whole-program virtual call optimization) to work correctly. This change moves the whole-program virtual call optimization pass after the function attribute inference passes, and enables the attribute inference passes at opt level 1, so that virtual constant propagation has a chance to work correctly for linkonce_odr functions. Differential Revision: http://reviews.llvm.org/D20643 llvm-svn: 270765
2016-05-26 05:26:14 +08:00
if (OptLevel > 1) {
// Indirect call promotion. This should promote all the targets that are
// left by the earlier promotion pass that promotes intra-module targets.
// This two-step promotion is to save the compile time. For LTO, it should
// produce the same result as if we only do promotion here.
Add call branch annotation for ICP promoted direct call in SamplePGO mode. Summary: SamplePGO uses branch_weight annotation to represent callsite hotness. When ICP promotes an indirect call to direct call, we need to make sure the direct call is annotated with branch_weight in SamplePGO mode, so that downstream function inliner can use hot callsite heuristic. Reviewers: davidxl, eraman, xur Reviewed By: davidxl, xur Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D30282 llvm-svn: 296028
2017-02-24 06:15:18 +08:00
PM.add(
createPGOIndirectCallPromotionLegacyPass(true, !PGOSampleUse.empty()));
Move whole-program virtual call optimization pass after function attribute inference in LTO pipeline. As a result of D18634 we no longer infer certain attributes on linkonce_odr functions at compile time, and may only infer them at LTO time. The readnone attribute in particular is required for virtual constant propagation (part of whole-program virtual call optimization) to work correctly. This change moves the whole-program virtual call optimization pass after the function attribute inference passes, and enables the attribute inference passes at opt level 1, so that virtual constant propagation has a chance to work correctly for linkonce_odr functions. Differential Revision: http://reviews.llvm.org/D20643 llvm-svn: 270765
2016-05-26 05:26:14 +08:00
// Propagate constants at call sites into the functions they call. This
// opens opportunities for globalopt (and inlining) by substituting function
// pointers passed as arguments to direct uses of functions.
PM.add(createIPSCCPPass());
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
Move whole-program virtual call optimization pass after function attribute inference in LTO pipeline. As a result of D18634 we no longer infer certain attributes on linkonce_odr functions at compile time, and may only infer them at LTO time. The readnone attribute in particular is required for virtual constant propagation (part of whole-program virtual call optimization) to work correctly. This change moves the whole-program virtual call optimization pass after the function attribute inference passes, and enables the attribute inference passes at opt level 1, so that virtual constant propagation has a chance to work correctly for linkonce_odr functions. Differential Revision: http://reviews.llvm.org/D20643 llvm-svn: 270765
2016-05-26 05:26:14 +08:00
// Infer attributes about definitions. The readnone attribute in particular is
// required for virtual constant propagation.
[PM] Port the PostOrderFunctionAttrs pass to the new pass manager and convert one test to use this. This is a particularly significant milestone because it required a working per-function AA framework which can be queried over each function from within a CGSCC transform pass (and additionally a module analysis to be accessible). This is essentially *the* point of the entire pass manager rewrite. A CGSCC transform is able to query for multiple different function's analysis results. It works. The whole thing appears to actually work and accomplish the original goal. While we were able to hack function attrs and basic-aa to "work" in the old pass manager, this port doesn't use any of that, it directly leverages the new fundamental functionality. For this to work, the CGSCC framework also has to support SCC-based behavior analysis, etc. The only part of the CGSCC pass infrastructure not sorted out at this point are the updates in the face of inlining and running function passes that mutate the call graph. The changes are pretty boring and boiler-plate. Most of the work was factored into more focused preperatory patches. But this is what wires it all together. llvm-svn: 261203
2016-02-18 19:03:11 +08:00
PM.add(createPostOrderFunctionAttrsLegacyPass());
[attrs] Split the late-revisit pattern for deducing norecurse in a top-down manner into a true top-down or RPO pass over the call graph. There are specific patterns of function attributes, notably the norecurse attribute, which are most effectively propagated top-down because all they us caller information. Walk in RPO over the call graph SCCs takes the form of a module pass run immediately after the CGSCC pass managers postorder walk of the SCCs, trying again to deduce norerucrse for each singular SCC in the call graph. This removes a very legacy pass manager specific trick of using a lazy revisit list traversed during finalization of the CGSCC pass. There is no analogous finalization step in the new pass manager, and a lazy revisit list is just trying to produce an RPO iteration of the call graph. We can do that more directly if more expensively. It seems unlikely that this will be the expensive part of any compilation though as we never examine the function bodies here. Even in an LTO run over a very large module, this should be a reasonable fast set of operations over a reasonably small working set -- the function call graph itself. In the future, if this really is a compile time performance issue, we can look at building support for both post order and RPO traversals directly into a pass manager that builds and maintains the PO list of SCCs. Differential Revision: http://reviews.llvm.org/D15785 llvm-svn: 257163
2016-01-08 18:55:52 +08:00
PM.add(createReversePostOrderFunctionAttrsPass());
Move whole-program virtual call optimization pass after function attribute inference in LTO pipeline. As a result of D18634 we no longer infer certain attributes on linkonce_odr functions at compile time, and may only infer them at LTO time. The readnone attribute in particular is required for virtual constant propagation (part of whole-program virtual call optimization) to work correctly. This change moves the whole-program virtual call optimization pass after the function attribute inference passes, and enables the attribute inference passes at opt level 1, so that virtual constant propagation has a chance to work correctly for linkonce_odr functions. Differential Revision: http://reviews.llvm.org/D20643 llvm-svn: 270765
2016-05-26 05:26:14 +08:00
Introduce GlobalSplit pass. This pass splits globals into elements using inrange annotations on getelementptr indices. Differential Revision: https://reviews.llvm.org/D22295 llvm-svn: 287178
2016-11-17 07:40:26 +08:00
// Split globals using inrange annotations on GEP indices. This can help
// improve the quality of generated code when virtual constant propagation or
// control flow integrity are enabled.
PM.add(createGlobalSplitPass());
Move whole-program virtual call optimization pass after function attribute inference in LTO pipeline. As a result of D18634 we no longer infer certain attributes on linkonce_odr functions at compile time, and may only infer them at LTO time. The readnone attribute in particular is required for virtual constant propagation (part of whole-program virtual call optimization) to work correctly. This change moves the whole-program virtual call optimization pass after the function attribute inference passes, and enables the attribute inference passes at opt level 1, so that virtual constant propagation has a chance to work correctly for linkonce_odr functions. Differential Revision: http://reviews.llvm.org/D20643 llvm-svn: 270765
2016-05-26 05:26:14 +08:00
// Apply whole-program devirtualization and virtual constant propagation.
IPO: Const correctness for summaries passed into passes. Pass const qualified summaries into importers and unqualified summaries into exporters. This lets us const-qualify the summary argument to thinBackend. Differential Revision: https://reviews.llvm.org/D31230 llvm-svn: 298534
2017-03-23 02:22:59 +08:00
PM.add(createWholeProgramDevirtPass(ExportSummary, nullptr));
Move whole-program virtual call optimization pass after function attribute inference in LTO pipeline. As a result of D18634 we no longer infer certain attributes on linkonce_odr functions at compile time, and may only infer them at LTO time. The readnone attribute in particular is required for virtual constant propagation (part of whole-program virtual call optimization) to work correctly. This change moves the whole-program virtual call optimization pass after the function attribute inference passes, and enables the attribute inference passes at opt level 1, so that virtual constant propagation has a chance to work correctly for linkonce_odr functions. Differential Revision: http://reviews.llvm.org/D20643 llvm-svn: 270765
2016-05-26 05:26:14 +08:00
// That's all we need at opt level 1.
if (OptLevel == 1)
return;
// Now that we internalized some globals, see if we can hack on them!
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
PM.add(createGlobalOptimizerPass());
[PassManagerBuilder] Add a few more scalar optimization passes This patch does two things: 1. mem2reg is now run immediately after globalopt. Now that globalopt can localize variables more aggressively, it makes sense to lower them to SSA form earlier rather than later so they can benefit from the full set of optimization passes. 2. More scalar optimizations are run after the loop optimizations in LTO mode. The loop optimizations (especially indvars) can clean up scalar code sufficiently to make it worthwhile running more scalar passes. I've particularly added SCCP here as it isn't run anywhere else in the LTO pass pipeline. Mem2reg is super cheap and shouldn't affect compilation time at all. The rest of the added passes are in the LTO pipeline only so doesn't affect the vast majority of compilations, just the link step. llvm-svn: 255634
2015-12-15 17:24:01 +08:00
// Promote any localized global vars.
PM.add(createPromoteMemoryToRegisterPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Linking modules together can lead to duplicated global constants, only
// keep one copy of each constant.
PM.add(createConstantMergePass());
// Remove unused arguments from functions.
PM.add(createDeadArgEliminationPass());
// Reduce the code after globalopt and ipsccp. Both can open up significant
// simplification opportunities, and both can propagate functions through
// function pointers. When this happens, we often have to resolve varargs
// calls, etc, so let instcombine do this.
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(PM);
Add an extension point for peephole optimizers. This extension point allows adding passes that perform peephole optimizations similar to the instruction combiner. These passes will be inserted after each instance of the instruction combiner pass. Differential Revision: http://reviews.llvm.org/D3905 llvm-svn: 209595
2014-05-25 18:27:02 +08:00
addExtensionsToPM(EP_Peephole, PM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Inline small functions
Handle inlining in populateLTOPassManager like in populateModulePassManager. No functionality change. llvm-svn: 216178
2014-08-21 21:35:30 +08:00
bool RunInliner = Inliner;
if (RunInliner) {
PM.add(Inliner);
Inliner = nullptr;
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
PM.add(createPruneEHPass()); // Remove dead EH info.
// Optimize globals again if we ran the inliner.
if (RunInliner)
PM.add(createGlobalOptimizerPass());
PM.add(createGlobalDCEPass()); // Remove dead functions.
// If we didn't decide to inline a function, check to see if we can
// transform it to pass arguments by value instead of by reference.
PM.add(createArgumentPromotionPass());
// The IPO passes may leave cruft around. Clean up after them.
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(PM);
Add an extension point for peephole optimizers. This extension point allows adding passes that perform peephole optimizations similar to the instruction combiner. These passes will be inserted after each instance of the instruction combiner pass. Differential Revision: http://reviews.llvm.org/D3905 llvm-svn: 209595
2014-05-25 18:27:02 +08:00
addExtensionsToPM(EP_Peephole, PM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
PM.add(createJumpThreadingPass());
Random cleanup: No need to use a std::vector here, since createInternalizePass uses an ArrayRef. llvm-svn: 189632
2013-08-30 08:48:37 +08:00
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Break up allocas
Remove the ScalarReplAggregates pass Nearly all the changes to this pass have been done while maintaining and updating other parts of LLVM. LLVM has had another pass, SROA, which has superseded ScalarReplAggregates for quite some time. Differential Revision: http://reviews.llvm.org/D21316 llvm-svn: 272737
2016-06-15 08:19:09 +08:00
PM.add(createSROAPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Run a few AA driven optimizations here and now, to cleanup the code.
[PM] Port the PostOrderFunctionAttrs pass to the new pass manager and convert one test to use this. This is a particularly significant milestone because it required a working per-function AA framework which can be queried over each function from within a CGSCC transform pass (and additionally a module analysis to be accessible). This is essentially *the* point of the entire pass manager rewrite. A CGSCC transform is able to query for multiple different function's analysis results. It works. The whole thing appears to actually work and accomplish the original goal. While we were able to hack function attrs and basic-aa to "work" in the old pass manager, this port doesn't use any of that, it directly leverages the new fundamental functionality. For this to work, the CGSCC framework also has to support SCC-based behavior analysis, etc. The only part of the CGSCC pass infrastructure not sorted out at this point are the updates in the face of inlining and running function passes that mutate the call graph. The changes are pretty boring and boiler-plate. Most of the work was factored into more focused preperatory patches. But this is what wires it all together. llvm-svn: 261203
2016-02-18 19:03:11 +08:00
PM.add(createPostOrderFunctionAttrsLegacyPass()); // Add nocapture.
[PM/AA] Rebuild LLVM's alias analysis infrastructure in a way compatible with the new pass manager, and no longer relying on analysis groups. This builds essentially a ground-up new AA infrastructure stack for LLVM. The core ideas are the same that are used throughout the new pass manager: type erased polymorphism and direct composition. The design is as follows: - FunctionAAResults is a type-erasing alias analysis results aggregation interface to walk a single query across a range of results from different alias analyses. Currently this is function-specific as we always assume that aliasing queries are *within* a function. - AAResultBase is a CRTP utility providing stub implementations of various parts of the alias analysis result concept, notably in several cases in terms of other more general parts of the interface. This can be used to implement only a narrow part of the interface rather than the entire interface. This isn't really ideal, this logic should be hoisted into FunctionAAResults as currently it will cause a significant amount of redundant work, but it faithfully models the behavior of the prior infrastructure. - All the alias analysis passes are ported to be wrapper passes for the legacy PM and new-style analysis passes for the new PM with a shared result object. In some cases (most notably CFL), this is an extremely naive approach that we should revisit when we can specialize for the new pass manager. - BasicAA has been restructured to reflect that it is much more fundamentally a function analysis because it uses dominator trees and loop info that need to be constructed for each function. All of the references to getting alias analysis results have been updated to use the new aggregation interface. All the preservation and other pass management code has been updated accordingly. The way the FunctionAAResultsWrapperPass works is to detect the available alias analyses when run, and add them to the results object. This means that we should be able to continue to respect when various passes are added to the pipeline, for example adding CFL or adding TBAA passes should just cause their results to be available and to get folded into this. The exception to this rule is BasicAA which really needs to be a function pass due to using dominator trees and loop info. As a consequence, the FunctionAAResultsWrapperPass directly depends on BasicAA and always includes it in the aggregation. This has significant implications for preserving analyses. Generally, most passes shouldn't bother preserving FunctionAAResultsWrapperPass because rebuilding the results just updates the set of known AA passes. The exception to this rule are LoopPass instances which need to preserve all the function analyses that the loop pass manager will end up needing. This means preserving both BasicAAWrapperPass and the aggregating FunctionAAResultsWrapperPass. Now, when preserving an alias analysis, you do so by directly preserving that analysis. This is only necessary for non-immutable-pass-provided alias analyses though, and there are only three of interest: BasicAA, GlobalsAA (formerly GlobalsModRef), and SCEVAA. Usually BasicAA is preserved when needed because it (like DominatorTree and LoopInfo) is marked as a CFG-only pass. I've expanded GlobalsAA into the preserved set everywhere we previously were preserving all of AliasAnalysis, and I've added SCEVAA in the intersection of that with where we preserve SCEV itself. One significant challenge to all of this is that the CGSCC passes were actually using the alias analysis implementations by taking advantage of a pretty amazing set of loop holes in the old pass manager's analysis management code which allowed analysis groups to slide through in many cases. Moving away from analysis groups makes this problem much more obvious. To fix it, I've leveraged the flexibility the design of the new PM components provides to just directly construct the relevant alias analyses for the relevant functions in the IPO passes that need them. This is a bit hacky, but should go away with the new pass manager, and is already in many ways cleaner than the prior state. Another significant challenge is that various facilities of the old alias analysis infrastructure just don't fit any more. The most significant of these is the alias analysis 'counter' pass. That pass relied on the ability to snoop on AA queries at different points in the analysis group chain. Instead, I'm planning to build printing functionality directly into the aggregation layer. I've not included that in this patch merely to keep it smaller. Note that all of this needs a nearly complete rewrite of the AA documentation. I'm planning to do that, but I'd like to make sure the new design settles, and to flesh out a bit more of what it looks like in the new pass manager first. Differential Revision: http://reviews.llvm.org/D12080 llvm-svn: 247167
2015-09-10 01:55:00 +08:00
PM.add(createGlobalsAAWrapperPass()); // IP alias analysis.
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
Add an option to turn off the expensive GVN load PRE part of GVN. llvm-svn: 153902
2012-04-03 06:16:50 +08:00
PM.add(createLICMPass()); // Hoist loop invariants.
[PM] MLSM has been enabled for a way. Reclaim a cl::opt. llvm-svn: 293401
2017-01-29 07:45:37 +08:00
PM.add(createMergedLoadStoreMotionPass()); // Merge ld/st in diamonds.
[NewGVN] Add a flag to enable the pass via `-mllvm`. NewGVN can be tested passing `-mllvm -enable-newgvn` to clang. Differential Revision: https://reviews.llvm.org/D28059 llvm-svn: 290548
2016-12-27 02:26:19 +08:00
PM.add(NewGVN ? createNewGVNPass()
: createGVNPass(DisableGVNLoadPRE)); // Remove redundancies.
Add an option to turn off the expensive GVN load PRE part of GVN. llvm-svn: 153902
2012-04-03 06:16:50 +08:00
PM.add(createMemCpyOptPass()); // Remove dead memcpys.
Random cleanup: No need to use a std::vector here, since createInternalizePass uses an ArrayRef. llvm-svn: 189632
2013-08-30 08:48:37 +08:00
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Nuke dead stores.
PM.add(createDeadStoreEliminationPass());
LTO: Add more loop simplification passes to LTO Similar to r202051, add missing loop simplification passes to the LTO optimization pipeline. Patch by Rafael Espindola. llvm-svn: 206306
2014-04-16 01:48:15 +08:00
// More loops are countable; try to optimize them.
PM.add(createIndVarSimplifyPass());
PM.add(createLoopDeletionPass());
Add a new pass "Loop Interchange" This pass interchanges loops to provide a more cache-friendly memory access. For e.g. given a loop like - for(int i=0;i<N;i++) for(int j=0;j<N;j++) A[j][i] = A[j][i]+B[j][i]; is interchanged to - for(int j=0;j<N;j++) for(int i=0;i<N;i++) A[j][i] = A[j][i]+B[j][i]; This pass is currently disabled by default. To give a brief introduction it consists of 3 stages- LoopInterchangeLegality : Checks the legality of loop interchange based on Dependency matrix. LoopInterchangeProfitability: A very basic heuristic has been added to check for profitibility. This will evolve over time. LoopInterchangeTransform : Which does the actual transform. LNT Performance tests shows improvement in Polybench/linear-algebra/kernels/mvt and Polybench/linear-algebra/kernels/gemver becnmarks. TODO: 1) Add support for reductions and lcssa phi. 2) Improve profitability model. 3) Improve loop selection algorithm to select best loop for interchange. Currently the innermost loop is selected for interchange. 4) Improve compile time regression found in llvm lnt due to this pass. 5) Fix issues in Dependency Analysis module. A special thanks to Hal for reviewing this code. Review: http://reviews.llvm.org/D7499 llvm-svn: 231458
2015-03-06 18:11:25 +08:00
if (EnableLoopInterchange)
PM.add(createLoopInterchangePass());
[LTO] Add a run of LoopUnroll Loop trip counts can often be resolved during LTO. We should obviously be unrolling small loops once those trip counts have been resolved, but we weren't. llvm-svn: 257767
2016-01-14 23:00:09 +08:00
if (!DisableUnrollLoops)
Increases full-unroll threshold. Summary: The default threshold for fully unroll is too conservative. This patch doubles the full-unroll threshold This change will affect the following speccpu2006 benchmarks (performance numbers were collected from Intel Sandybridge): Performance: 403 0.11% 433 0.51% 445 0.48% 447 3.50% 453 1.49% 464 0.75% Code size: 403 0.56% 433 0.96% 445 2.16% 447 2.96% 453 0.94% 464 8.02% The compiler time overhead is similar with code size. Reviewers: davidxl, mkuper, mzolotukhin, hfinkel, chandlerc Reviewed By: hfinkel, chandlerc Subscribers: mehdi_amini, zzheng, efriedma, haicheng, hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D28368 llvm-svn: 295538
2017-02-18 11:46:51 +08:00
PM.add(createSimpleLoopUnrollPass(OptLevel)); // Unroll small loops
Add an option to the LTO code generator to disable vectorization during LTO We used to always vectorize (slp and loop vectorize) in the LTO pass pipeline. r220345 changed it so that we used the PassManager's fields 'LoopVectorize' and 'SLPVectorize' out of the desire to be able to disable vectorization using the cl::opt flags 'vectorize-loops'/'slp-vectorize' which the before mentioned fields default to. Unfortunately, this turns off vectorization because those fields default to false. This commit adds flags to the LTO library to disable lto vectorization which reconciles the desire to optionally disable vectorization during LTO and the desired behavior of defaulting to enabled vectorization. We really want tools to set PassManager flags directly to enable/disable vectorization and not go the route via cl::opt flags *in* PassManagerBuilder.cpp. llvm-svn: 220652
2014-10-27 05:50:58 +08:00
PM.add(createLoopVectorizePass(true, LoopVectorize));
[LTO] Add a run of LoopUnroll Loop trip counts can often be resolved during LTO. We should obviously be unrolling small loops once those trip counts have been resolved, but we weren't. llvm-svn: 257767
2016-01-14 23:00:09 +08:00
// The vectorizer may have significantly shortened a loop body; unroll again.
if (!DisableUnrollLoops)
Increases full-unroll threshold. Summary: The default threshold for fully unroll is too conservative. This patch doubles the full-unroll threshold This change will affect the following speccpu2006 benchmarks (performance numbers were collected from Intel Sandybridge): Performance: 403 0.11% 433 0.51% 445 0.48% 447 3.50% 453 1.49% 464 0.75% Code size: 403 0.56% 433 0.96% 445 2.16% 447 2.96% 453 0.94% 464 8.02% The compiler time overhead is similar with code size. Reviewers: davidxl, mkuper, mzolotukhin, hfinkel, chandlerc Reviewed By: hfinkel, chandlerc Subscribers: mehdi_amini, zzheng, efriedma, haicheng, hfinkel, llvm-commits Differential Revision: https://reviews.llvm.org/D28368 llvm-svn: 295538
2017-02-18 11:46:51 +08:00
PM.add(createLoopUnrollPass(OptLevel));
LTO: Add the loop vectorizer to the LTO pipeline. During the LTO phase LICM will move loop invariant global variables out of loops (informed by GlobalModRef). This makes more loops countable presenting opportunity for the loop vectorizer. Adding the loop vectorizer improves some TSVC benchmarks and twolf/ref dataset (5%) on x86-64. radar://15970632 llvm-svn: 202051
2014-02-25 02:19:31 +08:00
[PassManagerBuilder] Add a few more scalar optimization passes This patch does two things: 1. mem2reg is now run immediately after globalopt. Now that globalopt can localize variables more aggressively, it makes sense to lower them to SSA form earlier rather than later so they can benefit from the full set of optimization passes. 2. More scalar optimizations are run after the loop optimizations in LTO mode. The loop optimizations (especially indvars) can clean up scalar code sufficiently to make it worthwhile running more scalar passes. I've particularly added SCCP here as it isn't run anywhere else in the LTO pass pipeline. Mem2reg is super cheap and shouldn't affect compilation time at all. The rest of the added passes are in the LTO pipeline only so doesn't affect the vast majority of compilations, just the link step. llvm-svn: 255634
2015-12-15 17:24:01 +08:00
// Now that we've optimized loops (in particular loop induction variables),
// we may have exposed more scalar opportunities. Run parts of the scalar
// optimizer again at this point.
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(PM); // Initial cleanup
[PassManagerBuilder] Add a few more scalar optimization passes This patch does two things: 1. mem2reg is now run immediately after globalopt. Now that globalopt can localize variables more aggressively, it makes sense to lower them to SSA form earlier rather than later so they can benefit from the full set of optimization passes. 2. More scalar optimizations are run after the loop optimizations in LTO mode. The loop optimizations (especially indvars) can clean up scalar code sufficiently to make it worthwhile running more scalar passes. I've particularly added SCCP here as it isn't run anywhere else in the LTO pass pipeline. Mem2reg is super cheap and shouldn't affect compilation time at all. The rest of the added passes are in the LTO pipeline only so doesn't affect the vast majority of compilations, just the link step. llvm-svn: 255634
2015-12-15 17:24:01 +08:00
PM.add(createCFGSimplificationPass()); // if-convert
PM.add(createSCCPPass()); // Propagate exposed constants
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(PM); // Clean up again
[PassManagerBuilder] Add a few more scalar optimization passes This patch does two things: 1. mem2reg is now run immediately after globalopt. Now that globalopt can localize variables more aggressively, it makes sense to lower them to SSA form earlier rather than later so they can benefit from the full set of optimization passes. 2. More scalar optimizations are run after the loop optimizations in LTO mode. The loop optimizations (especially indvars) can clean up scalar code sufficiently to make it worthwhile running more scalar passes. I've particularly added SCCP here as it isn't run anywhere else in the LTO pass pipeline. Mem2reg is super cheap and shouldn't affect compilation time at all. The rest of the added passes are in the LTO pipeline only so doesn't affect the vast majority of compilations, just the link step. llvm-svn: 255634
2015-12-15 17:24:01 +08:00
PM.add(createBitTrackingDCEPass());
Reapply: Add slp vectorization to LTO passes. The bug it exposed has been fixed by r207983. <radar://16641956> llvm-svn: 208013
2014-05-06 07:14:46 +08:00
// More scalar chains could be vectorized due to more alias information
LTO: respect command-line options that disable vectorization. Summary: Patches 202051 and 208013 added calls to LTO's PassManager which unconditionally add LoopVectorizePass and SLPVectorizerPass instead of following the logic in PassManagerBuilder::populateModulePassManager and honoring the -vectorize-loops -run-slp-after-loop-vectorization flags. Reviewers: nadav, aschwaighofer, yijiang Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D5884 llvm-svn: 220345
2014-10-22 07:18:21 +08:00
if (RunSLPAfterLoopVectorization)
if (SLPVectorize)
PM.add(createSLPVectorizerPass()); // Vectorize parallel scalar chains.
Reapply: Add slp vectorization to LTO passes. The bug it exposed has been fixed by r207983. <radar://16641956> llvm-svn: 208013
2014-05-06 07:14:46 +08:00
Add an AlignmentFromAssumptions Pass This adds a ScalarEvolution-powered transformation that updates load, store and memory intrinsic pointer alignments based on invariant((a+q) & b == 0) expressions. Many of the simple cases we can get with ValueTracking, but we still need something like this for the more complicated cases (such as those with an offset) that require some algebra. Note that gcc's __builtin_assume_aligned's optional third argument provides exactly for this kind of 'misalignment' offset for which this kind of logic is necessary. The primary motivation is to fixup alignments for vector loads/stores after vectorization (and unrolling). This pass is added to the optimization pipeline just after the SLP vectorizer runs (which, admittedly, does not preserve SE, although I imagine it could). Regardless, I actually don't think that the preservation matters too much in this case: SE computes lazily, and this pass won't issue any SE queries unless there are any assume intrinsics, so there should be no real additional cost in the common case (SLP does preserve DT and LoopInfo). llvm-svn: 217344
2014-09-08 04:05:11 +08:00
// After vectorization, assume intrinsics may tell us more about pointer
// alignments.
PM.add(createAlignmentFromAssumptionsPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Cleanup and simplify the code after the scalar optimizations.
InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2 As part of r251146 InstCombine was extended to call computeKnownBits on every value in the function to determine whether it happens to be constant. This increases typical compiletime by 1-3% (5% in irgen+opt time) in my measurements. On the other hand this case did not trigger once in the whole llvm-testsuite. This patch introduces the notion of ExpensiveCombines which are only enabled for OptLevel > 2. I removed the check in InstructionSimplify as that is called from various places where the OptLevel is not known but given the rarity of the situation I think a check in InstCombine is enough. Differential Revision: http://reviews.llvm.org/D16835 llvm-svn: 263047
2016-03-10 02:47:11 +08:00
addInstructionCombiningPass(PM);
Add an extension point for peephole optimizers. This extension point allows adding passes that perform peephole optimizations similar to the instruction combiner. These passes will be inserted after each instance of the instruction combiner pass. Differential Revision: http://reviews.llvm.org/D3905 llvm-svn: 209595
2014-05-25 18:27:02 +08:00
addExtensionsToPM(EP_Peephole, PM);
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
PM.add(createJumpThreadingPass());
libLTO, llvm-lto, gold: Introduce flag for controlling optimization level. This change also introduces a link-time optimization level of 1. This optimization level runs only the globaldce pass as well as cleanup passes for passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets. http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html llvm-svn: 232769
2015-03-20 06:01:00 +08:00
}
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
libLTO, llvm-lto, gold: Introduce flag for controlling optimization level. This change also introduces a link-time optimization level of 1. This optimization level runs only the globaldce pass as well as cleanup passes for passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets. http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html llvm-svn: 232769
2015-03-20 06:01:00 +08:00
void PassManagerBuilder::addLateLTOOptimizationPasses(
legacy::PassManagerBase &PM) {
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Delete basic blocks, which optimization passes may have killed.
Factor FlattenCFG out from SimplifyCFG Patch by: Mei Ye llvm-svn: 187764
2013-08-06 10:43:45 +08:00
PM.add(createCFGSimplificationPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
Enable EliminateAvailableExternally pass in the LTO pipeline. Summary: For LTO we need to enable this pass in the LTO pipeline, as it is skipped during the "-flto -c" compile step (when PrepareForLTO is set). Reviewers: rnk Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11919 llvm-svn: 244622
2015-08-12 00:26:41 +08:00
// Drop bodies of available externally objects to improve GlobalDCE.
PM.add(createEliminateAvailableExternallyPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
// Now that we have optimized the program, discard unreachable functions.
PM.add(createGlobalDCEPass());
Add control of function merging to the PMBuilder. llvm-svn: 217731
2014-09-14 05:46:00 +08:00
// FIXME: this is profitable (for compiler time) to do at -O0 too, but
// currently it damages debug info.
if (MergeFunctions)
PM.add(createMergeFunctionsPass());
Move methods in PassManagerBuilder offline. llvm-svn: 136727
2011-08-03 05:50:27 +08:00
}
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
Define the ThinLTO Pipeline (experimental) Summary: On the contrary to Full LTO, ThinLTO can afford to shift compile time from the frontend to the linker: both phases are parallel (even if it is not totally "free": projects like clang are reusing product from the "compile phase" for multiple link, think about libLLVMSupport reused for opt, llc, etc.). This pipeline is based on the proposal in D13443 for full LTO. We didn't move forward on this proposal because the LTO link was far too long after that. We believe that we can afford it with ThinLTO. The ThinLTO pipeline integrates in the regular O2/O3 flow: - The compile phase perform the inliner with a somehow lighter function simplification. (TODO: tune the inliner thresholds here) This is intendend to simplify the IR and get rid of obvious things like linkonce_odr that will be inlined. - The link phase will run the pipeline from the start, extended with some specific passes that leverage the augmented knowledge we have during LTO. Especially after the inliner is done, a sequence of globalDCE/globalOpt is performed, followed by another run of the "function simplification" passes. It is not clear if this part of the pipeline will stay as is, as the split model of ThinLTO does not allow the same benefit as FullLTO without added tricks. The measurements on the public test suite as well as on our internal suite show an overall net improvement. The binary size for the clang executable is reduced by 5%. We're still tuning it with the bringup of ThinLTO and it will evolve, but this should provide a good starting point. Reviewers: tejohnson Differential Revision: http://reviews.llvm.org/D17115 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261029
2016-02-17 07:02:29 +08:00
void PassManagerBuilder::populateThinLTOPassManager(
legacy::PassManagerBase &PM) {
PerformThinLTO = true;
if (VerifyInput)
PM.add(createVerifierPass());
IPO: Const correctness for summaries passed into passes. Pass const qualified summaries into importers and unqualified summaries into exporters. This lets us const-qualify the summary argument to thinBackend. Differential Revision: https://reviews.llvm.org/D31230 llvm-svn: 298534
2017-03-23 02:22:59 +08:00
if (ImportSummary) {
PMB: Add an importing WPD pass to the start of the ThinLTO backend pipeline. Differential Revision: https://reviews.llvm.org/D30008 llvm-svn: 295260
2017-02-16 07:48:38 +08:00
// These passes import type identifier resolutions for whole-program
// devirtualization and CFI. They must run early because other passes may
// disturb the specific instruction patterns that these passes look for,
// creating dependencies on resolutions that may not appear in the summary.
//
// For example, GVN may transform the pattern assume(type.test) appearing in
// two basic blocks into assume(phi(type.test, type.test)), which would
// transform a dependency on a WPD resolution into a dependency on a type
// identifier resolution for CFI.
//
// Also, WPD has access to more precise information than ICP and can
// devirtualize more effectively, so it should operate on the IR first.
IPO: Const correctness for summaries passed into passes. Pass const qualified summaries into importers and unqualified summaries into exporters. This lets us const-qualify the summary argument to thinBackend. Differential Revision: https://reviews.llvm.org/D31230 llvm-svn: 298534
2017-03-23 02:22:59 +08:00
PM.add(createWholeProgramDevirtPass(nullptr, ImportSummary));
PM.add(createLowerTypeTestsPass(nullptr, ImportSummary));
PMB: Add an importing WPD pass to the start of the ThinLTO backend pipeline. Differential Revision: https://reviews.llvm.org/D30008 llvm-svn: 295260
2017-02-16 07:48:38 +08:00
}
IPO, LTO: Plumb the summary from the LTO API into the pass manager. Differential Revision: https://reviews.llvm.org/D28840 llvm-svn: 292661
2017-01-21 06:18:52 +08:00
Define the ThinLTO Pipeline (experimental) Summary: On the contrary to Full LTO, ThinLTO can afford to shift compile time from the frontend to the linker: both phases are parallel (even if it is not totally "free": projects like clang are reusing product from the "compile phase" for multiple link, think about libLLVMSupport reused for opt, llc, etc.). This pipeline is based on the proposal in D13443 for full LTO. We didn't move forward on this proposal because the LTO link was far too long after that. We believe that we can afford it with ThinLTO. The ThinLTO pipeline integrates in the regular O2/O3 flow: - The compile phase perform the inliner with a somehow lighter function simplification. (TODO: tune the inliner thresholds here) This is intendend to simplify the IR and get rid of obvious things like linkonce_odr that will be inlined. - The link phase will run the pipeline from the start, extended with some specific passes that leverage the augmented knowledge we have during LTO. Especially after the inliner is done, a sequence of globalDCE/globalOpt is performed, followed by another run of the "function simplification" passes. It is not clear if this part of the pipeline will stay as is, as the split model of ThinLTO does not allow the same benefit as FullLTO without added tricks. The measurements on the public test suite as well as on our internal suite show an overall net improvement. The binary size for the clang executable is reduced by 5%. We're still tuning it with the bringup of ThinLTO and it will evolve, but this should provide a good starting point. Reviewers: tejohnson Differential Revision: http://reviews.llvm.org/D17115 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 261029
2016-02-17 07:02:29 +08:00
populateModulePassManager(PM);
if (VerifyOutput)
PM.add(createVerifierPass());
PerformThinLTO = false;
}
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
void PassManagerBuilder::populateLTOPassManager(legacy::PassManagerBase &PM) {
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
if (LibraryInfo)
[PM] Separate the TargetLibraryInfo object from the immutable pass. The pass is really just a means of accessing a cached instance of the TargetLibraryInfo object, and this way we can re-use that object for the new pass manager as its result. Lots of delta, but nothing interesting happening here. This is the common pattern that is developing to allow analyses to live in both the old and new pass manager -- a wrapper pass in the old pass manager emulates the separation intrinsic to the new pass manager between the result and pass for analyses. llvm-svn: 226157
2015-01-15 18:41:28 +08:00
PM.add(new TargetLibraryInfoWrapperPass(*LibraryInfo));
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
Verifier: Remove the separate -verify-di pass Remove `DebugInfoVerifierLegacyPass` and the `-verify-di` pass. Instead, call into the `DebugInfoVerifier` from inside `VerifierLegacyPass::finalizeModule()`. This better matches the logic in `verifyModule()` (used by the new PassManager), avoids requiring two separate passes to verify the IR, and makes the API for "add a pass to verify the IR" simple. Note: the `-verify-debug-info` flag still works (for now, at least; eventually it might make sense to just remove it). llvm-svn: 232772
2015-03-20 06:24:17 +08:00
if (VerifyInput)
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
PM.add(createVerifierPass());
WholeProgramDevirt: introduce. This pass implements whole program optimization of virtual calls in cases where we know (via bitset information) that the list of callees is fixed. This includes the following: - Single implementation devirtualization: if a virtual call has a single possible callee, replace all calls with a direct call to that callee. - Virtual constant propagation: if the virtual function's return type is an integer <=64 bits and all possible callees are readnone, for each class and each list of constant arguments: evaluate the function, store the return value alongside the virtual table, and rewrite each virtual call as a load from the virtual table. - Uniform return value optimization: if the conditions for virtual constant propagation hold and each function returns the same constant value, replace each virtual call with that constant. - Unique return value optimization for i1 return values: if the conditions for virtual constant propagation hold and a single vtable's function returns 0, or a single vtable's function returns 1, replace each virtual call with a comparison of the vptr against that vtable's address. Differential Revision: http://reviews.llvm.org/D16795 llvm-svn: 260312
2016-02-10 06:50:34 +08:00
if (OptLevel != 0)
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
addLTOOptimizationPasses(PM);
PMB: Run the whole-program-devirt pass during LTO at --lto-O0. The whole-program-devirt pass needs to run at -O0 because only it knows about the llvm.type.checked.load intrinsic: it needs to both lower the intrinsic itself and handle it in the summary. Differential Revision: https://reviews.llvm.org/D33571 llvm-svn: 304019
2017-05-27 02:27:13 +08:00
else {
// The whole-program-devirt pass needs to run at -O0 because only it knows
// about the llvm.type.checked.load intrinsic: it needs to both lower the
// intrinsic itself and handle it in the summary.
PM.add(createWholeProgramDevirtPass(ExportSummary, nullptr));
}
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
Cross-DSO control flow integrity (LLVM part). An LTO pass that generates a __cfi_check() function that validates a call based on a hash of the call-site-known type and the target pointer. llvm-svn: 255693
2015-12-16 07:00:08 +08:00
// Create a function that performs CFI checks for cross-DSO calls with targets
// in the current module.
PM.add(createCrossDSOCFIPass());
IR: New representation for CFI and virtual call optimization pass metadata. The bitset metadata currently used in LLVM has a few problems: 1. It has the wrong name. The name "bitset" refers to an implementation detail of one use of the metadata (i.e. its original use case, CFI). This makes it harder to understand, as the name makes no sense in the context of virtual call optimization. 2. It is represented using a global named metadata node, rather than being directly associated with a global. This makes it harder to manipulate the metadata when rebuilding global variables, summarise it as part of ThinLTO and drop unused metadata when associated globals are dropped. For this reason, CFI does not currently work correctly when both CFI and vcall opt are enabled, as vcall opt needs to rebuild vtable globals, and fails to associate metadata with the rebuilt globals. As I understand it, the same problem could also affect ASan, which rebuilds globals with a red zone. This patch solves both of those problems in the following way: 1. Rename the metadata to "type metadata". This new name reflects how the metadata is currently being used (i.e. to represent type information for CFI and vtable opt). The new name is reflected in the name for the associated intrinsic (llvm.type.test) and pass (LowerTypeTests). 2. Attach metadata directly to the globals that it pertains to, rather than using the "llvm.bitsets" global metadata node as we are doing now. This is done using the newly introduced capability to attach metadata to global variables (r271348 and r271358). See also: http://lists.llvm.org/pipermail/llvm-dev/2016-June/100462.html Differential Revision: http://reviews.llvm.org/D21053 llvm-svn: 273729
2016-06-25 05:21:32 +08:00
// Lower type metadata and the type.test intrinsic. This pass supports Clang's
// control flow integrity mechanisms (-fsanitize=cfi*) and needs to run at
// link time if CFI is enabled. The pass does nothing if CFI is disabled.
IPO: Const correctness for summaries passed into passes. Pass const qualified summaries into importers and unqualified summaries into exporters. This lets us const-qualify the summary argument to thinBackend. Differential Revision: https://reviews.llvm.org/D31230 llvm-svn: 298534
2017-03-23 02:22:59 +08:00
PM.add(createLowerTypeTestsPass(ExportSummary, nullptr));
libLTO, llvm-lto, gold: Introduce flag for controlling optimization level. This change also introduces a link-time optimization level of 1. This optimization level runs only the globaldce pass as well as cleanup passes for passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets. http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html llvm-svn: 232769
2015-03-20 06:01:00 +08:00
if (OptLevel != 0)
addLateLTOOptimizationPasses(PM);
Verifier: Remove the separate -verify-di pass Remove `DebugInfoVerifierLegacyPass` and the `-verify-di` pass. Instead, call into the `DebugInfoVerifier` from inside `VerifierLegacyPass::finalizeModule()`. This better matches the logic in `verifyModule()` (used by the new PassManager), avoids requiring two separate passes to verify the IR, and makes the API for "add a pass to verify the IR" simple. Note: the `-verify-debug-info` flag still works (for now, at least; eventually it might make sense to just remove it). llvm-svn: 232772
2015-03-20 06:24:17 +08:00
if (VerifyOutput)
Move some logic to populateLTOPassManager. This will avoid code duplication in the next commit which calls it directly from the gold plugin. llvm-svn: 216211
2014-08-22 04:03:44 +08:00
PM.add(createVerifierPass());
}
Move C++ code out of the C headers and into either C++ headers or the C++ files themselves. This enables people to use just a C compiler to interoperate with LLVM. llvm-svn: 180063
2013-04-23 06:47:22 +08:00
inline PassManagerBuilder *unwrap(LLVMPassManagerBuilderRef P) {
return reinterpret_cast<PassManagerBuilder*>(P);
}
inline LLVMPassManagerBuilderRef wrap(PassManagerBuilder *P) {
return reinterpret_cast<LLVMPassManagerBuilderRef>(P);
}
Use empty parens for empty function parameter list instead of '(void)'. llvm-svn: 168049
2012-11-16 00:51:49 +08:00
LLVMPassManagerBuilderRef LLVMPassManagerBuilderCreate() {
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
PassManagerBuilder *PMB = new PassManagerBuilder();
return wrap(PMB);
}
void LLVMPassManagerBuilderDispose(LLVMPassManagerBuilderRef PMB) {
PassManagerBuilder *Builder = unwrap(PMB);
delete Builder;
}
void
LLVMPassManagerBuilderSetOptLevel(LLVMPassManagerBuilderRef PMB,
unsigned OptLevel) {
PassManagerBuilder *Builder = unwrap(PMB);
Builder->OptLevel = OptLevel;
}
void
LLVMPassManagerBuilderSetSizeLevel(LLVMPassManagerBuilderRef PMB,
unsigned SizeLevel) {
PassManagerBuilder *Builder = unwrap(PMB);
Builder->SizeLevel = SizeLevel;
}
void
LLVMPassManagerBuilderSetDisableUnitAtATime(LLVMPassManagerBuilderRef PMB,
LLVMBool Value) {
PassManagerBuilder *Builder = unwrap(PMB);
Builder->DisableUnitAtATime = Value;
}
void
LLVMPassManagerBuilderSetDisableUnrollLoops(LLVMPassManagerBuilderRef PMB,
LLVMBool Value) {
PassManagerBuilder *Builder = unwrap(PMB);
Builder->DisableUnrollLoops = Value;
}
void
LLVMPassManagerBuilderSetDisableSimplifyLibCalls(LLVMPassManagerBuilderRef PMB,
LLVMBool Value) {
Remove the simplify-libcalls pass (finally) This commit completely removes what is left of the simplify-libcalls pass. All of the functionality has now been migrated to the instcombine and functionattrs passes. The following C API functions are now NOPs: 1. LLVMAddSimplifyLibCallsPass 2. LLVMPassManagerBuilderSetDisableSimplifyLibCalls llvm-svn: 184459
2013-06-21 03:48:07 +08:00
// NOTE: The simplify-libcalls pass has been removed.
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
}
void
LLVMPassManagerBuilderUseInlinerWithThreshold(LLVMPassManagerBuilderRef PMB,
unsigned Threshold) {
PassManagerBuilder *Builder = unwrap(PMB);
Builder->Inliner = createFunctionInliningPass(Threshold);
}
void
LLVMPassManagerBuilderPopulateFunctionPassManager(LLVMPassManagerBuilderRef PMB,
LLVMPassManagerRef PM) {
PassManagerBuilder *Builder = unwrap(PMB);
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
legacy::FunctionPassManager *FPM = unwrap<legacy::FunctionPassManager>(PM);
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
Builder->populateFunctionPassManager(*FPM);
}
void
LLVMPassManagerBuilderPopulateModulePassManager(LLVMPassManagerBuilderRef PMB,
LLVMPassManagerRef PM) {
PassManagerBuilder *Builder = unwrap(PMB);
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
legacy::PassManagerBase *MPM = unwrap(PM);
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
Builder->populateModulePassManager(*MPM);
}
void LLVMPassManagerBuilderPopulateLTOPassManager(LLVMPassManagerBuilderRef PMB,
LLVMPassManagerRef PM,
Use LLVMBool instead of 'bool' in the C API. Based on a patch by Peter Zotov! llvm-svn: 176793
2013-03-11 05:58:22 +08:00
LLVMBool Internalize,
LLVMBool RunInliner) {
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
PassManagerBuilder *Builder = unwrap(PMB);
[PM] Remove the old 'PassManager.h' header file at the top level of LLVM's include tree and the use of using declarations to hide the 'legacy' namespace for the old pass manager. This undoes the primary modules-hostile change I made to keep out-of-tree targets building. I sent an email inquiring about whether this would be reasonable to do at this phase and people seemed fine with it, so making it a reality. This should allow us to start bootstrapping with modules to a certain extent along with making it easier to mix and match headers in general. The updates to any code for users of LLVM are very mechanical. Switch from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h". Qualify the types which now produce compile errors with "legacy::". The most common ones are "PassManager", "PassManagerBase", and "FunctionPassManager". llvm-svn: 229094
2015-02-13 18:01:29 +08:00
legacy::PassManagerBase *LPM = unwrap(PM);
Handle inlining in populateLTOPassManager like in populateModulePassManager. No functionality change. llvm-svn: 216178
2014-08-21 21:35:30 +08:00
// A small backwards compatibility hack. populateLTOPassManager used to take
// an RunInliner option.
if (RunInliner && !Builder->Inliner)
Builder->Inliner = createFunctionInliningPass();
Builder->populateLTOPassManager(*LPM);
Add a C interface to PassManagerBuilder. It is missing the addExtension functionality since in the C api a pass is created and added to a pass manager in a single call. llvm-svn: 137159
2011-08-10 06:17:34 +08:00
}