Commit Graph

72 Commits

Author SHA1 Message Date
Max Kazantsev e4c220e8f2 [IRCE][NFC] Add another assert that AddRecExpr's step is not zero
One more assertion of this kind. It is a preparation step for generalizing
to the case of stride not equal to +1/-1.

llvm-svn: 309663
2017-08-01 06:49:29 +00:00
Max Kazantsev 85da7543f9 [IRCE][NFC] Add assert that AddRecExpr's step is not zero
We should never return zero steps, ensure this fact by adding
a sanity check when we are analyzing the induction variable.

llvm-svn: 309661
2017-08-01 06:27:51 +00:00
Max Kazantsev 2c627a97fd [IRCE] Recognize loops with ne/eq latch conditions
In some particular cases eq/ne conditions can be turned into equivalent
slt/sgt conditions. This patch teaches parseLoopStructure to handle some
of these cases.

Differential Revision: https://reviews.llvm.org/D35010

llvm-svn: 308264
2017-07-18 04:53:48 +00:00
Max Kazantsev f80ffa1a78 [IRCE] Fix corner case with Start = INT_MAX
When iterating through loop

  for (int i = INT_MAX; i > 0; i--)

We fail to generate the pre-loop for it. It happens because we use the
overflown value in a comparison predicate when identifying whether or not
we need it.

In old logic, we used SLE predicate against Greatest value which exceeds all
seen values of the IV and might be overflown. Now we use the GreatestSeen
value of this IV with SLT predicate.

Also added a test that ensures that a pre-loop is generated for such loops.

Differential Revision: https://reviews.llvm.org/D35347

llvm-svn: 308001
2017-07-14 06:35:03 +00:00
Max Kazantsev 6c466a376e [IRCE][NFC] Better get SCEV for 1 in calculateSubRanges
A slightly more efficient way to get constant, we avoid resolving in getSCEV and excessive
invocations, and we don't create a ConstantInt if 'true' branch is taken.

Differential Revision: https://reviews.llvm.org/D34672

llvm-svn: 306503
2017-06-28 04:57:45 +00:00
Anna Thomas 7218032019 [IRCE] Canonicalize pre/post loops after the blocks are added into parent loop
Summary:
We were canonizalizing the pre loop (into loop-simplify form) before
the post loop blocks were added into parent loop. This is incorrect when IRCE is
done on a subloop. The post-loop blocks are created, but not yet added to the
parent loop. So, loop-simplification on the pre-loop incorrectly updates
LoopInfo.

This patch corrects the ordering so that pre and post loop blocks are added to
parent loop (if any), and then the loops are canonicalized to LCSSA and
LoopSimplifyForm.

Reviewers: reames, sanjoy, apilipenko

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33846

llvm-svn: 304800
2017-06-06 14:54:01 +00:00
Chandler Carruth 6bda14b313 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787
2017-06-06 11:49:48 +00:00
Chandler Carruth 29c22d2835 [LegacyPM] Make the 'addLoop' method accept a loop to add rather than
having it internally allocate the loop.

This is a much more flexible API and necessary in the new loop unswitch
to reasonably support both new and old PMs in common code. It also just
seems like a cleaner separation of concerns.

NFC, this should just be a pure refactoring.

Differential Revision: https://reviews.llvm.org/D33528

llvm-svn: 303834
2017-05-25 03:01:31 +00:00
Sanjoy Das ec892139bd [IRCE] Add a missing invariant check
Currently IRCE relies on the loops it transforms to be (semantically) of
the form:

  for (i = START; i < END; i++)
    ...

or

  for (i = START; i > END; i--)
    ...

However, we were not verifying the presence of the START < END entry
check (i.e. check before the first iteration).  We were only verifying
that the backedge was guarded by (i + 1) < END.

Usually this would work "fine" since (especially in Java) most loops do
actually have the START < END check, but of course that is not
guaranteed.

llvm-svn: 294375
2017-02-07 23:59:07 +00:00
Daniel Jasper aec2fa352f Revert @llvm.assume with operator bundles (r289755-r289757)
This creates non-linear behavior in the inliner (see more details in
r289755's commit thread).

llvm-svn: 290086
2016-12-19 08:22:17 +00:00
Hal Finkel 3ca4a6bcf1 Remove the AssumptionCache
After r289755, the AssumptionCache is no longer needed. Variables affected by
assumptions are now found by using the new operand-bundle-based scheme. This
new scheme is more computationally efficient, and also we need much less
code...

llvm-svn: 289756
2016-12-15 03:02:15 +00:00
Anna Thomas 65ca8e91cc [IRCE] Avoid loop optimizations on pre and post loops
Summary:
This patch will add loop metadata on the pre and post loops generated by IRCE.
Currently, we have metadata for disabling optimizations such as vectorization,
unrolling, loop distribution and LICM versioning (and confirmed that these
optimizations check for the metadata before proceeding with the transformation).

The pre and post loops generated by IRCE need not go through loop opts (since
these are slow paths).

Added two test cases as well.

Reviewers: sanjoy, reames

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D26806

llvm-svn: 289588
2016-12-13 21:05:21 +00:00
Davide Italiano d1279df752 [IRCE] Switch over to LLVM_DUMP_METHOD. NFCI.
llvm-svn: 279079
2016-08-18 15:55:49 +00:00
Justin Bogner b03fd12cef Replace "fallthrough" comments with LLVM_FALLTHROUGH
This is a mechanical change of comments in switches like fallthrough,
fall-through, or fall-thru to use the LLVM_FALLTHROUGH macro instead.

llvm-svn: 278902
2016-08-17 05:10:15 +00:00
Duncan P. N. Exon Smith 3bcaa81204 Scalar: Avoid dereferencing end() in InductiveRangeCheckElimination
BasicBlock::Create isn't designed to take iterators (which might be
end()), but pointers (which might be nullptr).  Fix the UB that was
converting end() to a BasicBlock* by calling BasicBlock::getNextNode()
in the first place.

llvm-svn: 278883
2016-08-17 01:16:17 +00:00
Sanjoy Das 35459f0e34 [IRCE] Change variable grouping; NFC
llvm-svn: 278619
2016-08-14 01:04:50 +00:00
Sanjoy Das 2143447c73 [IRCE] Create llvm::Loop instances for cloned out loops
llvm-svn: 278618
2016-08-14 01:04:46 +00:00
Sanjoy Das 7a18a238c6 [IRCE] Don't iterate on loops that were cloned out
IRCE has the ability to further version pre-loops and post-loops that it
created, but this isn't useful at all.  This change teaches IRCE to
leave behind some metadata in the loops it creates (by cloning the main
loop) so that these new loops are not re-processed by IRCE.

Today this bug is hidden by another bug -- IRCE does not update LoopInfo
properly so the loop pass manager does not re-invoke IRCE on the loops
it split out.  However, once the latter is fixed the bug addressed in
this change causes IRCE to infinite-loop in some cases (e.g. it splits
out a pre-loop, a pre-pre-loop from that, a pre-pre-pre-loop from that
and so on).

llvm-svn: 278617
2016-08-14 01:04:36 +00:00
Sanjoy Das 43fdc54303 [IRCE] Add better DEBUG diagnostic; NFC
NFC meaning IRCE should not _do_ anything different, but
-debug-only=irce will be a little friendlier.

llvm-svn: 278616
2016-08-14 01:04:31 +00:00
Sanjoy Das 2a2f14d7ab [IRCE] Be resilient in the face of non-simplified loops
Loops containing `indirectbr` may not be in simplified form, even after
running LoopSimplify.  Reject then gracefully, instead of tripping an
assert.

llvm-svn: 278611
2016-08-13 23:36:35 +00:00
Sanjoy Das f2b7bafae4 [IRCE] Use dyn_cast instead of explicit isa/cast; NFC
llvm-svn: 278607
2016-08-13 22:00:12 +00:00
Sanjoy Das d1d62a1354 [IRCE] Use range-for; NFC
llvm-svn: 278606
2016-08-13 22:00:09 +00:00
Sanjoy Das b8c2ebea08 [IRCE] Remove unused headers; NFC
llvm-svn: 277892
2016-08-06 00:02:01 +00:00
Sanjoy Das cf181867a6 [IRCE] Preserve loop-simplify form
Fixes PR28764.  Right now there is no way to test this, but (as
mentioned on the PR) with Michael Zolotukhin's yet to be checked in
LoopSimplify verfier, 8 of the llvm-lit tests for IRCE crash.

llvm-svn: 277891
2016-08-06 00:01:56 +00:00
Sanjoy Das 83a72850c7 [IRCE] Rename variable; NFC
There is nothing "Original" about "OriginalLoopInfo".

llvm-svn: 277506
2016-08-02 19:32:01 +00:00
Sanjoy Das f45e03e201 [IRCE] Preserve DomTree and LCSSA
This changes IRCE to "preserve" LCSSA and DomTree by recomputing them.
It still does not preserve LoopSimplify.

llvm-svn: 277505
2016-08-02 19:31:54 +00:00
Sanjoy Das bb969791b4 [IRCE] Add an option to skip profitability checks
If `-irce-skip-profitability-checks` is passed in, IRCE will kick in in
all cases where it is legal for it to kick in.  This flag is intended to
help diagnose and analyse performance issues.

llvm-svn: 276372
2016-07-22 00:40:56 +00:00
Sanjoy Das 81c00fe022 [IRCE] Use getTerminator instead of rbegin; NFC
llvm-svn: 273586
2016-06-23 18:03:26 +00:00
Sanjoy Das ee77a4828e [IRCE] Use C++11 style initializers; NFC
llvm-svn: 270815
2016-05-26 01:50:18 +00:00
Sanjoy Das a099268e85 [IRCE] Optimize conjunctions of range checks
After this change, we do the expected thing for cases like

```
Check0Passed = /* range check IRCE can optimize */
Check1Passed = /* range check IRCE can optimize */
if (!(Check0Passed && Check1Passed))
  throw_Exception();
```

llvm-svn: 270804
2016-05-26 00:09:02 +00:00
Sanjoy Das 8fe8892c2d [IRCE] Refactor out a parseRangeCheckFromCond; NFC
This will later hold more general logic to parse conjunctions of range
checks.

llvm-svn: 270802
2016-05-26 00:08:24 +00:00
Sanjoy Das 5fd7ac452e [IRCE] Return a Value*, not SCEV* from parseRangeCheck; NFC
This is better layering, since the caller needs to check if the index
was an add-rec anyway.

llvm-svn: 270582
2016-05-24 17:19:56 +00:00
Sanjoy Das aa83c47bab [IRCE] Optimize "uses" not branches; NFCI
This changes IRCE to optimize uses, and not branches.  This change is
NFCI since the uses we do inspect are in practice only ever going to be
the condition use in conditional branches; but this flexibility will
later allow us to analyze more complex expressions than just a direct
branch on a range check.

llvm-svn: 270500
2016-05-23 22:16:45 +00:00
Sanjoy Das c5b1169de2 [IRCE] Don't use an allocator for range checks; NFC
The InductiveRangeCheck struct is only five words long; so passing these
around value is fine.  The allocator makes the code look more complex
than it is.

llvm-svn: 270309
2016-05-21 02:52:13 +00:00
Sanjoy Das 59776734a3 [IRCE] Don't pass IRBuilder<> where unnecessary; NFC
llvm-svn: 270308
2016-05-21 02:31:51 +00:00
Andrew Kaylor 50271f787e Add opt-bisect support to additional passes that can be skipped
Differential Revision: http://reviews.llvm.org/D19882

llvm-svn: 268457
2016-05-03 22:32:30 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Duncan P. N. Exon Smith da68cbc4ad IR: RF_IgnoreMissingValues => RF_IgnoreMissingLocals, NFC
Clarify what this RemapFlag actually means.

  - Change the flag name to match its intended behaviour.
  - Clearly document that it's not supposed to affect globals.
  - Add a host of FIXMEs to indicate how to fix the behaviour to match
    the intent of the flag.

RF_IgnoreMissingLocals should only affect the behaviour of
RemapInstruction for function-local operands; namely, for operands of
type Argument, Instruction, and BasicBlock.  Currently, it is *only*
passed into RemapInstruction calls (and the transitive MapValue calls
that it makes).

When I split Metadata from Value I didn't understand the flag, and I
used it in a bunch of places for "global" metadata.

This commit doesn't have any functionality change, but prepares to
cleanup MapMetadata and MapValue.

llvm-svn: 265628
2016-04-07 00:26:43 +00:00
Sanjoy Das 2eac48de9e Return StringRef instead of a naked char*; NFC
llvm-svn: 262989
2016-03-09 02:34:19 +00:00
Sanjoy Das f13900f8ac [IRCE] Reflow comments; NFC
llvm-svn: 262988
2016-03-09 02:34:15 +00:00
Chandler Carruth 31088a9d58 [LPM] Factor all of the loop analysis usage updates into a common helper
routine.

We were getting this wrong in small ways and generally being very
inconsistent about it across loop passes. Instead, let's have a common
place where we do this. One minor downside is that this will require
some analyses like SCEV in more places than they are strictly needed.
However, this seems benign as these analyses are complete no-ops, and
without this consistency we can in many cases end up with the legacy
pass manager scheduling deciding to split up a loop pass pipeline in
order to run the function analysis half-way through. It is very, very
annoying to fix these without just being very pedantic across the board.

The only loop passes I've not updated here are ones that use
AU.setPreservesAll() such as IVUsers (an analysis) and the pass printer.
They seemed less relevant.

With this patch, almost all of the problems in PR24804 around loop pass
pipelines are fixed. The one remaining issue is that we run simplify-cfg
and instcombine in the middle of the loop pass pipeline. We've recently
added some loop variants of these passes that would seem substantially
cleaner to use, but this at least gets us much closer to the previous
state. Notably, the seven loop pass managers is down to three.

I've not updated the loop passes using LoopAccessAnalysis because that
analysis hasn't been fully wired into LoopSimplify/LCSSA, and it isn't
clear that those transforms want to support those forms anyways. They
all run late anyways, so this is harmless. Similarly, LSR is left alone
because it already carefully manages its forms and doesn't need to get
fused into a single loop pass manager with a bunch of other loop passes.

LoopReroll didn't use loop simplified form previously, and I've updated
the test case to match the trivially different output.

Finally, I've also factored all the pass initialization for the passes
that use this technique as well, so that should be done regularly and
reliably.

Thanks to James for the help reviewing and thinking about this stuff,
and Ben for help thinking about it as well!

Differential Revision: http://reviews.llvm.org/D17435

llvm-svn: 261316
2016-02-19 10:45:18 +00:00
Duncan P. N. Exon Smith 3a9c9e3dcd Scalar: Remove some implicit ilist iterator conversions, NFC
Remove some of the implicit ilist iterator conversions in
LLVMScalarOpts.  More to go.

llvm-svn: 250197
2015-10-13 18:26:00 +00:00
Sanjoy Das da0d79e0a0 [IRCE] Add INITIALIZE_PASS_DEPENDENCY invocations.
IRCE was just using INITIALIZE_PASS(), which is incorrect.

llvm-svn: 247122
2015-09-09 03:47:18 +00:00
Chandler Carruth 2f1fd1658f [PM] Port ScalarEvolution to the new pass manager.
This change makes ScalarEvolution a stand-alone object and just produces
one from a pass as needed. Making this work well requires making the
object movable, using references instead of overwritten pointers in
a number of places, and other refactorings.

I've also wired it up to the new pass manager and added a RUN line to
a test to exercise it under the new pass manager. This includes basic
printing support much like with other analyses.

But there is a big and somewhat scary change here. Prior to this patch
ScalarEvolution was never *actually* invalidated!!! Re-running the pass
just re-wired up the various other analyses and didn't remove any of the
existing entries in the SCEV caches or clear out anything at all. This
might seem OK as everything in SCEV that can uses ValueHandles to track
updates to the values that serve as SCEV keys. However, this still means
that as we ran SCEV over each function in the module, we kept
accumulating more and more SCEVs into the cache. At the end, we would
have a SCEV cache with every value that we ever needed a SCEV for in the
entire module!!! Yowzers. The releaseMemory routine would dump all of
this, but that isn't realy called during normal runs of the pipeline as
far as I can see.

To make matters worse, there *is* actually a key that we don't update
with value handles -- there is a map keyed off of Loop*s. Because
LoopInfo *does* release its memory from run to run, it is entirely
possible to run SCEV over one function, then over another function, and
then lookup a Loop* from the second function but find an entry inserted
for the first function! Ouch.

To make matters still worse, there are plenty of updates that *don't*
trip a value handle. It seems incredibly unlikely that today GVN or
another pass that invalidates SCEV can update values in *just* such
a way that a subsequent run of SCEV will incorrectly find lookups in
a cache, but it is theoretically possible and would be a nightmare to
debug.

With this refactoring, I've fixed all this by actually destroying and
recreating the ScalarEvolution object from run to run. Technically, this
could increase the amount of malloc traffic we see, but then again it is
also technically correct. ;] I don't actually think we're suffering from
tons of malloc traffic from SCEV because if we were, the fact that we
never clear the memory would seem more likely to have come up as an
actual problem before now. So, I've made the simple fix here. If in fact
there are serious issues with too much allocation and deallocation,
I can work on a clever fix that preserves the allocations (while
clearing the data) between each run, but I'd prefer to do that kind of
optimization with a test case / benchmark that shows why we need such
cleverness (and that can test that we actually make it faster). It's
possible that this will make some things faster by making the SCEV
caches have higher locality (due to being significantly smaller) so
until there is a clear benchmark, I think the simple change is best.

Differential Revision: http://reviews.llvm.org/D12063

llvm-svn: 245193
2015-08-17 02:08:17 +00:00
Cong Hou ab23bfbc0e Create a wrapper pass for BranchProbabilityInfo.
This new wrapper pass is useful when we want to do branch probability analysis conditionally (e.g. only in PGO mode) but don't want to add one more pass dependence.

http://reviews.llvm.org/D11241

llvm-svn: 242349
2015-07-15 22:48:29 +00:00
Alexander Kornienko f00654e31b Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)
Apparently, the style needs to be agreed upon first.

llvm-svn: 240390
2015-06-23 09:49:53 +00:00
Alexander Kornienko 70bc5f1398 Fixed/added namespace ending comments using clang-tidy. NFC
The patch is generated using this command:

tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
  -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
  llvm/lib/


Thanks to Eugene Kosov for the original patch!

llvm-svn: 240137
2015-06-19 15:57:42 +00:00
Sanjoy Das 45dc94a856 [IRCE] Fix how IRCE checks for no-sign-overflow.
IRCE requires the induction variables it handles to not sign-overflow.
The current scheme of checking if sext({X,+,S}) == {sext(X),+,sext(S)}
fails when SCEV simplifies sext(X) too.  After this change we //also//
check no-signed-wrap by looking at the flags set on the SCEVAddRecExpr.

llvm-svn: 233102
2015-03-24 19:29:22 +00:00
Sanjoy Das 337d46b36f [IRCE] Fix a regression introduced in r232444.
IRCE should not try to eliminate range checks that check an induction
variable against a loop-varying length.

llvm-svn: 233101
2015-03-24 19:29:18 +00:00
Benjamin Kramer 799003bf8c Re-sort includes with sort-includes.py and insert raw_ostream.h where it's used.
llvm-svn: 232998
2015-03-23 19:32:43 +00:00