Commit Graph

544 Commits

Author SHA1 Message Date
Cameron Zwarich 97dae4d361 If we can't avoid running loop-simplify twice for now, at least avoid running
iv-users twice.

llvm-svn: 125318
2011-02-10 23:53:14 +00:00
Eric Christopher da6bd45088 Revert this in an attempt to bring the builders back.
llvm-svn: 125257
2011-02-10 01:48:24 +00:00
Cameron Zwarich 58c8670ab2 Turn this pass ordering:
Natural Loop Information
 Loop Pass Manager
   Canonicalize natural loops
 Scalar Evolution Analysis
 Loop Pass Manager
   Induction Variable Users
   Canonicalize natural loops
   Induction Variable Users
   Loop Strength Reduction

into this:

Scalar Evolution Analysis
Loop Pass Manager
  Canonicalize natural loops
  Induction Variable Users
  Loop Strength Reduction

This fixes <rdar://problem/8869639>. I also filed PR9184 on doing this sort of
thing automatically, but it seems easier to just change the ordering of the
passes if this is the only case.

llvm-svn: 125254
2011-02-10 01:07:54 +00:00
Dan Gohman de7f699754 Don't split any loop backedges, including backedges of loops other than
the active loop. This is generally desirable, and it avoids trouble
in situations such as the testcase in PR9123, though the failure
mode depends on use-list order, so it is infeasible to test.

llvm-svn: 125065
2011-02-08 00:55:13 +00:00
Cameron Zwarich b703654edc Remove code for updating dominance frontiers and some outdated references to
dominance and post-dominance frontiers.

llvm-svn: 123725
2011-01-18 04:11:31 +00:00
Chris Lattner bf0aa927cc split dom frontier handling stuff out to its own DominanceFrontier header,
so that Dominators.h is *just* domtree.  Also prune #includes a bit.

llvm-svn: 122714
2011-01-02 22:09:33 +00:00
Dan Gohman 20d9ce21ef Move SCEV::dominates and properlyDominates to ScalarEvolution.
llvm-svn: 119570
2010-11-17 21:41:58 +00:00
Dan Gohman afd6db9932 Move SCEV::isLoopInvariant and hasComputableLoopEvolution to be member
functions of ScalarEvolution, in preparation for memoization and
other optimizations.

llvm-svn: 119562
2010-11-17 21:23:15 +00:00
Owen Anderson a4fefc1949 Passes do not need to recursively initialize passes that they preserve, if
they do not also require them.  This allows us to reduce inter-pass linkage
dependencies.

llvm-svn: 116854
2010-10-19 20:08:44 +00:00
Owen Anderson 6c18d1aac0 Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which
must be called in the pass's constructor.  This function uses static dependency declarations to recursively initialize
the pass's dependencies.

Clients that only create passes through the createFooPass() APIs will require no changes.  Clients that want to use the
CommandLine options for passes will need to manually call the appropriate initialization functions in PassInitialization.h
before parsing commandline arguments.

I have tested this with all standard configurations of clang and llvm-gcc on Darwin.  It is possible that there are problems
with the static dependencies that will only be visible with non-standard options.  If you encounter any crash in pass
registration/creation, please send the testcase to me directly.

llvm-svn: 116820
2010-10-19 17:21:58 +00:00
Owen Anderson 8ac477ffb5 Begin adding static dependence information to passes, which will allow us to
perform initialization without static constructors AND without explicit initialization
by the client.  For the moment, passes are required to initialize both their
(potential) dependencies and any passes they preserve.  I hope to be able to relax
the latter requirement in the future.

llvm-svn: 116334
2010-10-12 19:48:12 +00:00
Dan Gohman 2fd85d7cd2 Filter out illegal formulae after updating offsets, not before, so that
formulae which become illegal as a result of the offset updating don't
escape.

This is for rdar://8529692. No testcase yet, because the given cases
hit use-list ordering differences.

llvm-svn: 116093
2010-10-08 19:33:26 +00:00
Dan Gohman 5947e1626a Delete the FormulaSorter class and inline its one method into its
one user. This code will be restructured soon and FormulaSorter
is getting in the way.

llvm-svn: 116012
2010-10-07 23:52:18 +00:00
Dan Gohman 1b61fd9bff Fix a spello.
llvm-svn: 116011
2010-10-07 23:43:09 +00:00
Dan Gohman 34f37e0d04 Charge a formula for explicit multiplies on scaled registers too,
not just base registers.

llvm-svn: 116010
2010-10-07 23:41:58 +00:00
Dan Gohman 49d638b45a Use size_t for consistency.
llvm-svn: 116009
2010-10-07 23:37:58 +00:00
Dan Gohman 8e72611058 When merging one use into another, transfer the offsets from
the old use to the new one.

llvm-svn: 116008
2010-10-07 23:36:45 +00:00
Dan Gohman a7b68d6d95 Fix LSR to keep the RegUseTracker up to date when combining users.
This doesn't usually matter, because the other heuristics usually
succeed regardless, but it's good to keep the register use
bookkeeping consistent.

llvm-svn: 116005
2010-10-07 23:33:43 +00:00
Owen Anderson df7a4f2515 Now with fewer extraneous semicolons!
llvm-svn: 115996
2010-10-07 22:25:06 +00:00
Dan Gohman 16ef49686c Delete an unused function.
llvm-svn: 114841
2010-09-27 16:58:21 +00:00
Gabor Greif a06741b356 do not rely on the implicit-dereference semantics of dyn_cast_or_null
llvm-svn: 114278
2010-09-18 11:55:34 +00:00
Dan Gohman 110ed64fbb Revert 112442 and 112440 until the compile time problems introduced
by 112440 are resolved.

llvm-svn: 112692
2010-09-01 01:45:53 +00:00
Dan Gohman 002ff89cbd Optionally rerun dedicated-register filtering after applying
other filtering techniques, as those may allow it to filter
out more obviously unprofitable candidates.

llvm-svn: 112441
2010-08-29 16:39:22 +00:00
Dan Gohman f031792cc6 Fix several areas in LSR to do a better job keeping the main
LSRInstance data structures up to date. This fixes some
pessimizations caused by stale data which will be exposed
in an upcoming change.

llvm-svn: 112440
2010-08-29 16:32:54 +00:00
Dan Gohman e9e0873b08 Refactor the three main groups of code out of
NarrowSearchSpaceUsingHeuristics into separate functions.

llvm-svn: 112439
2010-08-29 16:09:42 +00:00
Dan Gohman 37a0f68036 Delete a bogus check.
llvm-svn: 112438
2010-08-29 15:30:29 +00:00
Dan Gohman b6a520d63c Add some comments.
llvm-svn: 112437
2010-08-29 15:27:08 +00:00
Dan Gohman bf673e0652 Move this debug output into GenerateAllReuseFormula, to declutter
the high-level logic.

llvm-svn: 112436
2010-08-29 15:21:38 +00:00
Dan Gohman d366b6d5c8 Delete an unused declaration.
llvm-svn: 112435
2010-08-29 15:19:11 +00:00
Dan Gohman 4f13bbfefc Do one lookup instead of two.
llvm-svn: 112434
2010-08-29 15:18:49 +00:00
Dan Gohman 129a816ee6 Process the step before the start, because it's usually the simpler
of the two.

llvm-svn: 111495
2010-08-19 01:02:31 +00:00
Dan Gohman 89fdbaf99a Instead of having CollectSubexpr's categorize operands as interesting or
uninteresting, just put all the operands on one list and make
GenerateReassociations make the decision about what's interesting.
This is simpler, and it avoids an extra ScalarEvolution::getAddExpr call.

llvm-svn: 111133
2010-08-16 15:50:00 +00:00
Dan Gohman 9b7632df26 Put add operands in ScalarEvolution-canonical order, when convenient.
This isn't necessary, because ScalarEvolution sorts them anyway,
but it's tidier this way.

llvm-svn: 111132
2010-08-16 15:39:27 +00:00
Dan Gohman 081ffcd00b Fix LSR's ExtractImmediate and ExtractSymbol to avoid calling
ScalarEvolution::getAddExpr, which can be pretty expensive, when nothing
has changed, which is pretty common.

llvm-svn: 111042
2010-08-13 21:17:19 +00:00
Owen Anderson a7aed18624 Reapply r110396, with fixes to appease the Linux buildbot gods.
llvm-svn: 110460
2010-08-06 18:33:48 +00:00
Owen Anderson bda59bd247 Revert r110396 to fix buildbots.
llvm-svn: 110410
2010-08-06 00:23:35 +00:00
Owen Anderson 755aceb5d0 Don't use PassInfo* as a type identifier for passes. Instead, use the address of the static
ID member as the sole unique type identifier.  Clean up APIs related to this change.

llvm-svn: 110396
2010-08-05 23:42:04 +00:00
Dan Gohman ba81fc16a5 Fix whitespace.
llvm-svn: 110223
2010-08-04 17:43:57 +00:00
Oscar Fuentes 40b31ad3ee Prefix `next' iterator operation with `llvm::'.
Fixes potential ambiguity problems on VS 2010.

Patch by nobled!

llvm-svn: 110029
2010-08-02 06:00:15 +00:00
Owen Anderson a57b97e7e7 Fix batch of converting RegisterPass<> to INTIALIZE_PASS().
llvm-svn: 109045
2010-07-21 22:09:45 +00:00
Dan Gohman 1415208292 Don't merge uses when they are targetting fixup sites with
different widths. In a use with a narrower fixup, formulae
may be wider than the fixup, in which case the high bits
aren't necessarily meaningful, so it isn't safe to reuse
them for uses with wider fixups.

This fixes PR7618, though the testcase is too large for a
reasonable regression test, since it heavily dependes on
hitting LSR's heuristics in a certain way.

llvm-svn: 108455
2010-07-15 20:24:58 +00:00
Dan Gohman a1501b9c50 Use dbgs() instead of errs() in a DEBUG.
llvm-svn: 108453
2010-07-15 20:12:42 +00:00
Dan Gohman 4afd412d6b Watch out for a constant offset cancelling out a base register, forming
a zero. This situation arrises in Fortran code with induction variables
that start at 1 instead of 0. This fixes PR7651.

llvm-svn: 108424
2010-07-15 15:14:45 +00:00
Gabor Greif 8ae3095286 use getArgOperand instead of getOperand
llvm-svn: 107270
2010-06-30 09:15:28 +00:00
Dan Gohman fb9712bdae In GenerateReassociations, don't bother thinking about individual
SCEVUnknown values which are loop-variant, as LSR can't do anything
interesting with these values in any case. This fixes very slow compile
times on loops which have large numbers of such values.

llvm-svn: 106897
2010-06-25 22:32:18 +00:00
Dan Gohman 963b1c142e A few minor micro-optimizations.
llvm-svn: 106764
2010-06-24 16:57:52 +00:00
Dan Gohman 47ddf76d89 Teach getExactSDiv to evaluate x/1 to x up front, as it's a common
enough special case, and it theoretically allows more folding because
it works even when x is unanalyzable.

llvm-svn: 106763
2010-06-24 16:51:25 +00:00
Dan Gohman ab5422200b Fix copy+pasto issues in isMulSExtable.
llvm-svn: 106759
2010-06-24 16:45:11 +00:00
Dan Gohman 1081f1a0f5 Fix OptimizeMax to handle an odd case where one of the max operands
is another max which folds. This fixes PR7454.

llvm-svn: 106594
2010-06-22 23:07:13 +00:00
Dan Gohman dd41bba517 Use A.append(...) instead of A.insert(A.end(), ...) when A is a
SmallVector, and other SmallVector simplifications.

llvm-svn: 106452
2010-06-21 19:47:52 +00:00
Dan Gohman 32655906e4 Add a TODO comment.
llvm-svn: 106397
2010-06-19 21:30:18 +00:00
Dan Gohman 51d00092b6 Include the use kind along with the expression in the key of the
use sharing map. The reconcileNewOffset logic already forces a
separate use if the kinds differ, so incorporating the kind in the
key means we can track more sharing opportunities.

More sharing means fewer total uses to track, which means smaller
problem sizes, which means the conservative throttles don't kick
in as often.

llvm-svn: 106396
2010-06-19 21:29:59 +00:00
Dan Gohman 297fb8b9fc Don't include things in anonymous namespaces that don't need it.
llvm-svn: 106395
2010-06-19 21:21:39 +00:00
Dan Gohman 67b4403101 Don't track users of undef values; they aren't interesting for
register pressure.

llvm-svn: 105501
2010-06-04 23:16:05 +00:00
Dan Gohman 9b48b856ea DominatorTree.getNode can return null for unreachable blocks.
llvm-svn: 104290
2010-05-20 22:46:54 +00:00
Dan Gohman 86110fa2bb Minor code cleanups.
llvm-svn: 104287
2010-05-20 22:25:20 +00:00
Dan Gohman 6295f2ebb8 Make Solve check its own post-condition, to reduce clutter in the
top-level LSRInstance logic.

llvm-svn: 104278
2010-05-20 20:59:23 +00:00
Dan Gohman a4ca28a3ae Add comments.
llvm-svn: 104276
2010-05-20 20:52:00 +00:00
Dan Gohman 927bcaadda More code cleanups. Use iterators instead of indices when indices
aren't needed.

llvm-svn: 104273
2010-05-20 20:33:18 +00:00
Dan Gohman 4c4043cf34 Fix OptimizeShadowIV to set Changed. Change OptimizeLoopTermCond to set
Changed directly instead of using a return value.

Rename FilterOutUndesirableDedicatedRegisters's Changed variable to
distinguish it from LSRInstance's Changed member.

llvm-svn: 104269
2010-05-20 20:05:31 +00:00
Dan Gohman 8ec018cedf Add some comments.
llvm-svn: 104268
2010-05-20 20:00:41 +00:00
Dan Gohman 8ce95cc3c5 Simplify this code. Don't do a DomTreeNode lookup for each visited block.
llvm-svn: 104267
2010-05-20 20:00:25 +00:00
Dan Gohman ab5fb7f559 Minor code cleanups.
llvm-svn: 104263
2010-05-20 19:44:23 +00:00
Dan Gohman ee2fea3cd7 When canonicalizing icmp operand order to put the loop invariant
operand on the left, the interesting operand is on the right. This
fixes a bug where LSR was failing to recognize ICmpZero uses,
which led it to be unable to reverse the induction variable in the
attached testcase.

Delete test/CodeGen/X86/stack-color-with-reg-2.ll, because its test
is extremely fragile and hard to meaningfully update.

llvm-svn: 104262
2010-05-20 19:26:52 +00:00
Dan Gohman fdf9874ba7 Set Changed to true when canonicalizing ICmp operand order; even though
it isn't a very interesting change, it's a change nonetheless.

llvm-svn: 104260
2010-05-20 19:16:03 +00:00
Dan Gohman 981563d0ba Rename a variable to avoid shadowing.
llvm-svn: 104234
2010-05-20 16:41:11 +00:00
Dan Gohman 6b733fc189 Minor code simplification.
llvm-svn: 104232
2010-05-20 16:23:28 +00:00
Dan Gohman 80a9608442 Move the code for deleting BaseRegs and LSRUses into helper functions,
and fix a bug that valgrind noticed where the code would std::swap an
element with itself.

llvm-svn: 104225
2010-05-20 15:17:54 +00:00
Dan Gohman 20fab456da Teach LSR how to cope better with unrolled loops on targets where
the addressing modes don't make this trivially easy. This allows
it to avoid falling into the less precise heuristics in more
cases.

llvm-svn: 104186
2010-05-19 23:43:12 +00:00
Dan Gohman beebef4137 Add a comment.
llvm-svn: 104089
2010-05-18 23:55:57 +00:00
Dan Gohman 50f8f2c23d Fix the predicate which checks for non-sensical formulae which have
constants in registers which partially cancel out their immediate fields.

llvm-svn: 104088
2010-05-18 23:48:08 +00:00
Dan Gohman 4cf99b5303 Factor out the code for recomputing an LSRUse's Regs set after some
of its formulae have been removed into a helper function, and also
teach it how to update the RegUseTracker.

llvm-svn: 104087
2010-05-18 23:42:37 +00:00
Dan Gohman a4eca05174 Factor out code for estimating search space complexity into a helper
function.

llvm-svn: 104082
2010-05-18 22:51:59 +00:00
Dan Gohman 63e9015248 Add some more debug output.
llvm-svn: 104080
2010-05-18 22:41:32 +00:00
Dan Gohman f1c7b1b42f Factor out the code for deleting a formula from an LSRUse into
a helper function.

llvm-svn: 104079
2010-05-18 22:39:15 +00:00
Dan Gohman 8aca7ef903 Make some debug output more informative.
llvm-svn: 104078
2010-05-18 22:37:37 +00:00
Dan Gohman 06ab08f795 Print an error message in Formula::print if the HasBaseReg flag
is inconsistent with the BaseRegs field. It's not print's job to
assert on an invalid condition, but it can make one more obvious.

llvm-svn: 104077
2010-05-18 22:35:55 +00:00
Dan Gohman 248c41d108 Rename RegUseTracker's RegUses member to RegUsesMap to avoid
confusion with LSRInstance's RegUses member.

llvm-svn: 104076
2010-05-18 22:33:00 +00:00
Douglas Gregor 6739a89117 Fixes for Microsoft Visual Studio 2010, from Steven Watanabe!
llvm-svn: 103457
2010-05-11 06:17:44 +00:00
Dan Gohman d0800241d2 When pruning candidate formulae out of an LSRUse, update the
LSRUse's Regs set after all pruning is done, rather than trying
to do it on the fly, which can produce an incomplete result.

This fixes a case where heuristic pruning was stripping all
formulae from a use, which led the solver to enter an infinite
loop.

Also, add a few asserts to diagnose this kind of situation.

llvm-svn: 103328
2010-05-07 23:36:59 +00:00
Dan Gohman 1d2ded75e2 Use getConstant instead of getIntegerSCEV. The two are basically the
same, now that getConstant has overloads consistent with ConstantInt::get.

llvm-svn: 102965
2010-05-03 22:09:21 +00:00
Dan Gohman 534ba376f6 Generalize LSR's OptimizeMax to handle the new kinds of max expressions
that indvars may use, now that indvars is recognizing le and ge loops.

llvm-svn: 102235
2010-04-24 03:13:44 +00:00
Dan Gohman 997bbc54d6 Fix LSR to tolerate cases where ScalarEvolution initially
misses an opportunity to fold add operands, but folds them
after LSR has separated them out. This fixes rdar://7886751.

llvm-svn: 102157
2010-04-23 01:55:05 +00:00
Dan Gohman e637ff5e9a Remove the Expr member from IVUsers. Instead of remembering the expression,
just ask ScalarEvolution for it on demand. This helps IVUsers be more robust
in the case of expressions changing underneath it. This fixes PR6862.

llvm-svn: 101819
2010-04-19 21:48:58 +00:00
Dan Gohman 27c8e79839 Delete this code, which is no longer needed.
llvm-svn: 101033
2010-04-12 08:00:22 +00:00
Dan Gohman 607e02b33a When determining a canonical insert position, don't climb deeper
into adjacent loops. Also, ensure that the insert position is
dominated by the loop latch of any loop in the post-inc set which
has a latch.

llvm-svn: 100906
2010-04-09 22:07:05 +00:00
Dan Gohman 42ec4eb351 When looking for loop-invariant users, look through no-op instructions,
so that an unfortunately placed bitcast doesn't pin a value in a
register.

llvm-svn: 100883
2010-04-09 19:12:34 +00:00
Dan Gohman d2df643ddb Refactor the code for computing the insertion point for an expression into
a separate function.

llvm-svn: 100845
2010-04-09 02:00:38 +00:00
Dan Gohman 9b5d0bb774 Avoid allocating a value of zero in a register if the initial formula
inputs happen to negate each other.

llvm-svn: 100828
2010-04-08 23:36:27 +00:00
Dan Gohman 4ce1fb1448 Add variants of ult, ule, etc. which take a uint64_t RHS, for convenience.
llvm-svn: 100824
2010-04-08 23:03:40 +00:00
Dan Gohman 4506539d84 When expanding expressions which are using post-inc mode for multiple loops,
ensure that the expansion is dominated by the increments of those loops.

llvm-svn: 100748
2010-04-08 05:57:57 +00:00
Dan Gohman d006ab90dd Generalize IVUsers to track arbitrary expressions rather than expressions
explicitly split into stride-and-offset pairs. Also, add the
ability to track multiple post-increment loops on the same expression.

This refines the concept of "normalizing" SCEV expressions used for
to post-increment uses, and introduces a dedicated utility routine for
normalizing and denormalizing expressions.

This fixes the expansion of expressions which are post-increment users
of more than one loop at a time. More broadly, this takes LSR another
step closer to being able to reason about more than one loop at a time.

llvm-svn: 100699
2010-04-07 22:27:08 +00:00
Dan Gohman d42e09d91e Ignore debug intrinsics in yet more places.
llvm-svn: 99580
2010-03-26 00:33:27 +00:00
Gabor Greif c78d720f02 rename use_const_iterator to const_use_iterator for consistency's sake
llvm-svn: 99564
2010-03-25 23:06:16 +00:00
Dan Gohman 29707de4fe Make SCEVExpander and LSR more aggressive about hoisting expressions out
of loops.

llvm-svn: 97642
2010-03-03 05:29:13 +00:00
Dan Gohman 52f5563973 Non-affine post-inc SCEV expansions have more code which must be
emitted after the increment. Make sure the insert position
reflects this. This fixes PR6453.

llvm-svn: 97537
2010-03-02 01:59:21 +00:00
Dan Gohman 8b0a419eb1 Spelling fixes.
llvm-svn: 97453
2010-03-01 17:49:51 +00:00
Dan Gohman 8c16b38262 Remove unused variables and parameters.
llvm-svn: 96780
2010-02-22 04:11:59 +00:00
Dan Gohman 4506fcb3c2 When emitting an instruction which depends on both a post-incremented
induction variable value and a loop-variant value, don't force the
insert position to be at the post-increment position, because it may
not be dominated by the loop-variant value. This fixes a
use-before-def problem noticed on PPC.

llvm-svn: 96774
2010-02-22 03:59:54 +00:00
Dan Gohman 4eebb94094 Rename getSDiv to getExactSDiv to reflect its behavior in cases where
the division would have a remainder.

llvm-svn: 96693
2010-02-19 19:35:48 +00:00