Commit Graph

30 Commits

Author SHA1 Message Date
Andrew Trick 3a99693c5a Improve on r192635, ExeDepsFix for avx, and add a test case.
rdar:15221834 False AVX register dependencies cause 5x slowdown on
flops-5/6 and significant slowdown on several others.

This was blocking the switch to MI-Sched.

llvm-svn: 192669
2013-10-15 03:39:43 +00:00
Andrew Trick b6d56be69d Fix the ExecutionDepsFix pass to handle AVX instructions.
This pass is needed to break false dependencies. Without it, unlucky
register assignment can result in wild (5x) swings in
performance. This pass was trying to handle AVX but not getting it
right. AVX doesn't have partial register defs, it has unused register
reads in which the high bits of a source operand are copied into the
unused bits of the dest.

Fixing this requires conservative liveness analysis. This is awkard
because the pass already has its own pseudo-liveness. However, proper
liveness is expensive, and we would like to use a generic utility to
compute it. The fix only invokes liveness on-demand. It is rare to
detect a case that needs undef-read dependence breaking, but when it
happens, it can be needed many times within a very large block.

I think the existing heuristic which uses a register window of 16 is
too conservative for loop-carried false dependencies. If the loop is a
reduction. The out-of-order engine may be able to execute several loop
iterations in parallel. However, I'll leave this tuning exercise for
next time.

llvm-svn: 192635
2013-10-14 22:19:03 +00:00
Craig Topper e1c1d363a5 Use SmallVectorImpl instead of SmallVector for iterators and references to avoid specifying the vector size unnecessarily.
llvm-svn: 185512
2013-07-03 05:11:49 +00:00
Michael J. Spencer df1ecbd734 Replace Count{Leading,Trailing}Zeros_{32,64} with count{Leading,Trailing}Zeros.
llvm-svn: 182680
2013-05-24 22:23:49 +00:00
Chandler Carruth ed0881b2a6 Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

llvm-svn: 169131
2012-12-03 16:50:05 +00:00
Jakob Stoklund Olesen 0736442683 Merge MRI::isPhysRegOrOverlapUsed() into isPhysRegUsed().
All callers of these functions really want the isPhysRegOrOverlapUsed()
functionality which also checks aliases. For historical reasons, targets
without register aliases were calling isPhysRegUsed() instead.

Change isPhysRegUsed() to also check aliases, and switch all
isPhysRegOrOverlapUsed() callers to isPhysRegUsed().

llvm-svn: 166117
2012-10-17 18:44:18 +00:00
Silviu Baranga 3c314990e6 Fixed a bug in the ExecutionDependencyFix pass that caused dependencies to not propagate through implicit defs.
llvm-svn: 165102
2012-10-03 08:29:36 +00:00
Benjamin Kramer bde9176663 Fix typos found by http://github.com/lyda/misspell-check
llvm-svn: 157885
2012-06-02 10:20:22 +00:00
Jakob Stoklund Olesen 54038d796c Switch all register list clients to the new MC*Iterator interface.
No functional change intended.

Sorry for the churn. The iterator classes are supposed to help avoid
giant commits like this one in the future. The TableGen-produced
register lists are getting quite large, and it may be necessary to
change the table representation.

This makes it possible to do so without changing all clients (again).

llvm-svn: 157854
2012-06-01 23:28:30 +00:00
Craig Topper 1d32658877 Use uint16_t to store register overlaps to reduce static data.
llvm-svn: 152001
2012-03-04 10:43:23 +00:00
Jakob Stoklund Olesen 3588a43e3a Move common code into an MRI function.
llvm-svn: 147071
2011-12-21 19:50:05 +00:00
Evan Cheng 7f8e563a69 Add bundle aware API for querying instruction properties and switch the code
generator to it. For non-bundle instructions, these behave exactly the same
as the MC layer API.

For properties like mayLoad / mayStore, look into the bundle and if any of the
bundled instructions has the property it would return true.
For properties like isPredicable, only return true if *all* of the bundled
instructions have the property.
For properties like canFoldAsLoad, isCompare, conservatively return false for
bundles.

llvm-svn: 146026
2011-12-07 07:15:52 +00:00
Jakob Stoklund Olesen 02845410f9 Fix PR11422.
This was a bug in keeping track of the available domains when merging
domain values.

The wrong domain mask caused ExecutionDepsFix to try to move VANDPSYrr
to the integer domain which is only available in AVX2.

Also add an assertion to catch future attempts at emitting AVX2
instructions.

llvm-svn: 145096
2011-11-23 04:03:08 +00:00
Jakob Stoklund Olesen e14ef7e6f8 Check all overlaps when looking for used registers.
A function using any RC alias is enough to enable the ExeDepsFix pass.

llvm-svn: 144636
2011-11-15 08:20:43 +00:00
Jakob Stoklund Olesen f8ad336bc4 Break false dependencies before partial register updates.
Two new TargetInstrInfo hooks lets the target tell ExecutionDepsFix
about instructions with partial register updates causing false unwanted
dependencies.

The ExecutionDepsFix pass will break the false dependencies if the
updated register was written in the previoius N instructions.

The small loop added to sse-domains.ll runs twice as fast with
dependency-breaking instructions inserted.

llvm-svn: 144602
2011-11-15 01:15:30 +00:00
Jakob Stoklund Olesen 543bef6ead Track register ages more accurately.
Keep track of the last instruction to define each register individually
instead of per DomainValue.  This lets us track more accurately when a
register was last written.

Also track register ages across basic blocks.  When entering a new
basic block, use the least stale predecessor def as a worst case
estimate for register age.

The register age is used to arbitrate between conflicting domains. The
most recently defined register wins.

llvm-svn: 144601
2011-11-15 01:15:25 +00:00
Jakob Stoklund Olesen 3dc89c9768 Collapse DomainValues across loop back-edges.
During the initial RPO traversal of the basic blocks, remember the ones
that are incomplete because of back-edges from predecessors that haven't
been visited yet.

After the initial RPO, revisit all those loop headers so the incoming
DomainValues on the back-edges can be properly collapsed.

This will properly fix execution domains on software pipelined code,
like the included test case.

llvm-svn: 144151
2011-11-09 01:06:56 +00:00
Jakob Stoklund Olesen 53ec977cd2 Link to the live DomainValue after merging.
When merging two uncollapsed DomainValues, place a link to the active
DomainValue from the passive DomainValue.  This allows old stale
references to the passive DomainValue to be updated to point to the
active DomainValue.

The new resolve() function finds the active DomainValue and updates the
pointer.

This change makes old live-out lists more useful since they may contain
uncollapsed DomainValues that have since been merged into other
DomainValues.

llvm-svn: 144149
2011-11-09 00:06:18 +00:00
Jakob Stoklund Olesen b7e44a3f5f Track reference count independently from clear().
This allows clear() to be called on a DomainValue with references.

llvm-svn: 144147
2011-11-08 23:26:00 +00:00
Jakob Stoklund Olesen 5d08293999 Call release() directly when cleaning up the remaining DomainValues.
There is no need to involve the LiveRegs array and kill() any longer.

llvm-svn: 144133
2011-11-08 22:05:17 +00:00
Jakob Stoklund Olesen 9e338bb0f3 Rename all methods to follow style guide.
No functional change.

llvm-svn: 144132
2011-11-08 21:57:47 +00:00
Jakob Stoklund Olesen 1438e191bd Handle reference counts in one function: release().
This new function will decrement the reference count, and collapse a
domain value when the last reference is gone.

This simplifies DomainValue reference counting, and decouples it from
the LiveRegs array.

llvm-svn: 144131
2011-11-08 21:57:44 +00:00
Jakob Stoklund Olesen 1205881820 Clear old DomainValue after merging.
The old value may still be referenced by some live-out list, and we
don't wan't to collapse those instructions twice.

This fixes the "Can only swizzle VMOVD" assertion in some armv7 SPEC
builds.

<rdar://problem/10413292>

llvm-svn: 144117
2011-11-08 20:57:04 +00:00
Jakob Stoklund Olesen a70e9417fb Kill and collapse outstanding DomainValues.
DomainValues that are only used by "don't care" instructions are now
collapsed to the first possible execution domain after all basic blocks
have been processed.  This typically means the PS domain on x86.

For example, the vsel_i64 and vsel_double functions in sse2-blend.ll are
completely collapsed to the PS domain instead of containing a mix of
execution domains created by isel.

llvm-svn: 144037
2011-11-07 23:08:21 +00:00
Jakob Stoklund Olesen 68e197e151 Use a reverse post order instead of a DFS order.
The enterBasicBlock() function is combining live-out values from
predecessor blocks.  The RPO traversal means that more predecessors
have been visited when that happens, only back-edges are missing.

llvm-svn: 144025
2011-11-07 21:59:29 +00:00
Jakob Stoklund Olesen 736cf46c3e Extract two methods. No functional change.
llvm-svn: 144020
2011-11-07 21:40:27 +00:00
Jakob Stoklund Olesen 44dcc589b3 MBB doesn't need to be a class member.
llvm-svn: 144015
2011-11-07 21:23:42 +00:00
Jakob Stoklund Olesen baffa7d35d Fix pass name after the source was moved.
llvm-svn: 144014
2011-11-07 21:23:39 +00:00
Jakob Stoklund Olesen bd5109f14d Rename class and clean up source.
No functional change intended.

llvm-svn: 140664
2011-09-28 00:01:56 +00:00
Jakob Stoklund Olesen 934b7d7645 Rename SSEDomainFix -> lib/CodeGen/ExecutionDepsFix.
I'll clean up the source in the next commit.

llvm-svn: 140663
2011-09-28 00:01:54 +00:00