DAG scheduling during isel. Most new functionality is currently
guarded by -enable-sched-cycles and -enable-sched-hazard.
Added InstrItineraryData::IssueWidth field, currently derived from
ARM itineraries, but could be initialized differently on other targets.
Added ScheduleHazardRecognizer::MaxLookAhead to indicate whether it is
active, and if so how many cycles of state it holds.
Added SchedulingPriorityQueue::HasReadyFilter to allowing gating entry
into the scheduler's available queue.
ScoreboardHazardRecognizer now accesses the ScheduleDAG in order to
get information about it's SUnits, provides RecedeCycle for bottom-up
scheduling, correctly computes scoreboard depth, tracks IssueCount, and
considers potential stall cycles when checking for hazards.
ScheduleDAGRRList now models machine cycles and hazards (under
flags). It tracks MinAvailableCycle, drives the hazard recognizer and
priority queue's ready filter, manages a new PendingQueue, properly
accounts for stall cycles, etc.
llvm-svn: 122541
section.
This helps because in practice sections form a dag with debug sections pointing
to text sections. Finishing up the text sections first makes the debug section
relaxation trivial.
llvm-svn: 122314
This implementation already exists as ConnectedVNInfoEqClasses in
LiveInterval.cpp, and it seems to be generally useful to have a light-weight way
of forming equivalence classes of small integers.
IntEqClasses doesn't allow enumeration of the elements in a class.
llvm-svn: 122293
it could only be tested indirectly, via instcombine, gvn or some other
pass that makes use of InstructionSimplify, which means that testcases
had to be carefully contrived to dance around any other transformations
that that pass did.
llvm-svn: 122264
createMachineVerifierPass and MachineFunction::verify.
The banner is printed before the machine code dump, just like the printer pass.
llvm-svn: 122113
may be called. If the entry block is empty, the insertion point iterator will be
the "end()" value. Calling ->getParent() on it (among others) causes problems.
Modify materializeFrameBaseRegister to take the machine basic block and insert
the frame base register at the beginning of that block. (It's very similar to
what the code does all ready. The only difference is that it will always insert
at the beginning of the entry block instead of after a previous materialization
of the frame base register. I doubt that that matters here.)
<rdar://problem/8782198>
llvm-svn: 122104
moves the iterator to end(), and it is valid to call it on end().
That means it is valid to call advanceTo() with any monotonic key sequence.
llvm-svn: 122092
IsSymbolRefDifferenceFullyResolved, it turns out this does change behavior on
enough cases for x86-32 that I would rather wait a bit on it.
- In practice, we will want to change this eventually because it only means we
generate less relocations (it also eliminates the need for the horrible
'.set' hack that Darwin requires in some places).
llvm-svn: 122042
This is a three-way interval list intersection between a virtual register, a
live interval union, and a loop. It will be used to identify interference-free
loops for live range splitting.
llvm-svn: 122034
IntervalMaps.
The IntervalMaps can have different template parameters, but the KeyT and Traits
types must be the same.
Tests are forthcoming.
llvm-svn: 121935
A MachineLoopRange contains the intervals of slot indexes covered by the blocks
in a loop. This representation of the loop blocks is more efficient to compare
against interfering registers during register coalescing.
llvm-svn: 121917
With this we don't need the EffectiveSize field anymore. Without that field
LayoutFragment only updates offsets and we don't need to invalidate the
current fragment when it is relaxed (only the ones following it).
This is also a very small improvement in the accuracy of the layout info as
we now use the after relaxation size immediately.
llvm-svn: 121857
registers that alias Reg, including itself. This is almost the same as the
existing getAliasSet() method, except for the inclusion of Reg.
The name matches the reflexive TRI::regsOverlap(x, y) relation.
It is very common to do stuff to a register and all its aliases:
stuff(Reg)
for (const unsigned *Alias = TRI->getAliasSet(Reg); *Alias; ++Alias)
stuff(*Alias);
That can now be written as the simpler:
for (const unsigned *Alias = TRI->getOverlaps(Reg); *Alias; ++Alias)
stuff(*Alias);
This change requires a bit more constant space for the alias lists because Reg
is included and because the empty alias list cannot be shared any longer.
If the getAliasSet method is eventually removed, this space can be reclaimed by
sharing overlap lists. For instance, %rax and %eax have identical overlap sets.
llvm-svn: 121800
AliasAnalysis consumers, PartialAlias will be treated as MayAlias.
For AliasAnalysis chaining, MayAlias says "procede to the next analysis".
PartialAlias will be used to indicate that the query should terminate,
even though it didn't reach MustAlias or NoAlias.
llvm-svn: 121507
the offset. Add a new fixup flag to represent this, and use it for the one fixups that I have a testcase for needing
this. It's quite likely that the other Thumb fixups will need this too, and to have their fixup encoding logic
adjusted accordingly.
llvm-svn: 121408
both forward and backward scheduling. Rename it to
ScoreboardHazardRecognizer (Scoreboard is one word). Remove integer
division from the scoreboard's critical path.
llvm-svn: 121274
This new register allocator is initially identical to RegAllocBasic, but it will
receive all of the tricks that RegAllocBasic won't get.
RegAllocGreedy will eventually replace linear scan.
llvm-svn: 121234
before:
4 assembler - Number of assembler layout and relaxation steps
78563 assembler - Number of emitted assembler fragments
8693904 assembler - Number of emitted object file bytes
271223 assembler - Number of evaluated fixups
330771677 assembler - Number of fragment layouts
5958 assembler - Number of relaxed instructions
2508361 mcexpr - Number of MCExpr evaluations
real 0m26.123s
user 0m25.694s
sys 0m0.388s
after:
4 assembler - Number of assembler layout and relaxation steps
78563 assembler - Number of emitted assembler fragments
8693904 assembler - Number of emitted object file bytes
271223 assembler - Number of evaluated fixups
231507 assembler - Number of fragment layouts
5958 assembler - Number of relaxed instructions
2508361 mcexpr - Number of MCExpr evaluations
real 0m2.500s
user 0m2.113s
sys 0m0.273s
And yes, the outputs are identical :-)
llvm-svn: 121207
zextOrTrunc(), and APSInt methods extend(), extOrTrunc() and new method
trunc(), to be const and to return a new value instead of modifying the
object in place.
llvm-svn: 121120
actuall addresses in a .o file, so it is better to let the MachO writer compute
it.
This is good for two reasons. First, areas that shouldn't care about
addresses now don't have access to it. Second, the layout of each section
is independent. I should use this in a subsequent commit to speed it up.
Most of the patch is just removing the section address computation. The two
interesting parts are the change on how we handle padding in the end
of sections and how MachO can get the address of a-b when a and b are in
different sections.
Since now the expression evaluation normally doesn't know the section address,
it will think that a-b needs relocation and let the MachO writer know. Once
it has computed the section addresses, it calls back the expression evaluation
with the section addresses to resolve these expressions.
The remaining problem is the handling of padding. Currently it will create
a special alignment fragment at the end. Since that fragment doesn't update
the alignment of the section, it needs the real address to be computed.
Since now the layout will not compute a-b with a and b in different sections,
the only effect that the special alignment fragment has is update the
address size of the section. This can also be done by the MachO writer.
llvm-svn: 121076
memcpy's like:
memcpy(A, B)
memcpy(A, C)
we cannot delete the first memcpy as dead if A and C might be aliases.
If so, we actually get:
memcpy(A, B)
memcpy(A, A)
which is not correct to transform into:
memcpy(A, A)
This patch was heavily influenced by Jakub Staszak's patch in PR8728, thanks
Jakub!
llvm-svn: 120974
foo = a - b
.long foo
instead of just
.long a - b
First, on darwin9 64 bits the assembler produces the wrong result. Second,
if "a" is the end of the section all darwin assemblers (9, 10 and mc) will not
consider a - b to be a constant but will if the dummy foo is created.
Split how we handle these cases. The first one is something MC should take care
of. The second one has to be handled by the caller.
llvm-svn: 120889
doing that if the target is darwin10 or newer.
This fixes
*) Direct object emission was producing objects without the workaround on
darwin9.
*) Assembly printing was producing objects with the workaround on linux.
llvm-svn: 120866
editing of the current interval.
These methods may cause coalescing, there are corresponding set*Unchecked
methods for editing without coalescing. The non-coalescing methods are useful
for applying monotonic transforms to all keys or values in a map without
accidentally coalescing transformed and untransformed intervals.
llvm-svn: 120829
contain only data. Handle them specially instead of using AddSectionToTheEnd.
This moves a hack from the generic assembler to the elf writer. It is also
a bit faster and should make other improvements easier.
llvm-svn: 120683
Scan the MachineFunction for DBG_VALUE instructions, and replace them with a
data structure similar to LiveIntervals. The live range of a DBG_VALUE is
determined by propagating it down the dominator tree until a new DBG_VALUE is
found. When a DBG_VALUE lives in a register, its live range is confined to the
live range of the register's value.
LiveDebugVariables runs before coalescing, so DBG_VALUEs are not artificially
extended when registers are joined.
The missing half will recreate DBG_VALUE instructions from the intervals when
register allocation is complete.
The pass is disabled by default. It can be enabled with the temporary command
line option -live-debug-variables.
llvm-svn: 120636
legalization time. Since at legalization time there is no mapping from
SDNode back to the corresponding LLVM instruction and the return
SDNode is target specific, this requires a target hook to check for
eligibility. Only x86 and ARM support this form of sibcall optimization
right now.
rdar://8707777
llvm-svn: 120501
- Use a DenseSet instead of a FoldingSet to cache
canonicalized nodes. This reduces the overhead
of double-hashing.
- Use reference counts in ImutAVLTree to much
more aggressively recover tree nodes that are
no longer usable. We can generate many
transient nodes while using add() and remove()
on ImmutableSet/ImmutableMaps to generate a final
set/map.
For the clang static analyzer (the main client
of these data structures), this results in
a slight speedup (0.5%) when analyzing sqlite3,
but much more importantly results in a 30-60%
reduction in peak memory usage when the analyzer
is analyzing a given function in a file. On
average that's about a ** 44% reduction ** in the
memory footprint of the static analyzer.
llvm-svn: 120459
about pairs of AA::Location's instead of looking for MemDep's
"Def" predicate. This is more powerful and general, handling
memset/memcpy/store all uniformly, and implementing PR8701 and
probably obsoleting parts of memcpyoptimizer.
This also fixes an obscure bug with init.trampoline and i8
stores, but I'm not surprised it hasn't been hit yet. Enhancing
init.trampoline to carry the size that it stores would allow
DSE to be much more aggressive about optimizing them.
llvm-svn: 120406
unbreaks test/Transforms/InstCombine/invariant.ll which was broken by r120382.
This is a fix-forward to do what I think Chris intended.
llvm-svn: 120388
This analysis is going to run immediately after LiveIntervals. It will stay
alive during register allocation and keep track of user variables mentioned in
DBG_VALUE instructions.
When the register allocator is moving values between registers and the stack, it
is very hard to keep track of DBG_VALUE instructions. We usually get it wrong.
This analysis maintains a data structure that makes it easy to update DBG_VALUE
instructions.
llvm-svn: 120385
Sometimes std::copy can become a memmove call, and that is not a good idea when
copying relatively few bytes as we are doing. We also get a small win by
changing two loops into one.
llvm-svn: 120265
We always disallowed overlapping inserts with different values, and this makes
the insertion code smaller and faster.
If an overwriting insert is needed, it can be added as a separate method that
trims any existing intervals before inserting. The immediate use cases for
IntervalMap don't need this - they only use disjoint insertions.
llvm-svn: 120264
Implement iterator::erase() in a simple version that erases nodes when they
become empty, but doesn't try to redistribute elements among siblings for better
packing.
Handle coalescing across leaf nodes which may require erasing entries.
llvm-svn: 120226
by my recent GVN improvement. Looking through a single layer of
PHI nodes when attempting to sink GEPs, we need to iteratively
look through arbitrary PHI nests.
llvm-svn: 120202
situations where on the common path an API can return a pointer to some direct
memory, but which on an exceptional path may need to return a pointer to a
temporary struct.
llvm-svn: 120201
- Double the vector's capacity when growing to avoid unneeccesary reallocation.
- Do the reallocation with realloc(3) which can expand the memory in place.
llvm-svn: 120183
The path also holds a reference to the root node, and that allows important
iterator accessors like start() and stop() to have no conditional code. (When
the compiler is clever enough to remove it.)
llvm-svn: 120165
Based on TR2/boost filesystem (v3) API, but modified to remove exceptions.
Do not include this file directly, when it is ready, it will be included by
include/llvm/System/Path.h.
llvm-svn: 120102
to use lowercase letters for the start of most
method names and to replace some method names
with more descriptive names (e.g., "getLeft()"
instead of "Left()"). No real functionality
change.
llvm-svn: 120070
This doesn't quite work yet because the calls to treeDecrement and treeIncrement
operate at the leaf level, not on pathNode(Level) as required.
llvm-svn: 120068
are constant. There was in fact one exception to this (phi nodes) - so
remove that exception (InstructionSimplify handles this so there should
be no loss).
llvm-svn: 120015
destination location of a memcpy/memmove. I'm not clear about whether
TBAA works on these, so I'm leaving it out for now. Dan, please revisit
this when convenient.
llvm-svn: 119928
allowing the memcpy to be eliminated.
Unfortunately, the requirements on byval's without explicit
alignment are really weak and impossible to predict in the
mid-level optimizer, so this doesn't kick in much with current
frontends. The fix is to change clang to set alignment on all
byval arguments.
llvm-svn: 119916
so don't claim they are. They are allocated using DAG.getNode, so attempts
to access MemSDNode fields results in reading off the end of the allocated
memory. This fixes crashes with "llc -debug" due to debug code trying to
print MemSDNode fields for these barrier nodes (since the crashes are not
deterministic, use valgrind to see this). Add some nasty checking to try
to catch this kind of thing in the future.
llvm-svn: 119901
It is now possible to navigate the B+-tree using NodeRef::subtree() and
NodeRef::size() without knowing the key and value template types used in the
tree.
llvm-svn: 119880
Daniel Dunbar