llvm-project/llvm/lib/CodeGen
Jakob Stoklund Olesen adc6a4ca5d Reapply r134047 now that the world is ready for it.
This patch will sometimes choose live range split points next to
interference instead of always splitting next to a register point. That
means spill code can now appear almost anywhere, and it was necessary
to fix code that didn't expect that.

The difficult places were:

- Between a CALL returning a value on the x87 stack and the
  corresponding FpPOP_RETVAL (was FpGET_ST0). Probably also near x87
  inline assembly, but that didn't actually show up in testing.

- Between a CALL popping arguments off the stack and the corresponding
  ADJCALLSTACKUP.

Both are fixed now. The only place spill code can't appear is after
terminators, see SplitAnalysis::getLastSplitPoint.

Original commit message:

Rewrite RAGreedy::splitAroundRegion, now with cool ASCII art.

This function has to deal with a lot of special cases, and the old
version got it wrong sometimes. In particular, it would sometimes leave
multiple uses in the stack interval in a single block. That causes bad
code with multiple reloads in the same basic block.

The new version handles block entry and exit in a single pass. It first
eliminates all the easy cases, and then goes on to create a local
interval for the blocks with difficult interference. Previously, we
would only create the local interval for completely isolated blocks.

It can happen that the stack interval becomes completely empty because
we could allocate a register in all edge bundles, and the new local
intervals deal with the interference. The empty stack interval is
harmless, but we need to remove a SplitKit assertion that checks for
empty intervals.

llvm-svn: 134125
2011-06-30 01:30:39 +00:00
..
AsmPrinter Handle debug info for i128 constants. 2011-06-24 20:46:11 +00:00
SelectionDAG Remove getRegClassForInlineAsmConstraint and all dependencies. 2011-06-30 01:20:03 +00:00
AggressiveAntiDepBreaker.cpp More refactoring. Move getRegClass from TargetOperandInfo to TargetInstrInfo. 2011-06-27 21:26:13 +00:00
AggressiveAntiDepBreaker.h Teach antidependency breakers to use RegisterClassInfo. 2011-06-16 21:56:21 +00:00
AllocationOrder.cpp Rename TRI::getAllocationOrder() to getRawAllocationOrder(). 2011-06-16 23:31:16 +00:00
AllocationOrder.h Get allocation orders from RegisterClassInfo when possible. 2011-06-06 21:02:04 +00:00
Analysis.cpp Minor code re-structuring. 2011-03-19 17:03:16 +00:00
AntiDepBreaker.h Update DBG_VALUEs while breaking anti dependencies. 2011-06-02 21:26:52 +00:00
BranchFolding.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
BranchFolding.h Add 132986 back, but avoid non-determinism if a bb address gets reused. 2011-06-14 15:31:54 +00:00
CMakeLists.txt Fix CMake build by removing this now dead file. 2011-06-28 02:03:12 +00:00
CalcSpillWeights.cpp More refactoring. Move getRegClass from TargetOperandInfo to TargetInstrInfo. 2011-06-27 21:26:13 +00:00
CallingConvLower.cpp Rename the ParmContext enum values to make a bit more sense and add a small 2011-06-10 20:37:36 +00:00
CodeGen.cpp Remove the experimental (and unused) pre-ra splitting pass. Greedy regalloc can split live ranges. 2011-06-27 23:40:45 +00:00
CodePlacementOpt.cpp Fix a ton of comment typos found by codespell. Patch by 2011-04-15 05:18:47 +00:00
CriticalAntiDepBreaker.cpp More refactoring. Move getRegClass from TargetOperandInfo to TargetInstrInfo. 2011-06-27 21:26:13 +00:00
CriticalAntiDepBreaker.h Teach antidependency breakers to use RegisterClassInfo. 2011-06-16 21:56:21 +00:00
DeadMachineInstructionElim.cpp Track live-out physical registers in MachineDCE. 2011-06-27 15:00:36 +00:00
DwarfEHPrepare.cpp The ARM stuff already calls the Resume function, not the Resume_or_Rethrow. It 2011-06-01 01:49:35 +00:00
ELF.h Fix a ton of comment typos found by codespell. Patch by 2011-04-15 05:18:47 +00:00
ELFCodeEmitter.cpp Get rid of a bunch of duplicated ELF enum values. 2010-07-16 07:53:29 +00:00
ELFCodeEmitter.h
ELFWriter.cpp Fix a FIXME by making GlobalVariable::getInitializer() return a 2011-06-19 18:37:11 +00:00
ELFWriter.h Fix a FIXME by making GlobalVariable::getInitializer() return a 2011-06-19 18:37:11 +00:00
EdgeBundles.cpp Function::getNumBlockIDs() should be used instead of Function::size() to set the upper limit on the block IDs since basic blocks might get removed (simplified away) after being initially numbered. Plus the test case, in which SelectionDAGBuilder::visitBr() calls llvm::MachineFunction::removeFromMBBNumbering(), which introduces the hole in numbering leading to an assert in llc (prior to the fix). 2011-06-16 00:03:21 +00:00
ExpandISelPseudos.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
GCMetadata.cpp Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which 2010-10-19 17:21:58 +00:00
GCMetadataPrinter.cpp
GCStrategy.cpp Rename TargetFrameInfo into TargetFrameLowering. Also, put couple of FIXMEs and fixes here and there. 2011-01-10 12:39:04 +00:00
IfConversion.cpp Sink SubtargetFeature and TargetInstrItineraries (renamed MCInstrItineraries) into MC. 2011-06-29 01:14:12 +00:00
InlineSpiller.cpp Avoid hoisting spills when looking at a copy from another register that is also 2011-05-11 18:25:10 +00:00
InterferenceCache.cpp Precompute interference for neighbor blocks as long as there is no interference. 2011-04-09 02:59:05 +00:00
InterferenceCache.h Precompute interference for neighbor blocks as long as there is no interference. 2011-04-09 02:59:05 +00:00
IntrinsicLowering.cpp Recognize inline asm 'rev /bin/bash, ' as a bswap intrinsic call. 2011-01-08 01:24:27 +00:00
LLVMTargetMachine.cpp Use the verbose asm flag instead of a new flag for decoding the LSDA. 2011-06-17 20:55:01 +00:00
LatencyPriorityQueue.cpp Various bits of framework needed for precise machine-level selection 2010-12-24 05:03:26 +00:00
LiveDebugVariables.cpp Remove an assertion to fix PR9872. 2011-05-08 19:21:08 +00:00
LiveDebugVariables.h Update LiveDebugVariables after live range splitting. 2011-05-06 18:00:02 +00:00
LiveInterval.cpp Replace a broken LiveInterval::MergeValueInAsValue() with something simpler. 2011-03-19 23:02:49 +00:00
LiveIntervalAnalysis.cpp When a physreg is live-in and live through a basic block, make sure its live 2011-04-30 19:12:33 +00:00
LiveIntervalUnion.cpp Speed up eviction by stopping collectInterferingVRegs as soon as the spill 2011-04-11 21:47:01 +00:00
LiveIntervalUnion.h Reuse live interval union between functions. This saves a bit of compile time 2011-04-11 23:57:14 +00:00
LiveRangeEdit.cpp Add some statistics to the splitting and spilling frameworks. 2011-05-05 17:22:53 +00:00
LiveRangeEdit.h Update LiveDebugVariables after live range splitting. 2011-05-06 18:00:02 +00:00
LiveStackAnalysis.cpp Teach TargetRegisterInfo how to cram stack slot indexes in with the virtual and 2011-01-09 21:17:37 +00:00
LiveVariables.cpp Reduce vector reallocations. 2011-03-08 17:28:36 +00:00
LocalStackSlotAllocation.cpp Rename TargetFrameInfo into TargetFrameLowering. Also, put couple of FIXMEs and fixes here and there. 2011-01-10 12:39:04 +00:00
LowerSubregs.cpp Trailing whitespace. 2011-02-25 22:53:20 +00:00
MachineBasicBlock.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
MachineBranchProbabilityInfo.cpp Introduce MachineBranchProbabilityInfo class, which has similar API to 2011-06-16 20:22:37 +00:00
MachineCSE.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
MachineDominators.cpp Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which 2010-10-19 17:21:58 +00:00
MachineFunction.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
MachineFunctionAnalysis.cpp Clean up a funky pass registration that got passed over when I got rid of static constructors. 2011-01-04 00:55:21 +00:00
MachineFunctionPass.cpp
MachineFunctionPrinterPass.cpp Reapply r110396, with fixes to appease the Linux buildbot gods. 2010-08-06 18:33:48 +00:00
MachineInstr.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
MachineLICM.cpp Sink SubtargetFeature and TargetInstrItineraries (renamed MCInstrItineraries) into MC. 2011-06-29 01:14:12 +00:00
MachineLoopInfo.cpp Begin adding static dependence information to passes, which will allow us to 2010-10-12 19:48:12 +00:00
MachineLoopRanges.cpp Add MachineLoopRange comparators for sorting loop lists by number and by area. 2010-12-17 18:13:52 +00:00
MachineModuleInfo.cpp Fixed version of 121434 with no new memory leaks. 2010-12-10 07:39:47 +00:00
MachineModuleInfoImpls.cpp
MachinePassRegistry.cpp
MachineRegisterInfo.cpp Remove RegClass2VRegMap from MachineRegisterInfo. 2011-06-27 23:54:40 +00:00
MachineSSAUpdater.cpp Fix PR7096. When a block containing multiple defs is tail duplicated, the 2010-05-10 17:14:26 +00:00
MachineSink.cpp Fix a couple of places where changes are made but not tracked. 2011-04-11 18:47:20 +00:00
MachineVerifier.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
Makefile
ObjectCodeEmitter.cpp
OcamlGC.cpp
OptimizePHIs.cpp Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which 2010-10-19 17:21:58 +00:00
PHIElimination.cpp Remove unused STL header includes. 2011-04-23 19:53:52 +00:00
PHIEliminationUtils.cpp Move the FindCopyInsertPoint method of PHIElimination to a new standalone 2010-12-05 19:51:05 +00:00
PHIEliminationUtils.h Move the FindCopyInsertPoint method of PHIElimination to a new standalone 2010-12-05 19:51:05 +00:00
Passes.cpp Update comment. 2011-04-30 03:13:08 +00:00
PeepholeOptimizer.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
PostRASchedulerList.cpp Teach antidependency breakers to use RegisterClassInfo. 2011-06-16 21:56:21 +00:00
ProcessImplicitDefs.cpp Place context in member variables instead of passing around pointers. 2011-03-14 20:57:14 +00:00
PrologEpilogInserter.cpp Move CallFrameSetupOpcode and CallFrameDestroyOpcode to TargetInstrInfo. 2011-06-28 21:14:33 +00:00
PrologEpilogInserter.h Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which 2010-10-19 17:21:58 +00:00
PseudoSourceValue.cpp Merge System into Support. 2010-11-29 18:16:10 +00:00
README.txt Fix a ton of comment typos found by codespell. Patch by 2011-04-15 05:18:47 +00:00
RegAllocBase.h Switch AllocationOrder to using RegisterClassInfo instead of a BitVector 2011-06-03 20:34:53 +00:00
RegAllocBasic.cpp There is only one register coalescer. Merge it into the base class and 2011-06-26 22:34:10 +00:00
RegAllocFast.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
RegAllocGreedy.cpp Reapply r134047 now that the world is ready for it. 2011-06-30 01:30:39 +00:00
RegAllocLinearScan.cpp Remove the experimental (and unused) pre-ra splitting pass. Greedy regalloc can split live ranges. 2011-06-27 23:40:45 +00:00
RegAllocPBQP.cpp There is only one register coalescer. Merge it into the base class and 2011-06-26 22:34:10 +00:00
RegisterClassInfo.cpp Add TargetRegisterInfo::getRawAllocationOrder(). 2011-06-16 17:42:25 +00:00
RegisterClassInfo.h Starting to refactor Target to separate out code that's needed to fully describe 2011-06-24 01:44:41 +00:00
RegisterCoalescer.cpp make compose and isMoveInstr static functions. 2011-06-29 20:55:48 +00:00
RegisterCoalescer.h make compose and isMoveInstr static functions. 2011-06-29 20:55:48 +00:00
RegisterScavenging.cpp Handle <def,undef> in the second loop as well. 2011-05-02 20:36:53 +00:00
RenderMachineFunction.cpp Add TargetRegisterInfo::getRawAllocationOrder(). 2011-06-16 17:42:25 +00:00
RenderMachineFunction.h Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which 2010-10-19 17:21:58 +00:00
ScheduleDAG.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
ScheduleDAGEmit.cpp Change all self assignments X=X to (void)X, so that we can turn on a 2010-12-23 00:58:24 +00:00
ScheduleDAGInstrs.cpp Sink SubtargetFeature and TargetInstrItineraries (renamed MCInstrItineraries) into MC. 2011-06-29 01:14:12 +00:00
ScheduleDAGInstrs.h Update DBG_VALUEs while breaking anti dependencies. 2011-06-02 21:26:52 +00:00
ScheduleDAGPrinter.cpp Pass the graph to the DOTGraphTraits.getEdgeAttributes(). 2011-02-27 04:11:03 +00:00
ScoreboardHazardRecognizer.cpp Sink SubtargetFeature and TargetInstrItineraries (renamed MCInstrItineraries) into MC. 2011-06-29 01:14:12 +00:00
ShadowStackGC.cpp Replace the existing forms of ConstantArray::get() with a single form 2011-06-22 09:24:39 +00:00
ShrinkWrapping.cpp Fix a ton of comment typos found by codespell. Patch by 2011-04-15 05:18:47 +00:00
SjLjEHPrepare.cpp fix the varargs version of StructType::get to not require an LLVMContext, making usage 2011-06-18 22:48:56 +00:00
SlotIndexes.cpp Use basic block numbers as indexes when mapping slot index ranges. 2011-04-02 06:03:31 +00:00
SpillPlacement.cpp Build the Hopfield network incrementally when splitting global live ranges. 2011-04-09 02:59:09 +00:00
SpillPlacement.h Build the Hopfield network incrementally when splitting global live ranges. 2011-04-09 02:59:09 +00:00
Spiller.cpp Remove unused STL header includes. 2011-04-23 19:53:52 +00:00
Spiller.h Change the Spiller interface to take a LiveRangeEdit reference. 2011-03-10 01:51:42 +00:00
SplitKit.cpp Reapply r134047 now that the world is ready for it. 2011-06-30 01:30:39 +00:00
SplitKit.h Reapply r134047 now that the world is ready for it. 2011-06-30 01:30:39 +00:00
Splitter.cpp merge SimpleRegisterCoalescing.h into RegisterCoalescer.h. 2011-06-26 21:54:28 +00:00
Splitter.h Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which 2010-10-19 17:21:58 +00:00
StackProtector.cpp Inline check that's used only once. 2011-03-29 17:12:55 +00:00
StackSlotColoring.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
StrongPHIElimination.cpp Fix a ton of comment typos found by codespell. Patch by 2011-04-15 05:18:47 +00:00
TailDuplication.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
TargetInstrInfoImpl.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
TargetLoweringObjectFileImpl.cpp Use the presence of the __compact_unwind section to indicate that a target 2011-06-23 05:13:28 +00:00
TwoAddressInstructionPass.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
UnreachableBlockElim.cpp Fix PR10029 - VerifyCoalescing failure on patterns_dfa.c of 445.gobmk. 2011-05-27 05:04:51 +00:00
VirtRegMap.cpp Also count identity copies. 2011-05-06 17:59:57 +00:00
VirtRegMap.h This method belonged in VirtRegMap. 2011-02-19 00:38:43 +00:00
VirtRegRewriter.cpp - Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and 2011-06-28 19:10:37 +00:00
VirtRegRewriter.h

README.txt

//===---------------------------------------------------------------------===//

Common register allocation / spilling problem:

        mul lr, r4, lr
        str lr, [sp, #+52]
        ldr lr, [r1, #+32]
        sxth r3, r3
        ldr r4, [sp, #+52]
        mla r4, r3, lr, r4

can be:

        mul lr, r4, lr
        mov r4, lr
        str lr, [sp, #+52]
        ldr lr, [r1, #+32]
        sxth r3, r3
        mla r4, r3, lr, r4

and then "merge" mul and mov:

        mul r4, r4, lr
        str lr, [sp, #+52]
        ldr lr, [r1, #+32]
        sxth r3, r3
        mla r4, r3, lr, r4

It also increase the likelihood the store may become dead.

//===---------------------------------------------------------------------===//

bb27 ...
        ...
        %reg1037 = ADDri %reg1039, 1
        %reg1038 = ADDrs %reg1032, %reg1039, %NOREG, 10
    Successors according to CFG: 0x8b03bf0 (#5)

bb76 (0x8b03bf0, LLVM BB @0x8b032d0, ID#5):
    Predecessors according to CFG: 0x8b0c5f0 (#3) 0x8b0a7c0 (#4)
        %reg1039 = PHI %reg1070, mbb<bb76.outer,0x8b0c5f0>, %reg1037, mbb<bb27,0x8b0a7c0>

Note ADDri is not a two-address instruction. However, its result %reg1037 is an
operand of the PHI node in bb76 and its operand %reg1039 is the result of the
PHI node. We should treat it as a two-address code and make sure the ADDri is
scheduled after any node that reads %reg1039.

//===---------------------------------------------------------------------===//

Use local info (i.e. register scavenger) to assign it a free register to allow
reuse:
        ldr r3, [sp, #+4]
        add r3, r3, #3
        ldr r2, [sp, #+8]
        add r2, r2, #2
        ldr r1, [sp, #+4]  <==
        add r1, r1, #1
        ldr r0, [sp, #+4]
        add r0, r0, #2

//===---------------------------------------------------------------------===//

LLVM aggressively lift CSE out of loop. Sometimes this can be negative side-
effects:

R1 = X + 4
R2 = X + 7
R3 = X + 15

loop:
load [i + R1]
...
load [i + R2]
...
load [i + R3]

Suppose there is high register pressure, R1, R2, R3, can be spilled. We need
to implement proper re-materialization to handle this:

R1 = X + 4
R2 = X + 7
R3 = X + 15

loop:
R1 = X + 4  @ re-materialized
load [i + R1]
...
R2 = X + 7 @ re-materialized
load [i + R2]
...
R3 = X + 15 @ re-materialized
load [i + R3]

Furthermore, with re-association, we can enable sharing:

R1 = X + 4
R2 = X + 7
R3 = X + 15

loop:
T = i + X
load [T + 4]
...
load [T + 7]
...
load [T + 15]
//===---------------------------------------------------------------------===//

It's not always a good idea to choose rematerialization over spilling. If all
the load / store instructions would be folded then spilling is cheaper because
it won't require new live intervals / registers. See 2003-05-31-LongShifts for
an example.

//===---------------------------------------------------------------------===//

With a copying garbage collector, derived pointers must not be retained across
collector safe points; the collector could move the objects and invalidate the
derived pointer. This is bad enough in the first place, but safe points can
crop up unpredictably. Consider:

        %array = load { i32, [0 x %obj] }** %array_addr
        %nth_el = getelementptr { i32, [0 x %obj] }* %array, i32 0, i32 %n
        %old = load %obj** %nth_el
        %z = div i64 %x, %y
        store %obj* %new, %obj** %nth_el

If the i64 division is lowered to a libcall, then a safe point will (must)
appear for the call site. If a collection occurs, %array and %nth_el no longer
point into the correct object.

The fix for this is to copy address calculations so that dependent pointers
are never live across safe point boundaries. But the loads cannot be copied
like this if there was an intervening store, so may be hard to get right.

Only a concurrent mutator can trigger a collection at the libcall safe point.
So single-threaded programs do not have this requirement, even with a copying
collector. Still, LLVM optimizations would probably undo a front-end's careful
work.

//===---------------------------------------------------------------------===//

The ocaml frametable structure supports liveness information. It would be good
to support it.

//===---------------------------------------------------------------------===//

The FIXME in ComputeCommonTailLength in BranchFolding.cpp needs to be
revisited. The check is there to work around a misuse of directives in inline
assembly.

//===---------------------------------------------------------------------===//

It would be good to detect collector/target compatibility instead of silently
doing the wrong thing.

//===---------------------------------------------------------------------===//

It would be really nice to be able to write patterns in .td files for copies,
which would eliminate a bunch of explicit predicates on them (e.g. no side 
effects).  Once this is in place, it would be even better to have tblgen 
synthesize the various copy insertion/inspection methods in TargetInstrInfo.

//===---------------------------------------------------------------------===//

Stack coloring improvements:

1. Do proper LiveStackAnalysis on all stack objects including those which are
   not spill slots.
2. Reorder objects to fill in gaps between objects.
   e.g. 4, 1, <gap>, 4, 1, 1, 1, <gap>, 4 => 4, 1, 1, 1, 1, 4, 4

//===---------------------------------------------------------------------===//

The scheduler should be able to sort nearby instructions by their address. For
example, in an expanded memset sequence it's not uncommon to see code like this:

  movl $0, 4(%rdi)
  movl $0, 8(%rdi)
  movl $0, 12(%rdi)
  movl $0, 0(%rdi)

Each of the stores is independent, and the scheduler is currently making an
arbitrary decision about the order.

//===---------------------------------------------------------------------===//

Another opportunitiy in this code is that the $0 could be moved to a register:

  movl $0, 4(%rdi)
  movl $0, 8(%rdi)
  movl $0, 12(%rdi)
  movl $0, 0(%rdi)

This would save substantial code size, especially for longer sequences like
this. It would be easy to have a rule telling isel to avoid matching MOV32mi
if the immediate has more than some fixed number of uses. It's more involved
to teach the register allocator how to do late folding to recover from
excessive register pressure.