llvm-project/llvm/test/Transforms
Haicheng Wu 0aae2bc260 [CGP] Split large data structres to sink more GEPs
Accessing the members of a large data structures needs a lot of GEPs which
usually have large offsets due to the size of the underlying data structure. If
the offsets are too large to fit into the r+i addressing mode, these GEPs cannot
be sunk to their users' blocks and many extra registers are needed then to carry
the values of these GEPs.

This patch tries to split a large data struct starting from %base like the
following.

Before:
BB0:
  %base     =

BB1:
  %gep0     = gep %base, off0
  %gep1     = gep %base, off1
  %gep2     = gep %base, off2

BB2:
  %load1    = load %gep0
  %load2    = load %gep1
  %load3    = load %gep2

After:
BB0:
  %base     =
  %new_base = gep %base, off0

BB1:
  %new_gep0 = %new_base
  %new_gep1 = gep %new_base, off1 - off0
  %new_gep2 = gep %new_base, off2 - off0

BB2:
  %load1    = load i32, i32* %new_gep0
  %load2    = load i32, i32* %new_gep1
  %load3    = load i32, i32* %new_gep2

In the above example, the struct is split into two parts. The first part still
starts from %base and the second part starts from %new_base. After the
splitting, %new_gep1 and %new_gep2 have smaller offsets and then can be sunk to
BB2 and folded into their users.

The algorithm to split data structure is simple and very similar to the work of
merging SExts. First, it collects GEPs that have large offsets when iterating
the blocks. Second, it splits the underlying data structures and updates the
collected GEPs to use smaller offsets.

Differential Revision: https://reviews.llvm.org/D42759

llvm-svn: 332015
2018-05-10 18:27:36 +00:00
..
ADCE [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
AddDiscriminators [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
AggressiveInstCombine [AggressiveInstCombine] convert a chain of 'and-shift' bits into masked compare 2018-05-09 23:08:15 +00:00
AlignmentFromAssumptions [AlignmentFromAssumptions] Set source and dest alignments of memory intrinsiscs separately 2018-02-22 18:55:59 +00:00
ArgumentPromotion [ArgumentPromotion] don't break musttail invariant PR36543 2018-03-02 00:59:27 +00:00
AtomicExpand
BDCE [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
BlockExtractor BlockExtractor: Don’t delete functions directly 2018-03-12 22:28:18 +00:00
BranchFolding
CallSiteSplitting [CallSiteSplit] Make sure we remove nonnull if the parameter turns out to be a constant. 2018-04-23 20:09:08 +00:00
CalledValuePropagation
CodeExtractor [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
CodeGenPrepare [CGP] Split large data structres to sink more GEPs 2018-05-10 18:27:36 +00:00
ConstProp [TargetLibraryInfo] fix finite mathlib function availability 2018-01-08 17:38:09 +00:00
ConstantHoisting [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
ConstantMerge
Coroutines [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
CorrelatedValuePropagation [CVP] simplify phi with constant incoming values that match common variable edge values 2018-04-10 20:42:39 +00:00
CrossDSOCFI
DCE [DCE] Salvage debug info from dead insts 2018-02-15 22:26:18 +00:00
DeadArgElim [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
DeadStoreElimination [DSE] Teach the pass about partial overwrite of atomic memory intrinsics 2018-05-10 15:12:49 +00:00
DivRemPairs
EarlyCSE [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
EliminateAvailableExternally
EntryExitInstrumenter [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
ExpandMemCmp/X86 [x86, MemCmpExpansion] allow 2 pairs of loads per block (PR33325) 2018-01-06 16:16:04 +00:00
Float2Int
ForcedFunctionAttrs
FunctionAttrs [CaptureTracking] Handle capturing of launder.invariant.group 2018-05-05 10:23:27 +00:00
FunctionImport [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
GCOVProfiling [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
GVN [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
GVNHoist [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
GVNSink
GlobalDCE
GlobalMerge
GlobalOpt [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
GlobalSplit
GuardWidening [LoopGuardWidening] Make PostDomTree optional 2018-04-27 23:15:56 +00:00
IPConstantProp [IPSCCP] do not break musttail invariant (PR36485) 2018-03-01 01:19:18 +00:00
IRCE [IRCE] Only check for NSW on equality predicates 2018-04-18 13:50:28 +00:00
IndVarSimplify [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
IndirectBrExpand Introduce the "retpoline" x86 mitigation technique for variant #2 of the speculative execution vulnerabilities disclosed today, specifically identified by CVE-2017-5715, "Branch Target Injection", and is one of the two halves to Spectre.. 2018-01-22 22:05:25 +00:00
InferAddressSpaces [AMDGPU] Switch to the new addr space mapping by default 2018-02-02 16:07:16 +00:00
InferFunctionAttrs
Inline [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
InstCombine [InstCombine] regenerate full checks; NFC 2018-05-10 17:05:38 +00:00
InstMerge [MergedLoadStoreMotion] Fix a debug invariant bug in mergeStores 2018-05-09 06:52:12 +00:00
InstNamer
InstSimplify [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
InterleavedAccess
Internalize
JumpThreading Fix the issue that ComputeValueKnownInPredecessors only handles the case when 2018-05-01 14:47:24 +00:00
LCSSA [LCSSA] Do not remove used PHI nodes in formLCSSAForInstructions 2018-05-08 06:59:47 +00:00
LICM [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoadStoreVectorizer [LoadStoreVectorize] Ignore interleaved invariant loads. 2018-04-24 15:28:47 +00:00
LoopDataPrefetch [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoopDeletion [LoopDeletion] Handle users in unreachable block 2018-01-12 07:24:43 +00:00
LoopDistribute [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoopIdiom [LoopIdiomRecognize] Add a test case to show incorrect transformation of an infinite loop with side effets into a countable loop using ctlz. 2018-05-03 23:50:29 +00:00
LoopInterchange [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoopLoadElim
LoopPredication [LoopPredication] Add profitability check based on BPI 2018-03-22 16:03:59 +00:00
LoopReroll [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoopRotate [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoopSimplify [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoopSimplifyCFG
LoopStrengthReduce Revert "[PowerPC] LSR tunings for PowerPC" 2018-03-09 16:08:55 +00:00
LoopUnroll [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
LoopUnswitch [NFC] Use FileCheck in test 2018-04-24 04:42:37 +00:00
LoopVectorize [LV] Add lit testcase for bitcast problem. NFC 2018-05-09 13:34:57 +00:00
LoopVersioning
LoopVersioningLICM
LowerAtomic
LowerExpectIntrinsic
LowerGuardIntrinsic
LowerInvoke
LowerSwitch
LowerTypeTests Remove explicit setting of the CFI jumptable section name, it does not appear 2018-05-07 21:30:15 +00:00
Mem2Reg [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
MemCpyOpt [MemCpyOpt] Skip optimizing basic blocks not reachable from entry 2018-04-23 19:55:04 +00:00
MergeFunc [PR37339] Fix assertion in FunctionComparator::cmpInlineAsm 2018-05-10 15:05:47 +00:00
MergeICmps [MergeICmp] Split blocks that do other work. 2018-04-09 13:14:06 +00:00
MetaRenamer Remove alignment argument from memcpy/memmove/memset in favour of alignment attributes (Step 1) 2018-01-19 17:13:12 +00:00
NameAnonGlobals
NaryReassociate Add early exit on reassociation of 0 expression. 2018-03-07 02:17:08 +00:00
NewGVN [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
ObjCARC [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
PGOProfile [ICP] Do not attempt type matching for variable length arguments. 2018-04-25 17:19:21 +00:00
PartiallyInlineLibCalls [PartiallyInlineLibCalls][x86] add TTI hook to allow sqrt inlining to depend on arg rather than result 2017-11-27 21:15:43 +00:00
PhaseOrdering [PhaseOrdering] remove stale comments; NFC 2018-05-09 23:10:46 +00:00
PlaceSafepoints Remove alignment argument from memcpy/memmove/memset in favour of alignment attributes (Step 1) 2018-01-19 17:13:12 +00:00
PreISelIntrinsicLowering
PruneEH [PM][FunctionAttrs] add NoUnwind attribute inference to PostOrderFunctionAttrs pass 2018-03-23 21:46:16 +00:00
Reassociate [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
Reg2Mem
RewriteStatepointsForGC [RewriteStatepoints] Fix stale parse points 2018-03-05 22:27:30 +00:00
SCCP Revert r328307: [IPSCCP] Use constant range information for comparisons of parameters. 2018-03-23 12:49:39 +00:00
SLPVectorizer [AMDGPU] Support horizontal vectorization of min/max. 2018-05-09 21:18:34 +00:00
SROA [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
SafeStack [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
SampleProfile [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
Scalarizer [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
SeparateConstOffsetFromGEP [AMDGPU] Change constant addr space to 4 2018-02-13 18:00:25 +00:00
SimpleLoopUnswitch [PM/LoopUnswitch] Fix a bug in the loop block set formation of the new 2018-04-24 10:33:08 +00:00
SimplifyCFG [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
Sink [Sink] Really really fix predicate in legality check 2018-01-11 21:28:57 +00:00
SpeculateAroundPHIs Add a new pass to speculate around PHI nodes with constant (integer) operands when profitable. 2017-11-28 11:32:31 +00:00
SpeculativeExecution
StraightLineStrengthReduce
StripDeadPrototypes
StripSymbols
StructurizeCFG StructurizeCFG: Test for branch divergence correctly 2018-04-04 10:58:15 +00:00
SyntheticCountsPropagation Add a pass to generate synthetic function entry counts. 2018-01-09 19:39:35 +00:00
TailCallElim [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
ThinLTOBitcodeWriter Fix build failures for r330387 on buildbots that don't build the X86 target 2018-04-20 02:26:12 +00:00
Util [DebugInfo] Add DILabel metadata and intrinsic llvm.dbg.label. 2018-05-09 02:40:45 +00:00
WholeProgramDevirt Runtime flag to control branch funnel threshold 2018-04-06 21:32:36 +00:00