[ModuloSchedule] Peel out prologs and epilogs, generate actual code
Summary:
This extends the PeelingModuloScheduleExpander to generate prolog and epilog code,
and correctly stitch uses through the prolog, kernel, epilog DAG.
The key concept in this patch is to ensure that all transforms are *local*; only a
function of a block and its immediate predecessor and successor. By defining the problem in this way
we can inductively rewrite the entire DAG using only local knowledge that is easy to
reason about.
For example, we assume that all prologs and epilogs are near-perfect clones of the
steady-state kernel. This means that if a block has an instruction that is predicated out,
we can redirect all users of that instruction to that equivalent instruction in our
immediate predecessor. As all blocks are clones, every instruction must have an equivalent in
every other block.
Similarly we can make the assumption by construction that if a value defined in a block is used
outside that block, the only possible user is its immediate successors. We maintain this
even for values that are used outside the loop by creating a limited form of LCSSA.
This code isn't small, but it isn't complex.
Enabled a bunch of testing from Hexagon. There are a couple of tests not enabled yet;
I'm about 80% sure there isn't buggy codegen but the tests are checking for patterns
that we don't produce. Those still need a bit more investigation. In the meantime we
(Google) are happy with the code produced by this on our downstream SMS implementation,
and believe it generates correct code.
Subscribers: mgorny, hiraditya, jsji, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D68205
llvm-svn: 373462
2019-10-02 20:46:44 +08:00
|
|
|
; RUN: llc -march=hexagon -enable-pipeliner -enable-bsb-sched=0 -join-liveintervals=false < %s -pipeliner-experimental-cg=true | FileCheck %s
|
2018-03-21 03:35:09 +08:00
|
|
|
|
[CodeGen] Add a new pass for PostRA sink
Summary:
This pass sinks COPY instructions into a successor block, if the COPY is not
used in the current block and the COPY is live-in to a single successor
(i.e., doesn't require the COPY to be duplicated). This avoids executing the
the copy on paths where their results aren't needed. This also exposes
additional opportunites for dead copy elimination and shrink wrapping.
These copies were either not handled by or are inserted after the MachineSink
pass. As an example of the former case, the MachineSink pass cannot sink
COPY instructions with allocatable source registers; for AArch64 these type
of copy instructions are frequently used to move function parameters (PhyReg)
into virtual registers in the entry block..
For the machine IR below, this pass will sink %w19 in the entry into its
successor (%bb.1) because %w19 is only live-in in %bb.1.
```
%bb.0:
%wzr = SUBSWri %w1, 1
%w19 = COPY %w0
Bcc 11, %bb.2
%bb.1:
Live Ins: %w19
BL @fun
%w0 = ADDWrr %w0, %w19
RET %w0
%bb.2:
%w0 = COPY %wzr
RET %w0
```
As we sink %w19 (CSR in AArch64) into %bb.1, the shrink-wrapping pass will be
able to see %bb.0 as a candidate.
With this change I observed 12% more shrink-wrapping candidate and 13% more dead copies deleted in spec2000/2006/2017 on AArch64.
Reviewers: qcolombet, MatzeB, thegameg, mcrosier, gberry, hfinkel, john.brawn, twoh, RKSimon, sebpop, kparzysz
Reviewed By: sebpop
Subscribers: evandro, sebpop, sfertile, aemerson, mgorny, javed.absar, kristof.beyls, llvm-commits
Differential Revision: https://reviews.llvm.org/D41463
llvm-svn: 328237
2018-03-23 04:06:47 +08:00
|
|
|
; XFAIL: *
|
|
|
|
; This test is failing after post-ra machine sinking.
|
|
|
|
|
2018-03-21 03:35:09 +08:00
|
|
|
; Test that we generate the correct Phi values when there is a Phi that
|
|
|
|
; references another Phi. We need to examine the other Phi to get the
|
|
|
|
; correct value. We need to do this even if we haven't generated the
|
|
|
|
; kernel code for the other Phi yet.
|
|
|
|
|
2018-03-27 00:33:16 +08:00
|
|
|
; CHECK: v[[REG0:[0-9]+]] = v[[REG1:[0-9]+]]
|
2018-03-21 03:35:09 +08:00
|
|
|
; CHECK: loop0
|
2018-03-27 00:33:16 +08:00
|
|
|
; Check for copy REG0 = REG1 (via vcombine):
|
|
|
|
; CHECK: v{{[0-9]+}}:[[REG0]] = vcombine(v{{[0-9]+}},v[[REG1]])
|
2018-03-21 03:35:09 +08:00
|
|
|
; CHECK: endloop0
|
|
|
|
|
|
|
|
; Function Attrs: nounwind
|
|
|
|
define void @f0() #0 {
|
|
|
|
b0:
|
|
|
|
br i1 undef, label %b1, label %b2
|
|
|
|
|
|
|
|
b1: ; preds = %b1, %b0
|
|
|
|
%v0 = phi i32 [ %v7, %b1 ], [ 0, %b0 ]
|
|
|
|
%v1 = phi <16 x i32> [ %v4, %b1 ], [ undef, %b0 ]
|
|
|
|
%v2 = phi <16 x i32> [ %v1, %b1 ], [ undef, %b0 ]
|
|
|
|
%v3 = tail call <16 x i32> @llvm.hexagon.V6.valignb(<16 x i32> %v1, <16 x i32> %v2, i32 62)
|
|
|
|
%v4 = tail call <16 x i32> @llvm.hexagon.V6.vaddh(<16 x i32> undef, <16 x i32> undef)
|
|
|
|
%v5 = tail call <16 x i32> @llvm.hexagon.V6.valignb(<16 x i32> %v4, <16 x i32> %v1, i32 2)
|
|
|
|
%v6 = tail call <16 x i32> @llvm.hexagon.V6.vabsdiffh(<16 x i32> %v3, <16 x i32> %v5)
|
|
|
|
store <16 x i32> %v6, <16 x i32>* null, align 64
|
|
|
|
%v7 = add nsw i32 %v0, 1
|
|
|
|
%v8 = icmp slt i32 %v7, undef
|
|
|
|
br i1 %v8, label %b1, label %b2
|
|
|
|
|
|
|
|
b2: ; preds = %b1, %b0
|
|
|
|
ret void
|
|
|
|
}
|
|
|
|
|
|
|
|
; Function Attrs: nounwind readnone
|
|
|
|
declare <16 x i32> @llvm.hexagon.V6.vaddh(<16 x i32>, <16 x i32>) #1
|
|
|
|
|
|
|
|
; Function Attrs: nounwind readnone
|
|
|
|
declare <16 x i32> @llvm.hexagon.V6.valignb(<16 x i32>, <16 x i32>, i32) #1
|
|
|
|
|
|
|
|
; Function Attrs: nounwind readnone
|
|
|
|
declare <16 x i32> @llvm.hexagon.V6.vabsdiffh(<16 x i32>, <16 x i32>) #1
|
|
|
|
|
|
|
|
attributes #0 = { nounwind "target-cpu"="hexagonv60" "target-features"="+hvxv60,+hvx-length64b" }
|
|
|
|
attributes #1 = { nounwind readnone }
|