maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-te...

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

249 lines
10 KiB
Plaintext
Raw Normal View History

AMDGPU: Skip other terminators before inserting s_cbranch_exec[n]z PHIElimination/createPHISourceCopy inserts non-branch terminators after the control flow pseudo if a successor phi reads a register defined by the control flow pseudo. If this happens, we need to split the expansion of the control flow pseudo to ensure all the branches are after all of the other mask management instructions. GlobalISel hit this in testscases that happened to be tail duplicated. The original testcase still does not work, since the same problem appears to be present in a later pass.
2020-07-23 09:24:21 +08:00
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
# RUN: llc -march=amdgcn -mcpu=fiji -verify-machineinstrs -run-pass=si-lower-control-flow -o - %s | FileCheck %s
# Test si-lower-control-flow insertion points when other terminator
# instructions are present besides the control flow pseudo and a
# branch.
# There's another terminator instruction between SI_IF and
# S_BRANCH. The S_CBRANCH_EXECZ should be inserted immediately before
# S_BRANCH.
---
name: other_terminator_sbranch_after_si_if
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: other_terminator_sbranch_after_si_if
; CHECK: bb.0:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
; CHECK: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed $vgpr0
; CHECK: [[COPY1:%[0-9]+]]:sreg_64_xexec = COPY $sgpr4_sgpr5
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 0, [[COPY]], implicit $exec
; CHECK: [[COPY2:%[0-9]+]]:sreg_64 = COPY $exec, implicit-def $exec
; CHECK: [[S_AND_B64_:%[0-9]+]]:sreg_64 = S_AND_B64 [[COPY2]], [[V_CMP_EQ_U32_e64_]], implicit-def dead $scc
; CHECK: [[S_XOR_B64_:%[0-9]+]]:sreg_64_xexec = S_XOR_B64 [[S_AND_B64_]], [[COPY2]], implicit-def dead $scc
; CHECK: $exec = S_MOV_B64_term killed [[S_AND_B64_]]
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term killed [[COPY1]], implicit $exec
; CHECK: S_CBRANCH_EXECZ %bb.1, implicit $exec
; CHECK: S_BRANCH %bb.2
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: S_BRANCH %bb.2
; CHECK: bb.2:
; CHECK: S_ENDPGM 0, implicit [[S_MOV_B64_term]]
bb.0:
successors: %bb.2, %bb.1
liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
%0:vgpr_32 = COPY killed $vgpr0
%1:sreg_64_xexec = COPY $sgpr4_sgpr5
%2:sreg_64_xexec = V_CMP_EQ_U32_e64 0, %0, implicit $exec
%3:sreg_64_xexec = SI_IF %2, %bb.1, implicit-def $exec, implicit-def dead $scc, implicit $exec
%4:sreg_64_xexec = S_MOV_B64_term killed %1, implicit $exec
S_BRANCH %bb.2
bb.1:
S_BRANCH %bb.2
bb.2:
S_ENDPGM 0, implicit %4
...
# S_CBRANCH_EXECZ should be inserted after the other terminator
---
name: other_terminator_fallthrough_after_si_if
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: other_terminator_fallthrough_after_si_if
; CHECK: bb.0:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
; CHECK: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed $vgpr0
; CHECK: [[COPY1:%[0-9]+]]:sreg_64_xexec = COPY $sgpr4_sgpr5
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 0, [[COPY]], implicit $exec
; CHECK: [[COPY2:%[0-9]+]]:sreg_64 = COPY $exec, implicit-def $exec
; CHECK: [[S_AND_B64_:%[0-9]+]]:sreg_64 = S_AND_B64 [[COPY2]], [[V_CMP_EQ_U32_e64_]], implicit-def dead $scc
; CHECK: [[S_XOR_B64_:%[0-9]+]]:sreg_64_xexec = S_XOR_B64 [[S_AND_B64_]], [[COPY2]], implicit-def dead $scc
; CHECK: $exec = S_MOV_B64_term killed [[S_AND_B64_]]
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term killed [[COPY1]], implicit $exec
; CHECK: S_CBRANCH_EXECZ %bb.2, implicit $exec
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: S_BRANCH %bb.2
; CHECK: bb.2:
; CHECK: S_ENDPGM 0, implicit [[S_MOV_B64_term]]
bb.0:
successors: %bb.2, %bb.1
liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
%0:vgpr_32 = COPY killed $vgpr0
%1:sreg_64_xexec = COPY $sgpr4_sgpr5
%2:sreg_64_xexec = V_CMP_EQ_U32_e64 0, %0, implicit $exec
%3:sreg_64_xexec = SI_IF %2, %bb.2, implicit-def $exec, implicit-def dead $scc, implicit $exec
%4:sreg_64_xexec = S_MOV_B64_term killed %1, implicit $exec
bb.1:
S_BRANCH %bb.2
bb.2:
S_ENDPGM 0, implicit %4
...
---
name: other_terminator_sbranch_after_si_else
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: other_terminator_sbranch_after_si_else
; CHECK: bb.0:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
[AMDGPU] Remove fix up operand from SI_ELSE Remove immediate operand from SI_ELSE which indicates if EXEC has been modified. Instead always emit code that handles EXEC and remove unnecessary instructions during pre-RA optimisation. This facilitates passes (i.e. SIWholeQuadMode) adding exec mask manipulation post control flow lowering, and pre control flow lower passes do not need to be aware of SI_ELSE handling. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D89644
2020-10-20 17:11:51 +08:00
; CHECK: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 %2, implicit-def $exec, implicit-def $scc, implicit $exec
AMDGPU: Remove code to handle tied si_else operands This has not used tied operands for a long time.
2020-09-02 23:36:22 +08:00
; CHECK: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed $vgpr0
; CHECK: [[COPY1:%[0-9]+]]:sreg_64_xexec = COPY $sgpr4_sgpr5
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 0, [[COPY]], implicit $exec
[AMDGPU] Remove fix up operand from SI_ELSE Remove immediate operand from SI_ELSE which indicates if EXEC has been modified. Instead always emit code that handles EXEC and remove unnecessary instructions during pre-RA optimisation. This facilitates passes (i.e. SIWholeQuadMode) adding exec mask manipulation post control flow lowering, and pre control flow lower passes do not need to be aware of SI_ELSE handling. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D89644
2020-10-20 17:11:51 +08:00
; CHECK: [[S_AND_B64_:%[0-9]+]]:sreg_64_xexec = S_AND_B64 $exec, [[S_OR_SAVEEXEC_B64_]], implicit-def $scc
; CHECK: $exec = S_XOR_B64_term $exec, [[S_AND_B64_]], implicit-def $scc
AMDGPU: Remove code to handle tied si_else operands This has not used tied operands for a long time.
2020-09-02 23:36:22 +08:00
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term killed [[COPY1]], implicit $exec
AMDGPU: Skip other terminators before inserting s_cbranch_exec[n]z PHIElimination/createPHISourceCopy inserts non-branch terminators after the control flow pseudo if a successor phi reads a register defined by the control flow pseudo. If this happens, we need to split the expansion of the control flow pseudo to ensure all the branches are after all of the other mask management instructions. GlobalISel hit this in testscases that happened to be tail duplicated. The original testcase still does not work, since the same problem appears to be present in a later pass.
2020-07-23 09:24:21 +08:00
; CHECK: S_CBRANCH_EXECZ %bb.1, implicit $exec
; CHECK: S_BRANCH %bb.2
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: S_BRANCH %bb.2
; CHECK: bb.2:
; CHECK: S_ENDPGM 0, implicit [[S_MOV_B64_term]]
bb.0:
successors: %bb.2, %bb.1
liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
%0:vgpr_32 = COPY killed $vgpr0
%1:sreg_64_xexec = COPY $sgpr4_sgpr5
%2:sreg_64_xexec = V_CMP_EQ_U32_e64 0, %0, implicit $exec
[AMDGPU] Remove fix up operand from SI_ELSE Remove immediate operand from SI_ELSE which indicates if EXEC has been modified. Instead always emit code that handles EXEC and remove unnecessary instructions during pre-RA optimisation. This facilitates passes (i.e. SIWholeQuadMode) adding exec mask manipulation post control flow lowering, and pre control flow lower passes do not need to be aware of SI_ELSE handling. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D89644
2020-10-20 17:11:51 +08:00
%3:sreg_64_xexec = SI_ELSE %2, %bb.1, implicit-def $exec, implicit-def dead $scc, implicit $exec
AMDGPU: Skip other terminators before inserting s_cbranch_exec[n]z PHIElimination/createPHISourceCopy inserts non-branch terminators after the control flow pseudo if a successor phi reads a register defined by the control flow pseudo. If this happens, we need to split the expansion of the control flow pseudo to ensure all the branches are after all of the other mask management instructions. GlobalISel hit this in testscases that happened to be tail duplicated. The original testcase still does not work, since the same problem appears to be present in a later pass.
2020-07-23 09:24:21 +08:00
%4:sreg_64_xexec = S_MOV_B64_term killed %1, implicit $exec
S_BRANCH %bb.2
bb.1:
S_BRANCH %bb.2
bb.2:
S_ENDPGM 0, implicit %4
...
---
name: other_terminator_sbranch_after_si_loop
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: other_terminator_sbranch_after_si_loop
; CHECK: bb.0:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
; CHECK: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed $vgpr0
; CHECK: [[COPY1:%[0-9]+]]:sreg_64_xexec = COPY $sgpr4_sgpr5
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 0, [[COPY]], implicit $exec
; CHECK: $exec = S_ANDN2_B64_term $exec, [[V_CMP_EQ_U32_e64_]], implicit-def $scc
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term killed [[COPY1]], implicit $exec
; CHECK: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; CHECK: S_BRANCH %bb.2
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: S_BRANCH %bb.2
; CHECK: bb.2:
; CHECK: S_ENDPGM 0, implicit [[S_MOV_B64_term]]
bb.0:
successors: %bb.2, %bb.1
liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
%0:vgpr_32 = COPY killed $vgpr0
%1:sreg_64_xexec = COPY $sgpr4_sgpr5
%2:sreg_64_xexec = V_CMP_EQ_U32_e64 0, %0, implicit $exec
SI_LOOP %2, %bb.1, implicit-def $exec, implicit-def dead $scc, implicit $exec
%4:sreg_64_xexec = S_MOV_B64_term killed %1, implicit $exec
S_BRANCH %bb.2
bb.1:
S_BRANCH %bb.2
bb.2:
S_ENDPGM 0, implicit %4
...
# The save exec result register of SI_IF is used by other terminators
# inserted to behave as a lowered phi. The output register of SI_IF
# was ignored, and the def was removed, so the S_MOV_B64_term uses
# would fail the verifier.
---
name: si_if_use
alignment: 1
legalized: true
regBankSelected: true
selected: true
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: si_if_use
; CHECK: bb.0:
; CHECK: successors: %bb.1(0x40000000), %bb.2(0x40000000)
; CHECK: liveins: $vgpr0, $vgpr1, $sgpr30_sgpr31
; CHECK: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed $vgpr0
; CHECK: [[COPY1:%[0-9]+]]:vgpr_32 = COPY killed $vgpr1
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 killed [[COPY]], killed [[COPY1]], implicit $exec
; CHECK: [[COPY2:%[0-9]+]]:sreg_64 = COPY $exec, implicit-def $exec
; CHECK: [[S_AND_B64_:%[0-9]+]]:sreg_64 = S_AND_B64 [[COPY2]], [[V_CMP_EQ_U32_e64_]], implicit-def dead $scc
; CHECK: [[S_XOR_B64_:%[0-9]+]]:sreg_64_xexec = S_XOR_B64 [[S_AND_B64_]], [[COPY2]], implicit-def dead $scc
; CHECK: $exec = S_MOV_B64_term killed [[S_AND_B64_]]
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term [[S_XOR_B64_]], implicit $exec
; CHECK: [[S_MOV_B64_term1:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term [[S_XOR_B64_]], implicit $exec
; CHECK: S_CBRANCH_EXECZ %bb.1, implicit $exec
; CHECK: S_BRANCH %bb.2
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: [[COPY3:%[0-9]+]]:sreg_64_xexec = COPY [[S_MOV_B64_term1]]
; CHECK: dead %7:vgpr_32 = GLOBAL_LOAD_DWORD undef %8:vreg_64, 0, 0, 0, 0, implicit $exec :: (volatile load 4, addrspace 1)
; CHECK: [[COPY4:%[0-9]+]]:sreg_64_xexec = COPY [[COPY3]]
; CHECK: bb.2:
AMDGPU: Don't sometimes allow instructions before lowered si_end_cf Since 6524a7a2b9ca072bd7f7b4355d1230e70c679d2f, this would sometimes not emit the or to exec at the beginning of the block, where it really has to be. If there is an instruction that defines one of the source operands, split the block and turn the si_end_cf into a terminator. This avoids regressions when regalloc fast is switched to inserting reloads at the beginning of the block, instead of spills at the end of the block. In a future change, this should always split the block.
2020-09-11 03:49:09 +08:00
; CHECK: successors: %bb.3(0x80000000)
AMDGPU: Skip other terminators before inserting s_cbranch_exec[n]z PHIElimination/createPHISourceCopy inserts non-branch terminators after the control flow pseudo if a successor phi reads a register defined by the control flow pseudo. If this happens, we need to split the expansion of the control flow pseudo to ensure all the branches are after all of the other mask management instructions. GlobalISel hit this in testscases that happened to be tail duplicated. The original testcase still does not work, since the same problem appears to be present in a later pass.
2020-07-23 09:24:21 +08:00
; CHECK: [[COPY5:%[0-9]+]]:sreg_64_xexec = COPY [[COPY4]]
AMDGPU: Don't sometimes allow instructions before lowered si_end_cf Since 6524a7a2b9ca072bd7f7b4355d1230e70c679d2f, this would sometimes not emit the or to exec at the beginning of the block, where it really has to be. If there is an instruction that defines one of the source operands, split the block and turn the si_end_cf into a terminator. This avoids regressions when regalloc fast is switched to inserting reloads at the beginning of the block, instead of spills at the end of the block. In a future change, this should always split the block.
2020-09-11 03:49:09 +08:00
; CHECK: $exec = S_OR_B64_term $exec, killed [[COPY5]], implicit-def $scc
; CHECK: bb.3:
; CHECK: successors: %bb.1(0x40000000), %bb.2(0x40000000)
AMDGPU: Skip other terminators before inserting s_cbranch_exec[n]z PHIElimination/createPHISourceCopy inserts non-branch terminators after the control flow pseudo if a successor phi reads a register defined by the control flow pseudo. If this happens, we need to split the expansion of the control flow pseudo to ensure all the branches are after all of the other mask management instructions. GlobalISel hit this in testscases that happened to be tail duplicated. The original testcase still does not work, since the same problem appears to be present in a later pass.
2020-07-23 09:24:21 +08:00
; CHECK: S_SLEEP 1
; CHECK: [[COPY6:%[0-9]+]]:sreg_64 = COPY $exec, implicit-def $exec
; CHECK: [[S_AND_B64_1:%[0-9]+]]:sreg_64 = S_AND_B64 [[COPY6]], [[V_CMP_EQ_U32_e64_]], implicit-def dead $scc
; CHECK: [[S_XOR_B64_1:%[0-9]+]]:sreg_64_xexec = S_XOR_B64 [[S_AND_B64_1]], [[COPY6]], implicit-def dead $scc
; CHECK: $exec = S_MOV_B64_term killed [[S_AND_B64_1]]
; CHECK: [[S_MOV_B64_term1:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term [[S_XOR_B64_1]], implicit $exec
; CHECK: [[S_MOV_B64_term2:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term [[S_XOR_B64_1]], implicit $exec
; CHECK: S_CBRANCH_EXECZ %bb.1, implicit $exec
; CHECK: S_BRANCH %bb.2
bb.0:
liveins: $vgpr0, $vgpr1, $sgpr30_sgpr31
%0:vgpr_32 = COPY killed $vgpr0
%1:vgpr_32 = COPY killed $vgpr1
%3:sreg_64_xexec = V_CMP_EQ_U32_e64 killed %0, killed %1, implicit $exec
%10:sreg_64_xexec = SI_IF %3, %bb.1, implicit-def $exec, implicit-def dead $scc, implicit $exec
%14:sreg_64_xexec = S_MOV_B64_term %10, implicit $exec
%13:sreg_64_xexec = S_MOV_B64_term %10, implicit $exec
S_BRANCH %bb.2
bb.1:
%11:sreg_64_xexec = COPY %13
dead %6:vgpr_32 = GLOBAL_LOAD_DWORD undef %8:vreg_64, 0, 0, 0, 0, implicit $exec :: (volatile load 4, addrspace 1)
%14:sreg_64_xexec = COPY %11
bb.2:
%12:sreg_64_xexec = COPY %14
SI_END_CF killed %12, implicit-def $exec, implicit-def dead $scc, implicit $exec
S_SLEEP 1
%9:sreg_64_xexec = SI_IF %3, %bb.1, implicit-def $exec, implicit-def dead $scc, implicit $exec
%14:sreg_64_xexec = S_MOV_B64_term %9, implicit $exec
%13:sreg_64_xexec = S_MOV_B64_term %9, implicit $exec
S_BRANCH %bb.2
...