maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/AMDGPU/atomic_optimizations_pixels...

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

419 lines
18 KiB
LLVM
Raw Normal View History

[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[AMDGPU][SimplifyCFG] Teach AMDGPUUnifyDivergentExitNodes to preserve {,Post}DomTree This is a (last big?) part of the patch series to make SimplifyCFG preserve DomTree. Currently, it still does not actually preserve it, even thought it is pretty much fully updated to preserve it. Once the default is flipped, a valid DomTree must be passed into simplifyCFG, which means that whatever pass calls simplifyCFG, should also be smart about DomTree's. As far as i can see from `check-llvm` with default flipped, this is the last LLVM test batch (other than bugpoint tests) that needed fixes to not break with default flipped. The changes here are boringly identical to the ones i did over 42+ times/commits recently already, so while AMDGPU is outside of my normal ecosystem, i'm going to go for post-commit review here, like in all the other 42+ changes. Note that while the pass is taught to preserve {,Post}DomTree, it still doesn't do that by default, because simplifycfg still doesn't do that by default, and flipping default in this pass will implicitly flip the default for simplifycfg. That will happen, but not right now.
2021-01-02 04:18:35 +08:00
; RUN: llc -mtriple=amdgcn-- -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX7 %s
[AMDGPU] Rationalize some check prefixes and use more common prefixes. NFC.
2021-03-20 00:43:52 +08:00
; RUN: llc -mtriple=amdgcn-- -mcpu=tonga -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX8 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx900 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX9 %s
[AMDGPU][SimplifyCFG] Teach AMDGPUUnifyDivergentExitNodes to preserve {,Post}DomTree This is a (last big?) part of the patch series to make SimplifyCFG preserve DomTree. Currently, it still does not actually preserve it, even thought it is pretty much fully updated to preserve it. Once the default is flipped, a valid DomTree must be passed into simplifyCFG, which means that whatever pass calls simplifyCFG, should also be smart about DomTree's. As far as i can see from `check-llvm` with default flipped, this is the last LLVM test batch (other than bugpoint tests) that needed fixes to not break with default flipped. The changes here are boringly identical to the ones i did over 42+ times/commits recently already, so while AMDGPU is outside of my normal ecosystem, i'm going to go for post-commit review here, like in all the other 42+ changes. Note that while the pass is taught to preserve {,Post}DomTree, it still doesn't do that by default, because simplifycfg still doesn't do that by default, and flipping default in this pass will implicitly flip the default for simplifycfg. That will happen, but not right now.
2021-01-02 04:18:35 +08:00
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1064 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1032 %s
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
declare i1 @llvm.amdgcn.wqm.vote(i1)
AMDGPU: Update more tests to use modern buffer intrinsics
2020-01-17 03:05:46 +08:00
declare i32 @llvm.amdgcn.raw.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32 immarg)
declare void @llvm.amdgcn.raw.buffer.store.f32(float, <4 x i32>, i32, i32, i32 immarg)
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; Show what the atomic optimization pass will do for raw buffers.
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
define amdgpu_ps void @add_i32_constant(<4 x i32> inreg %out, <4 x i32> inreg %inout) {
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX7-LABEL: add_i32_constant:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_mov_b64 s[10:11], exec
; GFX7-NEXT: ; implicit-def: $vgpr0
; GFX7-NEXT: s_and_saveexec_b64 s[8:9], s[10:11]
; GFX7-NEXT: s_cbranch_execz BB0_4
; GFX7-NEXT: ; %bb.1:
[AMDGPU] New llvm.amdgcn.ballot intrinsic Add a new llvm.amdgcn.ballot intrinsic modeled on the ballot function in GLSL and other shader languages. It returns a bitfield containing the result of its boolean argument in all active lanes, and zero in all inactive lanes. This is intended to replace the existing llvm.amdgcn.icmp and llvm.amdgcn.fcmp intrinsics after a suitable transition period. Use the new intrinsic in the atomic optimizer pass. Differential Revision: https://reviews.llvm.org/D65088
2020-03-05 21:23:59 +08:00
; GFX7-NEXT: s_mov_b64 s[12:13], exec
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX7-NEXT: v_mbcnt_lo_u32_b32_e64 v0, s12, 0
; GFX7-NEXT: v_mbcnt_hi_u32_b32_e32 v0, s13, v0
; GFX7-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
; GFX7-NEXT: ; implicit-def: $vgpr1
; GFX7-NEXT: s_and_saveexec_b64 s[10:11], vcc
; GFX7-NEXT: s_cbranch_execz BB0_3
; GFX7-NEXT: ; %bb.2:
; GFX7-NEXT: s_bcnt1_i32_b64 s12, s[12:13]
[AMDGPU][SelectionDAG] Don't combine uniform multiplies to MUL_[UI]24 Prefer to keep uniform (non-divergent) multiplies on the scalar ALU when possible. This significantly improves some game cases by eliminating v_readfirstlane instructions when the result feeds into a scalar operation, like the address calculation for a scalar load or store. Since isDivergent is only an approximation of whether a value is in SGPRs, it can potentially regress some situations where a uniform value ends up in a VGPR. These should be rare in real code, although the test changes do contain a number of examples. Most of the test changes are just using s_mul instead of v_mul/mad which is generally better for both register pressure and latency (at least on GFX10 where sgpr pressure doesn't affect occupancy and vector ALU instructions have significantly longer latency than scalar ALU). Some R600 tests now use MULLO_INT instead of MUL_UINT24. GlobalISel appears to handle more scenarios in the desirable way, although it can also be thrown off and fails to select the 24-bit multiplies in some cases. Alternative solution considered and rejected was to allow selecting MUL_[UI]24 to S_MUL_I32. I've rejected this because the definition of those SD operations works is don't-care on the most significant 8 bits, and this fact is used in some combines via SimplifyDemandedBits. Based on a patch by Nicolai Hähnle. Differential Revision: https://reviews.llvm.org/D97063
2020-08-03 21:03:18 +08:00
; GFX7-NEXT: s_mul_i32 s12, s12, 5
; GFX7-NEXT: v_mov_b32_e32 v1, s12
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX7-NEXT: buffer_atomic_add v1, off, s[4:7], 0 glc
; GFX7-NEXT: BB0_3:
; GFX7-NEXT: s_or_b64 exec, exec, s[10:11]
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: v_readfirstlane_b32 s4, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v0, 5, s4
; GFX7-NEXT: BB0_4: ; %Flow
; GFX7-NEXT: s_or_b64 exec, exec, s[8:9]
; GFX7-NEXT: s_wqm_b64 s[4:5], -1
; GFX7-NEXT: s_andn2_b64 vcc, exec, s[4:5]
; GFX7-NEXT: s_cbranch_vccnz BB0_6
; GFX7-NEXT: ; %bb.5: ; %if
; GFX7-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX7-NEXT: BB0_6: ; %UnifiedReturnBlock
; GFX7-NEXT: s_endpgm
;
[AMDGPU] Rationalize some check prefixes and use more common prefixes. NFC.
2021-03-20 00:43:52 +08:00
; GFX89-LABEL: add_i32_constant:
; GFX89: ; %bb.0: ; %entry
; GFX89-NEXT: s_mov_b64 s[10:11], exec
; GFX89-NEXT: ; implicit-def: $vgpr0
; GFX89-NEXT: s_and_saveexec_b64 s[8:9], s[10:11]
; GFX89-NEXT: s_cbranch_execz BB0_4
; GFX89-NEXT: ; %bb.1:
; GFX89-NEXT: s_mov_b64 s[12:13], exec
; GFX89-NEXT: v_mbcnt_lo_u32_b32 v0, s12, 0
; GFX89-NEXT: v_mbcnt_hi_u32_b32 v0, s13, v0
; GFX89-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
; GFX89-NEXT: ; implicit-def: $vgpr1
; GFX89-NEXT: s_and_saveexec_b64 s[10:11], vcc
; GFX89-NEXT: s_cbranch_execz BB0_3
; GFX89-NEXT: ; %bb.2:
; GFX89-NEXT: s_bcnt1_i32_b64 s12, s[12:13]
; GFX89-NEXT: s_mul_i32 s12, s12, 5
; GFX89-NEXT: v_mov_b32_e32 v1, s12
; GFX89-NEXT: buffer_atomic_add v1, off, s[4:7], 0 glc
; GFX89-NEXT: BB0_3:
; GFX89-NEXT: s_or_b64 exec, exec, s[10:11]
; GFX89-NEXT: s_waitcnt vmcnt(0)
; GFX89-NEXT: v_readfirstlane_b32 s4, v1
; GFX89-NEXT: v_mad_u32_u24 v0, v0, 5, s4
; GFX89-NEXT: BB0_4: ; %Flow
; GFX89-NEXT: s_or_b64 exec, exec, s[8:9]
; GFX89-NEXT: s_wqm_b64 s[4:5], -1
; GFX89-NEXT: s_andn2_b64 vcc, exec, s[4:5]
; GFX89-NEXT: s_cbranch_vccnz BB0_6
; GFX89-NEXT: ; %bb.5: ; %if
; GFX89-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX89-NEXT: BB0_6: ; %UnifiedReturnBlock
; GFX89-NEXT: s_endpgm
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
;
; GFX1064-LABEL: add_i32_constant:
; GFX1064: ; %bb.0: ; %entry
; GFX1064-NEXT: s_mov_b64 s[10:11], exec
; GFX1064-NEXT: ; implicit-def: $vgpr0
; GFX1064-NEXT: s_and_saveexec_b64 s[8:9], s[10:11]
; GFX1064-NEXT: s_cbranch_execz BB0_4
; GFX1064-NEXT: ; %bb.1:
[AMDGPU] New llvm.amdgcn.ballot intrinsic Add a new llvm.amdgcn.ballot intrinsic modeled on the ballot function in GLSL and other shader languages. It returns a bitfield containing the result of its boolean argument in all active lanes, and zero in all inactive lanes. This is intended to replace the existing llvm.amdgcn.icmp and llvm.amdgcn.fcmp intrinsics after a suitable transition period. Use the new intrinsic in the atomic optimizer pass. Differential Revision: https://reviews.llvm.org/D65088
2020-03-05 21:23:59 +08:00
; GFX1064-NEXT: s_mov_b64 s[12:13], exec
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: ; implicit-def: $vgpr1
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX1064-NEXT: v_mbcnt_lo_u32_b32 v0, s12, 0
; GFX1064-NEXT: v_mbcnt_hi_u32_b32 v0, s13, v0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
[AMDGPU] Experiments show that the GCNRegBankReassign pass significantly impacts the compilation time and there is no case for which we see any improvement in performance. This patch removes this pass and its associated test cases from the tree. Differential Revision: https://reviews.llvm.org/D101313 Change-Id: I0599169a7609c19a887f8d847a71e664030cc141
2021-04-27 03:48:12 +08:00
; GFX1064-NEXT: s_and_saveexec_b64 s[10:11], vcc
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: s_cbranch_execz BB0_3
; GFX1064-NEXT: ; %bb.2:
; GFX1064-NEXT: s_bcnt1_i32_b64 s12, s[12:13]
[AMDGPU][SelectionDAG] Don't combine uniform multiplies to MUL_[UI]24 Prefer to keep uniform (non-divergent) multiplies on the scalar ALU when possible. This significantly improves some game cases by eliminating v_readfirstlane instructions when the result feeds into a scalar operation, like the address calculation for a scalar load or store. Since isDivergent is only an approximation of whether a value is in SGPRs, it can potentially regress some situations where a uniform value ends up in a VGPR. These should be rare in real code, although the test changes do contain a number of examples. Most of the test changes are just using s_mul instead of v_mul/mad which is generally better for both register pressure and latency (at least on GFX10 where sgpr pressure doesn't affect occupancy and vector ALU instructions have significantly longer latency than scalar ALU). Some R600 tests now use MULLO_INT instead of MUL_UINT24. GlobalISel appears to handle more scenarios in the desirable way, although it can also be thrown off and fails to select the 24-bit multiplies in some cases. Alternative solution considered and rejected was to allow selecting MUL_[UI]24 to S_MUL_I32. I've rejected this because the definition of those SD operations works is don't-care on the most significant 8 bits, and this fact is used in some combines via SimplifyDemandedBits. Based on a patch by Nicolai Hähnle. Differential Revision: https://reviews.llvm.org/D97063
2020-08-03 21:03:18 +08:00
; GFX1064-NEXT: s_mul_i32 s12, s12, 5
; GFX1064-NEXT: v_mov_b32_e32 v1, s12
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: buffer_atomic_add v1, off, s[4:7], 0 glc
; GFX1064-NEXT: BB0_3:
[AMDGPU] Update VMEM scalar write hazard mitigation sequence Using s_waitcnt_depctr 0xffe3 is potentially faster than v_nop. Reviewed By: rampitec, foad Differential Revision: https://reviews.llvm.org/D83872
2020-07-16 10:07:26 +08:00
; GFX1064-NEXT: s_waitcnt_depctr 0xffe3
[AMDGPU] Experiments show that the GCNRegBankReassign pass significantly impacts the compilation time and there is no case for which we see any improvement in performance. This patch removes this pass and its associated test cases from the tree. Differential Revision: https://reviews.llvm.org/D101313 Change-Id: I0599169a7609c19a887f8d847a71e664030cc141
2021-04-27 03:48:12 +08:00
; GFX1064-NEXT: s_or_b64 exec, exec, s[10:11]
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: s_waitcnt vmcnt(0)
; GFX1064-NEXT: v_readfirstlane_b32 s4, v1
; GFX1064-NEXT: v_mad_u32_u24 v0, v0, 5, s4
; GFX1064-NEXT: BB0_4: ; %Flow
; GFX1064-NEXT: s_or_b64 exec, exec, s[8:9]
; GFX1064-NEXT: s_wqm_b64 s[4:5], -1
; GFX1064-NEXT: s_andn2_b64 vcc, exec, s[4:5]
; GFX1064-NEXT: s_cbranch_vccnz BB0_6
; GFX1064-NEXT: ; %bb.5: ; %if
; GFX1064-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX1064-NEXT: BB0_6: ; %UnifiedReturnBlock
; GFX1064-NEXT: s_endpgm
;
; GFX1032-LABEL: add_i32_constant:
; GFX1032: ; %bb.0: ; %entry
; GFX1032-NEXT: s_mov_b32 s9, exec_lo
; GFX1032-NEXT: ; implicit-def: $vgpr0
; GFX1032-NEXT: s_and_saveexec_b32 s8, s9
; GFX1032-NEXT: s_cbranch_execz BB0_4
; GFX1032-NEXT: ; %bb.1:
[AMDGPU] New llvm.amdgcn.ballot intrinsic Add a new llvm.amdgcn.ballot intrinsic modeled on the ballot function in GLSL and other shader languages. It returns a bitfield containing the result of its boolean argument in all active lanes, and zero in all inactive lanes. This is intended to replace the existing llvm.amdgcn.icmp and llvm.amdgcn.fcmp intrinsics after a suitable transition period. Use the new intrinsic in the atomic optimizer pass. Differential Revision: https://reviews.llvm.org/D65088
2020-03-05 21:23:59 +08:00
; GFX1032-NEXT: s_mov_b32 s10, exec_lo
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: ; implicit-def: $vgpr1
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX1032-NEXT: v_mbcnt_lo_u32_b32 v0, s10, 0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: v_cmp_eq_u32_e32 vcc_lo, 0, v0
; GFX1032-NEXT: s_and_saveexec_b32 s9, vcc_lo
; GFX1032-NEXT: s_cbranch_execz BB0_3
; GFX1032-NEXT: ; %bb.2:
; GFX1032-NEXT: s_bcnt1_i32_b32 s10, s10
[AMDGPU][SelectionDAG] Don't combine uniform multiplies to MUL_[UI]24 Prefer to keep uniform (non-divergent) multiplies on the scalar ALU when possible. This significantly improves some game cases by eliminating v_readfirstlane instructions when the result feeds into a scalar operation, like the address calculation for a scalar load or store. Since isDivergent is only an approximation of whether a value is in SGPRs, it can potentially regress some situations where a uniform value ends up in a VGPR. These should be rare in real code, although the test changes do contain a number of examples. Most of the test changes are just using s_mul instead of v_mul/mad which is generally better for both register pressure and latency (at least on GFX10 where sgpr pressure doesn't affect occupancy and vector ALU instructions have significantly longer latency than scalar ALU). Some R600 tests now use MULLO_INT instead of MUL_UINT24. GlobalISel appears to handle more scenarios in the desirable way, although it can also be thrown off and fails to select the 24-bit multiplies in some cases. Alternative solution considered and rejected was to allow selecting MUL_[UI]24 to S_MUL_I32. I've rejected this because the definition of those SD operations works is don't-care on the most significant 8 bits, and this fact is used in some combines via SimplifyDemandedBits. Based on a patch by Nicolai Hähnle. Differential Revision: https://reviews.llvm.org/D97063
2020-08-03 21:03:18 +08:00
; GFX1032-NEXT: s_mul_i32 s10, s10, 5
; GFX1032-NEXT: v_mov_b32_e32 v1, s10
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: buffer_atomic_add v1, off, s[4:7], 0 glc
; GFX1032-NEXT: BB0_3:
[AMDGPU] Update VMEM scalar write hazard mitigation sequence Using s_waitcnt_depctr 0xffe3 is potentially faster than v_nop. Reviewed By: rampitec, foad Differential Revision: https://reviews.llvm.org/D83872
2020-07-16 10:07:26 +08:00
; GFX1032-NEXT: s_waitcnt_depctr 0xffe3
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: s_or_b32 exec_lo, exec_lo, s9
; GFX1032-NEXT: s_waitcnt vmcnt(0)
; GFX1032-NEXT: v_readfirstlane_b32 s4, v1
; GFX1032-NEXT: v_mad_u32_u24 v0, v0, 5, s4
; GFX1032-NEXT: BB0_4: ; %Flow
; GFX1032-NEXT: s_or_b32 exec_lo, exec_lo, s8
; GFX1032-NEXT: s_wqm_b32 s4, -1
; GFX1032-NEXT: s_andn2_b32 vcc_lo, exec_lo, s4
; GFX1032-NEXT: s_cbranch_vccnz BB0_6
; GFX1032-NEXT: ; %bb.5: ; %if
; GFX1032-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX1032-NEXT: BB0_6: ; %UnifiedReturnBlock
; GFX1032-NEXT: s_endpgm
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
entry:
%cond1 = call i1 @llvm.amdgcn.wqm.vote(i1 true)
AMDGPU: Update more tests to use modern buffer intrinsics
2020-01-17 03:05:46 +08:00
%old = call i32 @llvm.amdgcn.raw.buffer.atomic.add(i32 5, <4 x i32> %inout, i32 0, i32 0, i32 0)
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
%cond2 = call i1 @llvm.amdgcn.wqm.vote(i1 true)
%cond = and i1 %cond1, %cond2
br i1 %cond, label %if, label %else
if:
%bitcast = bitcast i32 %old to float
AMDGPU: Update more tests to use modern buffer intrinsics
2020-01-17 03:05:46 +08:00
call void @llvm.amdgcn.raw.buffer.store.f32(float %bitcast, <4 x i32> %out, i32 0, i32 0, i32 0)
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
ret void
else:
ret void
}
define amdgpu_ps void @add_i32_varying(<4 x i32> inreg %out, <4 x i32> inreg %inout, i32 %val) {
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX7-LABEL: add_i32_varying:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_wqm_b64 s[8:9], -1
; GFX7-NEXT: buffer_atomic_add v0, off, s[4:7], 0 glc
; GFX7-NEXT: s_andn2_b64 vcc, exec, s[8:9]
; GFX7-NEXT: s_cbranch_vccnz BB1_2
; GFX7-NEXT: ; %bb.1: ; %if
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX7-NEXT: BB1_2: ; %else
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: add_i32_varying:
; GFX8: ; %bb.0: ; %entry
[AMDGPU] Move kill lowering to WQM pass and add live mask tracking Move implementation of kill intrinsics to WQM pass. Add live lane tracking by updating a stored exec mask when lanes are killed. Use live lane tracking to enable early termination of shader at any point in control flow. Reviewed By: piotr Differential Revision: https://reviews.llvm.org/D94746
2021-02-10 11:56:45 +08:00
; GFX8-NEXT: s_mov_b64 s[8:9], exec
; GFX8-NEXT: s_mov_b64 s[10:11], s[8:9]
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: v_mov_b32_e32 v2, v0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX8-NEXT: ; implicit-def: $vgpr0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: s_and_saveexec_b64 s[8:9], s[10:11]
; GFX8-NEXT: s_cbranch_execz BB1_4
; GFX8-NEXT: ; %bb.1:
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX8-NEXT: s_or_saveexec_b64 s[10:11], -1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX8-NEXT: s_mov_b64 exec, s[10:11]
; GFX8-NEXT: v_mbcnt_lo_u32_b32 v0, exec_lo, 0
; GFX8-NEXT: v_mbcnt_hi_u32_b32 v0, exec_hi, v0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: s_not_b64 exec, exec
; GFX8-NEXT: v_mov_b32_e32 v2, 0
; GFX8-NEXT: s_not_b64 exec, exec
; GFX8-NEXT: s_or_saveexec_b64 s[10:11], -1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX8-NEXT: v_add_u32_dpp v2, vcc, v2, v2 row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: s_nop 1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX8-NEXT: v_add_u32_dpp v2, vcc, v2, v2 row_shr:2 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: s_nop 1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX8-NEXT: v_add_u32_dpp v2, vcc, v2, v2 row_shr:4 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: s_nop 1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX8-NEXT: v_add_u32_dpp v2, vcc, v2, v2 row_shr:8 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: s_nop 1
; GFX8-NEXT: v_add_u32_dpp v2, vcc, v2, v2 row_bcast:15 row_mask:0xa bank_mask:0xf
; GFX8-NEXT: s_nop 1
; GFX8-NEXT: v_add_u32_dpp v2, vcc, v2, v2 row_bcast:31 row_mask:0xc bank_mask:0xf
; GFX8-NEXT: v_readlane_b32 s12, v2, 63
; GFX8-NEXT: s_nop 0
; GFX8-NEXT: v_mov_b32_dpp v1, v2 wave_shr:1 row_mask:0xf bank_mask:0xf
; GFX8-NEXT: s_mov_b64 exec, s[10:11]
; GFX8-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
; GFX8-NEXT: ; implicit-def: $vgpr0
; GFX8-NEXT: s_and_saveexec_b64 s[10:11], vcc
; GFX8-NEXT: s_cbranch_execz BB1_3
; GFX8-NEXT: ; %bb.2:
; GFX8-NEXT: v_mov_b32_e32 v0, s12
; GFX8-NEXT: buffer_atomic_add v0, off, s[4:7], 0 glc
; GFX8-NEXT: BB1_3:
; GFX8-NEXT: s_or_b64 exec, exec, s[10:11]
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_readfirstlane_b32 s4, v0
; GFX8-NEXT: v_mov_b32_e32 v0, v1
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: BB1_4: ; %Flow
; GFX8-NEXT: s_or_b64 exec, exec, s[8:9]
; GFX8-NEXT: s_wqm_b64 s[4:5], -1
; GFX8-NEXT: s_andn2_b64 vcc, exec, s[4:5]
; GFX8-NEXT: s_cbranch_vccnz BB1_6
; GFX8-NEXT: ; %bb.5: ; %if
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX8-NEXT: buffer_store_dword v0, off, s[0:3], 0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX8-NEXT: BB1_6: ; %UnifiedReturnBlock
; GFX8-NEXT: s_endpgm
;
; GFX9-LABEL: add_i32_varying:
; GFX9: ; %bb.0: ; %entry
[AMDGPU] Move kill lowering to WQM pass and add live mask tracking Move implementation of kill intrinsics to WQM pass. Add live lane tracking by updating a stored exec mask when lanes are killed. Use live lane tracking to enable early termination of shader at any point in control flow. Reviewed By: piotr Differential Revision: https://reviews.llvm.org/D94746
2021-02-10 11:56:45 +08:00
; GFX9-NEXT: s_mov_b64 s[8:9], exec
; GFX9-NEXT: s_mov_b64 s[10:11], s[8:9]
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: v_mov_b32_e32 v2, v0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX9-NEXT: ; implicit-def: $vgpr0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: s_and_saveexec_b64 s[8:9], s[10:11]
; GFX9-NEXT: s_cbranch_execz BB1_4
; GFX9-NEXT: ; %bb.1:
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX9-NEXT: s_or_saveexec_b64 s[10:11], -1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX9-NEXT: s_mov_b64 exec, s[10:11]
; GFX9-NEXT: v_mbcnt_lo_u32_b32 v0, exec_lo, 0
; GFX9-NEXT: v_mbcnt_hi_u32_b32 v0, exec_hi, v0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: s_not_b64 exec, exec
; GFX9-NEXT: v_mov_b32_e32 v2, 0
; GFX9-NEXT: s_not_b64 exec, exec
; GFX9-NEXT: s_or_saveexec_b64 s[10:11], -1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX9-NEXT: v_add_u32_dpp v2, v2, v2 row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: s_nop 1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX9-NEXT: v_add_u32_dpp v2, v2, v2 row_shr:2 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: s_nop 1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX9-NEXT: v_add_u32_dpp v2, v2, v2 row_shr:4 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: s_nop 1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX9-NEXT: v_add_u32_dpp v2, v2, v2 row_shr:8 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: s_nop 1
; GFX9-NEXT: v_add_u32_dpp v2, v2, v2 row_bcast:15 row_mask:0xa bank_mask:0xf
; GFX9-NEXT: s_nop 1
; GFX9-NEXT: v_add_u32_dpp v2, v2, v2 row_bcast:31 row_mask:0xc bank_mask:0xf
; GFX9-NEXT: v_readlane_b32 s12, v2, 63
; GFX9-NEXT: s_nop 0
; GFX9-NEXT: v_mov_b32_dpp v1, v2 wave_shr:1 row_mask:0xf bank_mask:0xf
; GFX9-NEXT: s_mov_b64 exec, s[10:11]
; GFX9-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
; GFX9-NEXT: ; implicit-def: $vgpr0
; GFX9-NEXT: s_and_saveexec_b64 s[10:11], vcc
; GFX9-NEXT: s_cbranch_execz BB1_3
; GFX9-NEXT: ; %bb.2:
; GFX9-NEXT: v_mov_b32_e32 v0, s12
; GFX9-NEXT: buffer_atomic_add v0, off, s[4:7], 0 glc
; GFX9-NEXT: BB1_3:
; GFX9-NEXT: s_or_b64 exec, exec, s[10:11]
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_readfirstlane_b32 s4, v0
; GFX9-NEXT: v_mov_b32_e32 v0, v1
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX9-NEXT: v_add_u32_e32 v0, s4, v0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: BB1_4: ; %Flow
; GFX9-NEXT: s_or_b64 exec, exec, s[8:9]
; GFX9-NEXT: s_wqm_b64 s[4:5], -1
; GFX9-NEXT: s_andn2_b64 vcc, exec, s[4:5]
; GFX9-NEXT: s_cbranch_vccnz BB1_6
; GFX9-NEXT: ; %bb.5: ; %if
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX9-NEXT: BB1_6: ; %UnifiedReturnBlock
; GFX9-NEXT: s_endpgm
;
; GFX1064-LABEL: add_i32_varying:
; GFX1064: ; %bb.0: ; %entry
[AMDGPU] Move kill lowering to WQM pass and add live mask tracking Move implementation of kill intrinsics to WQM pass. Add live lane tracking by updating a stored exec mask when lanes are killed. Use live lane tracking to enable early termination of shader at any point in control flow. Reviewed By: piotr Differential Revision: https://reviews.llvm.org/D94746
2021-02-10 11:56:45 +08:00
; GFX1064-NEXT: s_mov_b64 s[8:9], exec
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: v_mov_b32_e32 v1, v0
[AMDGPU] Move kill lowering to WQM pass and add live mask tracking Move implementation of kill intrinsics to WQM pass. Add live lane tracking by updating a stored exec mask when lanes are killed. Use live lane tracking to enable early termination of shader at any point in control flow. Reviewed By: piotr Differential Revision: https://reviews.llvm.org/D94746
2021-02-10 11:56:45 +08:00
; GFX1064-NEXT: s_mov_b64 s[10:11], s[8:9]
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: ; implicit-def: $vgpr0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: s_and_saveexec_b64 s[8:9], s[10:11]
; GFX1064-NEXT: s_cbranch_execz BB1_4
; GFX1064-NEXT: ; %bb.1:
; GFX1064-NEXT: s_not_b64 exec, exec
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: s_not_b64 exec, exec
; GFX1064-NEXT: s_or_saveexec_b64 s[10:11], -1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX1064-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: v_mov_b32_e32 v3, 0
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX1064-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:2 row_mask:0xf bank_mask:0xf bound_ctrl:1
; GFX1064-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:4 row_mask:0xf bank_mask:0xf bound_ctrl:1
; GFX1064-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:8 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: v_mov_b32_e32 v2, v1
; GFX1064-NEXT: v_permlanex16_b32 v2, v2, -1, -1
; GFX1064-NEXT: v_add_nc_u32_dpp v1, v2, v1 quad_perm:[0,1,2,3] row_mask:0xa bank_mask:0xf
; GFX1064-NEXT: v_readlane_b32 s12, v1, 31
; GFX1064-NEXT: v_mov_b32_e32 v2, s12
; GFX1064-NEXT: v_add_nc_u32_dpp v1, v2, v1 quad_perm:[0,1,2,3] row_mask:0xc bank_mask:0xf
; GFX1064-NEXT: v_readlane_b32 s12, v1, 15
; GFX1064-NEXT: v_mov_b32_dpp v3, v1 row_shr:1 row_mask:0xf bank_mask:0xf
; GFX1064-NEXT: v_readlane_b32 s13, v1, 31
; GFX1064-NEXT: v_writelane_b32 v3, s12, 16
; GFX1064-NEXT: s_mov_b64 exec, s[10:11]
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX1064-NEXT: v_mbcnt_lo_u32_b32 v0, exec_lo, 0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: s_or_saveexec_b64 s[10:11], -1
; GFX1064-NEXT: v_readlane_b32 s12, v1, 63
; GFX1064-NEXT: v_readlane_b32 s14, v1, 47
; GFX1064-NEXT: v_writelane_b32 v3, s13, 32
; GFX1064-NEXT: s_mov_b64 exec, s[10:11]
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX1064-NEXT: v_mbcnt_hi_u32_b32 v0, exec_hi, v0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: s_or_saveexec_b64 s[10:11], -1
; GFX1064-NEXT: v_writelane_b32 v3, s14, 48
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: s_mov_b64 exec, s[10:11]
; GFX1064-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
; GFX1064-NEXT: ; implicit-def: $vgpr0
[AMDGPU] Experiments show that the GCNRegBankReassign pass significantly impacts the compilation time and there is no case for which we see any improvement in performance. This patch removes this pass and its associated test cases from the tree. Differential Revision: https://reviews.llvm.org/D101313 Change-Id: I0599169a7609c19a887f8d847a71e664030cc141
2021-04-27 03:48:12 +08:00
; GFX1064-NEXT: s_and_saveexec_b64 s[10:11], vcc
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: s_cbranch_execz BB1_3
; GFX1064-NEXT: ; %bb.2:
; GFX1064-NEXT: v_mov_b32_e32 v0, s12
; GFX1064-NEXT: buffer_atomic_add v0, off, s[4:7], 0 glc
; GFX1064-NEXT: BB1_3:
[AMDGPU] Update VMEM scalar write hazard mitigation sequence Using s_waitcnt_depctr 0xffe3 is potentially faster than v_nop. Reviewed By: rampitec, foad Differential Revision: https://reviews.llvm.org/D83872
2020-07-16 10:07:26 +08:00
; GFX1064-NEXT: s_waitcnt_depctr 0xffe3
[AMDGPU] Experiments show that the GCNRegBankReassign pass significantly impacts the compilation time and there is no case for which we see any improvement in performance. This patch removes this pass and its associated test cases from the tree. Differential Revision: https://reviews.llvm.org/D101313 Change-Id: I0599169a7609c19a887f8d847a71e664030cc141
2021-04-27 03:48:12 +08:00
; GFX1064-NEXT: s_or_b64 exec, exec, s[10:11]
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: s_waitcnt vmcnt(0)
; GFX1064-NEXT: v_readfirstlane_b32 s4, v0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: v_mov_b32_e32 v0, v3
; GFX1064-NEXT: v_add_nc_u32_e32 v0, s4, v0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: BB1_4: ; %Flow
; GFX1064-NEXT: s_or_b64 exec, exec, s[8:9]
; GFX1064-NEXT: s_wqm_b64 s[4:5], -1
; GFX1064-NEXT: s_andn2_b64 vcc, exec, s[4:5]
; GFX1064-NEXT: s_cbranch_vccnz BB1_6
; GFX1064-NEXT: ; %bb.5: ; %if
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1064-NEXT: buffer_store_dword v0, off, s[0:3], 0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1064-NEXT: BB1_6: ; %UnifiedReturnBlock
; GFX1064-NEXT: s_endpgm
;
; GFX1032-LABEL: add_i32_varying:
; GFX1032: ; %bb.0: ; %entry
[AMDGPU] Move kill lowering to WQM pass and add live mask tracking Move implementation of kill intrinsics to WQM pass. Add live lane tracking by updating a stored exec mask when lanes are killed. Use live lane tracking to enable early termination of shader at any point in control flow. Reviewed By: piotr Differential Revision: https://reviews.llvm.org/D94746
2021-02-10 11:56:45 +08:00
; GFX1032-NEXT: s_mov_b32 s8, exec_lo
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: v_mov_b32_e32 v1, v0
[AMDGPU] Move kill lowering to WQM pass and add live mask tracking Move implementation of kill intrinsics to WQM pass. Add live lane tracking by updating a stored exec mask when lanes are killed. Use live lane tracking to enable early termination of shader at any point in control flow. Reviewed By: piotr Differential Revision: https://reviews.llvm.org/D94746
2021-02-10 11:56:45 +08:00
; GFX1032-NEXT: s_mov_b32 s9, s8
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: ; implicit-def: $vgpr0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: s_and_saveexec_b32 s8, s9
; GFX1032-NEXT: s_cbranch_execz BB1_4
; GFX1032-NEXT: ; %bb.1:
; GFX1032-NEXT: s_not_b32 exec_lo, exec_lo
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: s_not_b32 exec_lo, exec_lo
; GFX1032-NEXT: s_or_saveexec_b32 s9, -1
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX1032-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:1 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: v_mov_b32_e32 v3, 0
[AMDGPU][MC] Corrected bound_ctrl for compatibility with sp3 Enabled "bound_ctrl:1" and disabled "bound_ctrl:-1" syntax. Corrected printer to output "bound_ctrl:1" instead of "bound_ctrl:0". See bug 35397 for detailed issue description. Differential Revision: https://reviews.llvm.org/D97048
2021-02-22 19:59:40 +08:00
; GFX1032-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:2 row_mask:0xf bank_mask:0xf bound_ctrl:1
; GFX1032-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:4 row_mask:0xf bank_mask:0xf bound_ctrl:1
; GFX1032-NEXT: v_add_nc_u32_dpp v1, v1, v1 row_shr:8 row_mask:0xf bank_mask:0xf bound_ctrl:1
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: v_mov_b32_e32 v2, v1
; GFX1032-NEXT: v_permlanex16_b32 v2, v2, -1, -1
; GFX1032-NEXT: v_add_nc_u32_dpp v1, v2, v1 quad_perm:[0,1,2,3] row_mask:0xa bank_mask:0xf
; GFX1032-NEXT: v_readlane_b32 s11, v1, 31
; GFX1032-NEXT: v_mov_b32_dpp v3, v1 row_shr:1 row_mask:0xf bank_mask:0xf
; GFX1032-NEXT: v_readlane_b32 s10, v1, 15
; GFX1032-NEXT: s_mov_b32 exec_lo, s9
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX1032-NEXT: v_mbcnt_lo_u32_b32 v0, exec_lo, 0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: s_or_saveexec_b32 s9, -1
; GFX1032-NEXT: v_writelane_b32 v3, s10, 16
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: s_mov_b32 exec_lo, s9
; GFX1032-NEXT: v_cmp_eq_u32_e32 vcc_lo, 0, v0
; GFX1032-NEXT: ; implicit-def: $vgpr0
; GFX1032-NEXT: s_and_saveexec_b32 s9, vcc_lo
; GFX1032-NEXT: s_cbranch_execz BB1_3
; GFX1032-NEXT: ; %bb.2:
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: v_mov_b32_e32 v0, s11
; GFX1032-NEXT: s_mov_b32 s10, s11
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: buffer_atomic_add v0, off, s[4:7], 0 glc
; GFX1032-NEXT: BB1_3:
[AMDGPU] Update VMEM scalar write hazard mitigation sequence Using s_waitcnt_depctr 0xffe3 is potentially faster than v_nop. Reviewed By: rampitec, foad Differential Revision: https://reviews.llvm.org/D83872
2020-07-16 10:07:26 +08:00
; GFX1032-NEXT: s_waitcnt_depctr 0xffe3
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: s_or_b32 exec_lo, exec_lo, s9
; GFX1032-NEXT: s_waitcnt vmcnt(0)
; GFX1032-NEXT: v_readfirstlane_b32 s4, v0
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: v_mov_b32_e32 v0, v3
; GFX1032-NEXT: v_add_nc_u32_e32 v0, s4, v0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: BB1_4: ; %Flow
; GFX1032-NEXT: s_or_b32 exec_lo, exec_lo, s8
; GFX1032-NEXT: s_wqm_b32 s4, -1
; GFX1032-NEXT: s_andn2_b32 vcc_lo, exec_lo, s4
; GFX1032-NEXT: s_cbranch_vccnz BB1_6
; GFX1032-NEXT: ; %bb.5: ; %if
[AMDGPU] Move WQM Pass after MI Scheduler Exec mask manipulation inserted by SIWholeQuadMode barriers to instruction scheduling. Move the entire pass after the machine instruction scheduler and make changes so pass is correct for non-SSA operation. These changes should leave the pass still usable pre-scheduler, although tests have be updated to reflect post-scheduler results. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D88081
2020-10-27 08:55:55 +08:00
; GFX1032-NEXT: buffer_store_dword v0, off, s[0:3], 0
[AMDGPU] Use script to generate atomic optimizations test This is a preparation for introducing a llvm.amdgcn.ballot intrinsic in D65088.
2020-03-05 21:29:40 +08:00
; GFX1032-NEXT: BB1_6: ; %UnifiedReturnBlock
; GFX1032-NEXT: s_endpgm
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
entry:
%cond1 = call i1 @llvm.amdgcn.wqm.vote(i1 true)
AMDGPU: Update more tests to use modern buffer intrinsics
2020-01-17 03:05:46 +08:00
%old = call i32 @llvm.amdgcn.raw.buffer.atomic.add(i32 %val, <4 x i32> %inout, i32 0, i32 0, i32 0)
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
%cond2 = call i1 @llvm.amdgcn.wqm.vote(i1 true)
%cond = and i1 %cond1, %cond2
br i1 %cond, label %if, label %else
if:
%bitcast = bitcast i32 %old to float
AMDGPU: Update more tests to use modern buffer intrinsics
2020-01-17 03:05:46 +08:00
call void @llvm.amdgcn.raw.buffer.store.f32(float %bitcast, <4 x i32> %out, i32 0, i32 0, i32 0)
[AMDGPU] Fix the new atomic optimizer in pixel shaders. The new atomic optimizer I previously added in D51969 did not work correctly when a pixel shader was using derivatives, and had helper lanes active. To fix this we add an llvm.amdgcn.ps.live call that guards a branch around the entire atomic operation - ensuring that all helper lanes are inactive within the wavefront when we compute our atomic results. I've added a test case that can cause derivatives, and exposes the problem. Differential Revision: https://reviews.llvm.org/D53930 llvm-svn: 346128
2018-11-05 20:04:48 +08:00
ret void
else:
ret void
}