maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/AMDGPU/shrink-add-sub-constant.ll

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2248 lines
88 KiB
LLVM
Raw Normal View History

[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc < %s -march=amdgcn -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,SI
; RUN: llc < %s -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,VI
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; RUN: llc < %s -march=amdgcn -mcpu=gfx900 -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,GFX9
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; RUN: llc < %s -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,GFX10
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
; Test that add/sub with a constant is swapped to sub/add with negated
; constant to minimize code size.
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_x_sub_64(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_x_sub_64:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_subrev_i32_e32 v2, vcc, 64, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_x_sub_64:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_subrev_u32_e32 v0, vcc, 64, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_x_sub_64:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_subrev_u32_e32 v1, 64, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_x_sub_64:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_subrev_nc_u32_e32 v1, 64, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 %x, 64
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_x_sub_64_multi_use(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_x_sub_64_multi_use:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: buffer_load_dword v3, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_subrev_i32_e32 v2, vcc, 64, v2
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_subrev_i32_e32 v3, vcc, 64, v3
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: buffer_store_dword v3, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_x_sub_64_multi_use:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: flat_load_dword v4, v[0:1]
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(1) lgkmcnt(1)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_subrev_u32_e32 v2, vcc, 64, v3
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_subrev_u32_e32 v3, vcc, 64, v4
; VI-NEXT: flat_store_dword v[0:1], v2
; VI-NEXT: flat_store_dword v[0:1], v3
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_x_sub_64_multi_use:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: global_load_dword v2, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(1)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_subrev_u32_e32 v1, 64, v1
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_subrev_u32_e32 v2, 64, v2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
; GFX9-NEXT: global_store_dword v0, v2, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_x_sub_64_multi_use:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] New SIInsertHardClauses pass Enable clausing of memory loads on gfx10 by adding a new pass to insert the s_clause instructions that mark the start of each hard clause. Differential Revision: https://reviews.llvm.org/D79792
2020-05-01 23:43:12 +08:00
; GFX10-NEXT: s_clause 0x1
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
; GFX10-NEXT: global_load_dword v2, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(1)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_subrev_nc_u32_e32 v1, 64, v1
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_subrev_nc_u32_e32 v2, 64, v2
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
; GFX10-NEXT: global_store_dword v0, v2, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load volatile i32, i32 addrspace(1)* %gep
%y = load volatile i32, i32 addrspace(1)* %gep
%result0 = sub i32 %x, 64
%result1 = sub i32 %y, 64
store volatile i32 %result0, i32 addrspace(1)* %gep.out
store volatile i32 %result1, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_64_sub_x(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_64_sub_x:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_sub_i32_e32 v2, vcc, 64, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_64_sub_x:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_sub_u32_e32 v0, vcc, 64, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_64_sub_x:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_sub_u32_e32 v1, 64, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_64_sub_x:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u32_e32 v1, 64, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 64, %x
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_x_sub_65(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_x_sub_65:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xffffffbf, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_x_sub_65:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_add_u32_e32 v0, vcc, 0xffffffbf, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_x_sub_65:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_add_u32_e32 v1, 0xffffffbf, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_x_sub_65:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_add_nc_u32_e32 v1, 0xffffffbf, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 %x, 65
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_65_sub_x(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_65_sub_x:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_sub_i32_e32 v2, vcc, 0x41, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_65_sub_x:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_sub_u32_e32 v0, vcc, 0x41, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_65_sub_x:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_sub_u32_e32 v1, 0x41, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_65_sub_x:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u32_e32 v1, 0x41, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 65, %x
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_x_sub_neg16(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_x_sub_neg16:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 16, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_x_sub_neg16:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_add_u32_e32 v0, vcc, 16, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_x_sub_neg16:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_add_u32_e32 v1, 16, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_x_sub_neg16:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_add_nc_u32_e32 v1, 16, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 %x, -16
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_neg16_sub_x(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_neg16_sub_x:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_sub_i32_e32 v2, vcc, -16, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_neg16_sub_x:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_sub_u32_e32 v0, vcc, -16, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_neg16_sub_x:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_sub_u32_e32 v1, -16, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_neg16_sub_x:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u32_e32 v1, -16, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 -16, %x
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_x_sub_neg17(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_x_sub_neg17:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 17, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_x_sub_neg17:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_add_u32_e32 v0, vcc, 17, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_x_sub_neg17:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_add_u32_e32 v1, 17, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_x_sub_neg17:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_add_nc_u32_e32 v1, 17, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 %x, -17
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i32_neg17_sub_x(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i32_neg17_sub_x:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_sub_i32_e32 v2, vcc, 0xffffffef, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i32_neg17_sub_x:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_sub_u32_e32 v0, vcc, 0xffffffef, v0
; VI-NEXT: flat_store_dword v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i32_neg17_sub_x:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_sub_u32_e32 v1, 0xffffffef, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i32_neg17_sub_x:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u32_e32 v1, 0xffffffef, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i32, i32 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i32, i32 addrspace(1)* %gep
%result = sub i32 -17, %x
store i32 %result, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @s_test_i32_x_sub_64(i32 %x) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: s_test_i32_x_sub_64:
; SI: ; %bb.0:
; SI-NEXT: s_load_dword s0, s[0:1], 0x9
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_sub_i32 s0, s0, 64
; SI-NEXT: ;;#ASMSTART
; SI-NEXT: ; use s0
; SI-NEXT: ;;#ASMEND
; SI-NEXT: s_endpgm
;
; VI-LABEL: s_test_i32_x_sub_64:
; VI: ; %bb.0:
; VI-NEXT: s_load_dword s0, s[0:1], 0x24
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: s_sub_i32 s0, s0, 64
; VI-NEXT: ;;#ASMSTART
; VI-NEXT: ; use s0
; VI-NEXT: ;;#ASMEND
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: s_test_i32_x_sub_64:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dword s0, s[0:1], 0x24
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: s_sub_i32 s0, s0, 64
; GFX9-NEXT: ;;#ASMSTART
; GFX9-NEXT: ; use s0
; GFX9-NEXT: ;;#ASMEND
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: s_test_i32_x_sub_64:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dword s0, s[0:1], 0x24
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
; GFX10-NEXT: s_sub_i32 s0, s0, 64
; GFX10-NEXT: ;;#ASMSTART
; GFX10-NEXT: ; use s0
; GFX10-NEXT: ;;#ASMEND
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%result = sub i32 %x, 64
call void asm sideeffect "; use $0", "s"(i32 %result)
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i16_x_sub_64(i16 addrspace(1)* %out, i16 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i16_x_sub_64:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 1, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_ushort v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_subrev_i32_e32 v2, vcc, 64, v2
; SI-NEXT: buffer_store_short v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i16_x_sub_64:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 1, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_ushort v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_subrev_u16_e32 v0, 64, v0
; VI-NEXT: flat_store_short v[2:3], v0
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i16_x_sub_64:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 1, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_ushort v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_subrev_u16_e32 v1, 64, v1
; GFX9-NEXT: global_store_short v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i16_x_sub_64:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 1, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_ushort v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u16_e64 v1, v1, 64
; GFX10-NEXT: global_store_short v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i16, i16 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i16, i16 addrspace(1)* %out, i64 %tid.ext
%x = load i16, i16 addrspace(1)* %gep
%result = sub i16 %x, 64
store i16 %result, i16 addrspace(1)* %gep.out
ret void
}
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
define amdgpu_kernel void @v_test_i16_x_sub_64_zext_to_i32(i32 addrspace(1)* %out, i16 addrspace(1)* %in) #0 {
; SI-LABEL: v_test_i16_x_sub_64_zext_to_i32:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v1, 1, v0
; SI-NEXT: v_mov_b32_e32 v2, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_ushort v3, v[1:2], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: v_lshlrev_b32_e32 v1, 2, v0
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_subrev_i32_e32 v0, vcc, 64, v3
; SI-NEXT: v_and_b32_e32 v0, 0xffff, v0
; SI-NEXT: buffer_store_dword v0, v[1:2], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i16_x_sub_64_zext_to_i32:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v1, 1, v0
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; VI-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_mov_b32_e32 v2, s3
; VI-NEXT: v_add_u32_e32 v1, vcc, s2, v1
; VI-NEXT: v_addc_u32_e32 v2, vcc, 0, v2, vcc
; VI-NEXT: v_add_u32_e32 v3, vcc, s0, v0
; VI-NEXT: flat_load_ushort v0, v[1:2]
; VI-NEXT: v_mov_b32_e32 v4, s1
; VI-NEXT: v_addc_u32_e32 v4, vcc, 0, v4, vcc
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_subrev_u16_e32 v0, 64, v0
; VI-NEXT: flat_store_dword v[3:4], v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_i16_x_sub_64_zext_to_i32:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; GFX9-NEXT: v_lshlrev_b32_e32 v1, 1, v0
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_ushort v1, v1, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_subrev_u16_e32 v1, 64, v1
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX9-NEXT: s_endpgm
;
; GFX10-LABEL: v_test_i16_x_sub_64_zext_to_i32:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; GFX10-NEXT: v_lshlrev_b32_e32 v1, 1, v0
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_ushort v1, v1, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u16_e64 v1, v1, 64
; GFX10-NEXT: v_and_b32_e32 v1, 0xffff, v1
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i16, i16 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i32, i32 addrspace(1)* %out, i64 %tid.ext
%x = load i16, i16 addrspace(1)* %gep
%result = sub i16 %x, 64
%zext = zext i16 %result to i32
store i32 %zext, i32 addrspace(1)* %gep.out
ret void
}
AMDGPU: Mark all unspecified CC functions in tests as amdgpu_kernel Currently the default C calling convention functions are treated the same as compute kernels. Make this explicit so the default calling convention can be changed to a non-kernel. Converted with perl -pi -e 's/define void/define amdgpu_kernel void/' on the relevant test directories (and undoing in one place that actually wanted a non-kernel). llvm-svn: 298444
2017-03-22 05:39:51 +08:00
define amdgpu_kernel void @v_test_i16_x_sub_64_multi_use(i16 addrspace(1)* %out, i16 addrspace(1)* %in) #0 {
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; SI-LABEL: v_test_i16_x_sub_64_multi_use:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 1, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_ushort v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: buffer_load_ushort v3, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_subrev_i32_e32 v2, vcc, 64, v2
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_subrev_i32_e32 v3, vcc, 64, v3
; SI-NEXT: buffer_store_short v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: buffer_store_short v3, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_i16_x_sub_64_multi_use:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 1, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_ushort v3, v[0:1]
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: flat_load_ushort v4, v[0:1]
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(1) lgkmcnt(1)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_subrev_u16_e32 v2, 64, v3
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_subrev_u16_e32 v3, 64, v4
; VI-NEXT: flat_store_short v[0:1], v2
; VI-NEXT: flat_store_short v[0:1], v3
[AMDGPU] Regenerate add/sub shrink constant tests for an upcoming patch llvm-svn: 362230
2019-05-31 23:06:51 +08:00
; VI-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
;
; GFX9-LABEL: v_test_i16_x_sub_64_multi_use:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 1, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_ushort v1, v0, s[2:3]
; GFX9-NEXT: global_load_ushort v2, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(1)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_subrev_u16_e32 v1, 64, v1
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_subrev_u16_e32 v2, 64, v2
; GFX9-NEXT: global_store_short v0, v1, s[0:1]
; GFX9-NEXT: global_store_short v0, v2, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_i16_x_sub_64_multi_use:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 1, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] New SIInsertHardClauses pass Enable clausing of memory loads on gfx10 by adding a new pass to insert the s_clause instructions that mark the start of each hard clause. Differential Revision: https://reviews.llvm.org/D79792
2020-05-01 23:43:12 +08:00
; GFX10-NEXT: s_clause 0x1
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_ushort v1, v0, s[2:3]
; GFX10-NEXT: global_load_ushort v2, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(1)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u16_e64 v1, v1, 64
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_sub_nc_u16_e64 v2, v2, 64
; GFX10-NEXT: global_store_short v0, v1, s[0:1]
; GFX10-NEXT: global_store_short v0, v2, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds i16, i16 addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds i16, i16 addrspace(1)* %out, i64 %tid.ext
%x = load volatile i16, i16 addrspace(1)* %gep
%y = load volatile i16, i16 addrspace(1)* %gep
%result0 = sub i16 %x, 64
%result1 = sub i16 %y, 64
store volatile i16 %result0, i16 addrspace(1)* %gep.out
store volatile i16 %result1, i16 addrspace(1)* %gep.out
ret void
}
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
define amdgpu_kernel void @v_test_v2i16_x_sub_64_64(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_sub_64_64:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_subrev_i32_e32 v3, vcc, 64, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xffc00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_sub_64_64:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v4, 64
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_sub_u16_sdwa v2, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_subrev_u16_e32 v3, 64, v3
; VI-NEXT: v_or_b32_e32 v2, v3, v2
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_sub_64_64:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_i16 v1, v1, 64 op_sel_hi:[1,0]
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_sub_64_64:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_i16 v1, v1, 64 op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = sub <2 x i16> %x, <i16 64, i16 64>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_sub_7_64(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_sub_7_64:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, -7, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xffc00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_sub_7_64:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_mov_b32_e32 v4, 64
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u16_e32 v2, -7, v3
; VI-NEXT: v_sub_u16_sdwa v3, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_sub_7_64:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_mov_b32 s2, 0x400007
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_i16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_sub_7_64:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Fix for folding v2.16 literals. It was found some packed immediate operands (e.g. `<half 1.0, half 2.0>`) are incorrectly processed so one of two packed values were lost. Introduced new function to check immediate 32-bit operand can be folded. Converted condition about current op_sel flags value to fall-through. Fixes: SWDEV-247595 Reviewed By: rampitec Differential Revision: https://reviews.llvm.org/D87158
2020-09-05 03:44:01 +08:00
; GFX10-NEXT: v_pk_sub_i16 v1, v1, 0x400007
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = sub <2 x i16> %x, <i16 7, i16 64>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_sub_64_123(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_sub_64_123:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_subrev_i32_e32 v3, vcc, 64, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xff850000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_sub_64_123:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v4, 0xffffff85
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u16_sdwa v2, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_subrev_u16_e32 v3, 64, v3
; VI-NEXT: v_or_b32_e32 v2, v3, v2
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_sub_64_123:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_mov_b32 s2, 0x7b0040
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_i16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_sub_64_123:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Fix for folding v2.16 literals. It was found some packed immediate operands (e.g. `<half 1.0, half 2.0>`) are incorrectly processed so one of two packed values were lost. Introduced new function to check immediate 32-bit operand can be folded. Converted condition about current op_sel flags value to fall-through. Fixes: SWDEV-247595 Reviewed By: rampitec Differential Revision: https://reviews.llvm.org/D87158
2020-09-05 03:44:01 +08:00
; GFX10-NEXT: v_pk_sub_i16 v1, v1, 0x7b0040
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = sub <2 x i16> %x, <i16 64, i16 123>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
; Can fold 0 and inline immediate in other half.
define amdgpu_kernel void @v_test_v2i16_x_sub_7_0(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_sub_7_0:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, -7, v2
; SI-NEXT: s_mov_b32 s4, 0xffff
; SI-NEXT: v_bfi_b32 v2, s4, v3, v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_sub_7_0:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_and_b32_e32 v2, 0xffff0000, v3
; VI-NEXT: v_add_u16_e32 v3, -7, v3
; VI-NEXT: v_or_b32_e32 v2, v3, v2
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_sub_7_0:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_i16 v1, v1, 7
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_sub_7_0:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_i16 v1, v1, 7
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = sub <2 x i16> %x, <i16 7, i16 0>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
; Can fold 0 and inline immediate in other half.
define amdgpu_kernel void @v_test_v2i16_x_sub_0_16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_sub_0_16:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xfff00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_sub_0_16:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v1, -16
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_add_u16_sdwa v1, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: flat_store_dword v[2:3], v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_sub_0_16:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_i16 v1, v1, 16 op_sel:[0,1] op_sel_hi:[1,0]
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_sub_0_16:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_i16 v1, v1, 16 op_sel:[0,1] op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = sub <2 x i16> %x, <i16 0, i16 16>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_sub_0_1_0(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_sub_0_1_0:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 0x3c000000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_sub_0_1_0:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v1, 0x3c00
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_add_u16_sdwa v1, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: flat_store_dword v[2:3], v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_sub_0_1_0:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_brev_b32 s2, 35
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_i16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_sub_0_1_0:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_i16 v1, v1, 0xc400 op_sel:[0,1] op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = sub <2 x i16> %x, <i16 0, i16 -15360>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_sub_0_neg1_0(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_sub_0_neg1_0:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xbc000000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_sub_0_neg1_0:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v1, 0xffffbc00
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_add_u16_sdwa v1, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: flat_store_dword v[2:3], v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_sub_0_neg1_0:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_brev_b32 s2, 34
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_i16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_sub_0_neg1_0:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_i16 v1, v1, 0x4400 op_sel:[0,1] op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = sub <2 x i16> %x, <i16 0, i16 17408>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
; -32 isn't an inline immediate, but 32 is
define amdgpu_kernel void @v_test_v2i16_x_add_neg32_neg32(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg32_neg32:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_subrev_i32_e32 v3, vcc, 32, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xffe00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg32_neg32:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v4, 32
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_sub_u16_sdwa v2, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_subrev_u16_e32 v3, 32, v3
; VI-NEXT: v_or_b32_e32 v2, v3, v2
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg32_neg32:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 32 op_sel_hi:[1,0]
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg32_neg32:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 32 op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 -32, i16 -32>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_0_neg32(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_0_neg32:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xffe00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_0_neg32:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v1, 32
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_sub_u16_sdwa v1, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: flat_store_dword v[2:3], v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_0_neg32:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 32 op_sel:[0,1] op_sel_hi:[1,0]
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_0_neg32:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 32 op_sel:[0,1] op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 0, i16 -32>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_neg32_0(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg32_0:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_subrev_i32_e32 v3, vcc, 32, v2
; SI-NEXT: s_mov_b32 s4, 0xffff
; SI-NEXT: v_bfi_b32 v2, s4, v3, v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg32_0:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_and_b32_e32 v2, 0xffff0000, v3
; VI-NEXT: v_subrev_u16_e32 v3, 32, v3
; VI-NEXT: v_or_b32_e32 v2, v3, v2
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg32_0:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 32
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg32_0:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 32
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 -32, i16 0>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
; 16 and -16 are both inline immediates
define amdgpu_kernel void @v_test_v2i16_x_add_neg16_neg16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg16_neg16:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, -16, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xfff00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg16_neg16:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_mov_b32_e32 v4, -16
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u16_e32 v2, -16, v3
; VI-NEXT: v_add_u16_sdwa v3, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg16_neg16:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 16 op_sel_hi:[1,0]
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg16_neg16:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 16 op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 -16, i16 -16>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_0_neg16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_0_neg16:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xfff00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_0_neg16:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v1, -16
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_add_u16_sdwa v1, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-NEXT: flat_store_dword v[2:3], v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_0_neg16:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 16 op_sel:[0,1] op_sel_hi:[1,0]
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_0_neg16:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 16 op_sel:[0,1] op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 0, i16 -16>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_neg16_0(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg16_0:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, -16, v2
; SI-NEXT: s_mov_b32 s4, 0xffff
; SI-NEXT: v_bfi_b32 v2, s4, v3, v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg16_0:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_and_b32_e32 v2, 0xffff0000, v3
; VI-NEXT: v_add_u16_e32 v3, -16, v3
; VI-NEXT: v_or_b32_e32 v2, v3, v2
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg16_0:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 16
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg16_0:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 16
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 -16, i16 0>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_neg_fpone(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg_fpone:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, 0xffffc400, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xc4000000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg_fpone:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: s_movk_i32 s2, 0xc400
; VI-NEXT: v_mov_b32_e32 v4, s2
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u16_e32 v2, s2, v3
; VI-NEXT: v_add_u16_sdwa v3, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg_fpone:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_mov_b32 s2, 0x3c003c00
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg_fpone:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 0x3c00 op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 -15360, i16 -15360>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_neg_negfpone(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg_negfpone:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x4400, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 0x44000000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg_negfpone:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: s_movk_i32 s2, 0x4400
; VI-NEXT: v_mov_b32_e32 v4, s2
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u16_e32 v2, s2, v3
; VI-NEXT: v_add_u16_sdwa v3, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg_negfpone:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_mov_b32 s2, 0xbc00bc00
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg_negfpone:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 0xbc00 op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 17408, i16 17408>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_neg_fptwo(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg_fptwo:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, 0x4000, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, 2.0, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg_fptwo:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: s_movk_i32 s2, 0x4000
; VI-NEXT: v_mov_b32_e32 v4, s2
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u16_e32 v2, s2, v3
; VI-NEXT: v_add_u16_sdwa v3, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg_fptwo:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_mov_b32 s2, 0xc000c000
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg_fptwo:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 0xc000 op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 16384, i16 16384>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_neg_negfptwo(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg_negfptwo:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Add bfi immediate pattern Differential Revision: https://reviews.llvm.org/D88246
2020-09-25 00:02:30 +08:00
; SI-NEXT: v_add_i32_e32 v3, vcc, 0xffffc000, v2
; SI-NEXT: s_mov_b32 s4, 0xffff0000
; SI-NEXT: v_bfi_b32 v2, s4, v2, v3
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; SI-NEXT: v_add_i32_e32 v2, vcc, -2.0, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg_negfptwo:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: flat_load_dword v3, v[0:1]
; VI-NEXT: s_movk_i32 s2, 0xc000
; VI-NEXT: v_mov_b32_e32 v4, s2
; VI-NEXT: v_add_u32_e32 v0, vcc, s0, v2
; VI-NEXT: v_mov_b32_e32 v1, s1
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; VI-NEXT: v_add_u16_e32 v2, s2, v3
; VI-NEXT: v_add_u16_sdwa v3, v3, v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: v_or_b32_e32 v2, v2, v3
; VI-NEXT: flat_store_dword v[0:1], v2
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg_negfptwo:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
; GFX9-NEXT: s_mov_b32 s2, 0x40004000
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, s2
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg_negfptwo:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 0x4000 op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 -16384, i16 -16384>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_undef_neg32(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_undef_neg32:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v2, 0xffff0000, v2
; SI-NEXT: v_add_i32_e32 v2, vcc, 0xffe00000, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_undef_neg32:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v1, 32
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_sub_u16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; VI-NEXT: flat_store_dword v[2:3], v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_undef_neg32:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 32 op_sel:[0,1] op_sel_hi:[1,0]
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_undef_neg32:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 32 op_sel:[0,1] op_sel_hi:[1,0]
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 undef, i16 -32>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
define amdgpu_kernel void @v_test_v2i16_x_add_neg32_undef(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
; SI-LABEL: v_test_v2i16_x_add_neg32_undef:
; SI: ; %bb.0:
; SI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x9
; SI-NEXT: s_mov_b32 s7, 0xf000
; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; SI-NEXT: v_mov_b32_e32 v1, 0
; SI-NEXT: s_waitcnt lgkmcnt(0)
; SI-NEXT: s_mov_b64 s[4:5], s[2:3]
; SI-NEXT: buffer_load_dword v2, v[0:1], s[4:7], 0 addr64
; SI-NEXT: s_mov_b64 s[2:3], s[6:7]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_subrev_i32_e32 v2, vcc, 32, v2
; SI-NEXT: v_and_b32_e32 v2, 0xffff, v2
; SI-NEXT: buffer_store_dword v2, v[0:1], s[0:3], 0 addr64
; SI-NEXT: s_endpgm
;
; VI-LABEL: v_test_v2i16_x_add_neg32_undef:
; VI: ; %bb.0:
; VI-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
; VI-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; VI-NEXT: s_waitcnt lgkmcnt(0)
; VI-NEXT: v_mov_b32_e32 v1, s3
; VI-NEXT: v_add_u32_e32 v0, vcc, s2, v2
; VI-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: flat_load_dword v0, v[0:1]
; VI-NEXT: v_mov_b32_e32 v3, s1
; VI-NEXT: v_add_u32_e32 v2, vcc, s0, v2
; VI-NEXT: v_addc_u32_e32 v3, vcc, 0, v3, vcc
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; VI-NEXT: v_subrev_u16_e32 v0, 32, v0
; VI-NEXT: flat_store_dword v[2:3], v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; VI-NEXT: s_endpgm
;
; GFX9-LABEL: v_test_v2i16_x_add_neg32_undef:
; GFX9: ; %bb.0:
; GFX9-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX9-NEXT: v_pk_sub_u16 v1, v1, 32
; GFX9-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
; GFX9-NEXT: s_endpgm
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
;
; GFX10-LABEL: v_test_v2i16_x_add_neg32_undef:
; GFX10: ; %bb.0:
; GFX10-NEXT: s_load_dwordx4 s[0:3], s[0:1], 0x24
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_lshlrev_b32_e32 v0, 2, v0
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: ; implicit-def: $vcc_hi
; GFX10-NEXT: s_waitcnt lgkmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: global_load_dword v1, v0, s[2:3]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_waitcnt vmcnt(0)
AMDGPU: Match global saddr addressing mode The previous implementation was incorrect, and based off incorrect instruction definitions. Unfortunately we can't match natural addressing in a lot of cases due to the shift/scale applied in getelementptrs. This relies on reducing the 64-bit shift to 32-bits.
2020-08-14 04:51:07 +08:00
; GFX10-NEXT: v_pk_sub_u16 v1, v1, 32
; GFX10-NEXT: global_store_dword v0, v1, s[0:1]
AMDGPU: Add baseline test for missing pattern The optimization to turn an add into a sub isn't triggering when the pattern to use the zeroed high bits is used.
2020-01-08 02:18:51 +08:00
; GFX10-NEXT: s_endpgm
AMDGPU: Add baseline test for vector sub x, c canonicalization This will catch regressions from D62341, and show improvements from a future patch to fix them. llvm-svn: 363888
2019-06-20 06:37:08 +08:00
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64
%gep = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %in, i64 %tid.ext
%gep.out = getelementptr inbounds <2 x i16>, <2 x i16> addrspace(1)* %out, i64 %tid.ext
%x = load <2 x i16>, <2 x i16> addrspace(1)* %gep
%result = add <2 x i16> %x, <i16 -32, i16 undef>
store <2 x i16> %result, <2 x i16> addrspace(1)* %gep.out
ret void
}
AMDGPU: Undo sub x, c -> add x, -c canonicalization This is worse if the original constant is an inline immediate. This should also be done for 64-bit adds, but requires fixing operand folding bugs first. llvm-svn: 293540
2017-01-31 03:30:24 +08:00
declare i32 @llvm.amdgcn.workitem.id.x() #1
attributes #0 = { nounwind }
attributes #1 = { nounwind readnone }