maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/AMDGPU/idot4u.ll

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2273 lines
98 KiB
LLVM
Raw Normal View History

[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple=amdgcn -mcpu=gfx700 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX7 %s
; RUN: llc -mtriple=amdgcn -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX8 %s
; RUN: llc -mtriple=amdgcn -mcpu=gfx900 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX9-NODL %s
; RUN: llc -mtriple=amdgcn -mcpu=gfx906 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX9-DL %s
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; RUN: llc -mtriple=amdgcn -mcpu=gfx1011 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX10-DL %s
; RUN: llc -mtriple=amdgcn -mcpu=gfx1012 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX10-DL %s
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
define amdgpu_kernel void @udot4_acc32(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot4_acc32:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
; GFX7-NEXT: s_load_dword s4, s[0:1], 0x0
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v1, v1, v5, s4
; GFX7-NEXT: v_mad_u32_u24 v1, v3, v6, v1
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v4, v7, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_acc32:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_movk_i32 s2, 0xff
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v3, v[0:1]
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s7
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: s_load_dword s3, s[0:1], 0x0
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_and_b32_e32 v1, s2, v3
; GFX8-NEXT: v_bfe_u32 v4, v3, 8, 8
; GFX8-NEXT: v_bfe_u32 v6, v3, 16, 8
; GFX8-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_and_b32_e32 v2, s2, v0
; GFX8-NEXT: v_bfe_u32 v5, v0, 8, 8
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
; GFX8-NEXT: v_mad_u32_u24 v1, v1, v2, s3
; GFX8-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX8-NEXT: v_mad_u32_u24 v1, v4, v5, v1
; GFX8-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX8-NEXT: v_mad_u32_u24 v1, v6, v7, v1
; GFX8-NEXT: v_mad_u32_u24 v2, v3, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: v_mov_b32_e32 v0, s0
; GFX8-NEXT: v_mov_b32_e32 v1, s1
; GFX8-NEXT: flat_store_dword v[0:1], v2
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_acc32:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-NODL-NEXT: s_load_dword s0, s[2:3], 0x0
; GFX9-NODL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:BYTE_0
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v4, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v5, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_add3_u32 v2, v3, s0, v4
; GFX9-NODL-NEXT: v_add3_u32 v1, v2, v5, v1
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: global_store_dword v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_acc32:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-DL-NEXT: s_load_dword s0, s[2:3], 0x0
; GFX9-DL-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
; GFX9-DL-NEXT: v_dot4_u32_u8 v0, v2, v3, s0
; GFX9-DL-NEXT: global_store_dword v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_acc32:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX10-DL-NEXT: s_load_dword s2, s[0:1], 0x0
; GFX10-DL-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
; GFX10-DL-NEXT: v_dot4_u32_u8 v1, v1, v2, s2
; GFX10-DL-NEXT: global_store_dword v0, v1, s[0:1]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
<4 x i8> addrspace(1)* %src2,
i32 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%cv1e0 = zext i8 %v1e0 to i32
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%cv2e0 = zext i8 %v2e0 to i32
%mul1 = mul nuw nsw i32 %cv1e0, %cv2e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%cv1e1 = zext i8 %v1e1 to i32
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%cv2e1 = zext i8 %v2e1 to i32
%mul2 = mul nuw nsw i32 %cv1e1, %cv2e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%cv1e2 = zext i8 %v1e2 to i32
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%cv2e2 = zext i8 %v2e2 to i32
%mul3 = mul nuw nsw i32 %cv1e2, %cv2e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%cv1e3 = zext i8 %v1e3 to i32
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%cv2e3 = zext i8 %v2e3 to i32
%mul4 = mul nuw nsw i32 %cv1e3, %cv2e3
%acc = load i32, i32 addrspace(1)* %dst, align 4
%mad1 = add i32 %mul1, %acc
%mad2 = add i32 %mad1, %mul2
%mad3 = add i32 %mad2, %mul3
%mad4 = add i32 %mad3, %mul4
store i32 %mad4, i32 addrspace(1)* %dst, align 4
ret void
}
define amdgpu_kernel void @udot4_acc16(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot4_acc16:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_ushort v8, off, s[0:3], 0
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v1, v1, v5, v8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v1, v3, v6, v1
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v4, v7, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_acc16:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v4, v[0:1]
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_mov_b32_e32 v3, s1
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: v_mov_b32_e32 v2, s0
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: flat_load_ushort v1, v[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v5, s0
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_lshrrev_b32_e32 v8, 8, v4
; GFX8-NEXT: v_and_b32_e32 v6, s0, v4
; GFX8-NEXT: v_and_b32_e32 v8, s0, v8
; GFX8-NEXT: v_and_b32_sdwa v10, v4, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX8-NEXT: v_lshrrev_b32_e32 v4, 24, v4
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_lshrrev_b32_e32 v9, 8, v0
; GFX8-NEXT: v_and_b32_e32 v7, s0, v0
; GFX8-NEXT: v_and_b32_e32 v9, s0, v9
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mad_u16 v1, v6, v7, v1
; GFX8-NEXT: v_and_b32_sdwa v5, v0, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX8-NEXT: v_mad_u16 v1, v8, v9, v1
; GFX8-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX8-NEXT: v_mad_u16 v1, v10, v5, v1
; GFX8-NEXT: v_mad_u16 v0, v4, v0, v1
; GFX8-NEXT: flat_store_short v[2:3], v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_acc16:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-NODL-NEXT: global_load_ushort v4, v1, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v6, 8, v2
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v7, 8, v3
; GFX9-NODL-NEXT: v_and_b32_e32 v0, s0, v2
; GFX9-NODL-NEXT: v_and_b32_e32 v5, s0, v3
; GFX9-NODL-NEXT: v_and_b32_e32 v6, s0, v6
; GFX9-NODL-NEXT: v_and_b32_e32 v7, s0, v7
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v0, v5, v4
; GFX9-NODL-NEXT: v_and_b32_sdwa v8, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-NODL-NEXT: v_and_b32_sdwa v9, v3, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v6, v7, v0
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v8, v9, v0
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v2, v3, v0
; GFX9-NODL-NEXT: global_store_short v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_acc16:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: v_mov_b32_e32 v1, 0
; GFX9-DL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-DL-NEXT: global_load_ushort v4, v1, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v6, 8, v2
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v7, 8, v3
; GFX9-DL-NEXT: v_and_b32_e32 v0, s0, v2
; GFX9-DL-NEXT: v_and_b32_e32 v5, s0, v3
; GFX9-DL-NEXT: v_and_b32_e32 v6, s0, v6
; GFX9-DL-NEXT: v_and_b32_e32 v7, s0, v7
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v0, v5, v4
; GFX9-DL-NEXT: v_and_b32_sdwa v8, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-DL-NEXT: v_and_b32_sdwa v9, v3, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v6, v7, v0
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v8, v9, v0
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v2, v3, v0
; GFX9-DL-NEXT: global_store_short v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_acc16:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX10-DL-NEXT: global_load_ushort v3, v0, s[2:3]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v1
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v6, 8, v2
; GFX10-DL-NEXT: v_and_b32_e32 v4, s0, v1
; GFX10-DL-NEXT: v_and_b32_e32 v10, s0, v2
; GFX10-DL-NEXT: v_and_b32_e32 v5, s0, v5
; GFX10-DL-NEXT: v_and_b32_e32 v6, s0, v6
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
; GFX10-DL-NEXT: v_mad_u16 v3, v4, v10, v3
; GFX10-DL-NEXT: v_and_b32_sdwa v4, v1, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX10-DL-NEXT: v_and_b32_sdwa v10, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v1, 24, v1
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX10-DL-NEXT: v_mad_u16 v3, v5, v6, v3
; GFX10-DL-NEXT: v_mad_u16 v3, v4, v10, v3
; GFX10-DL-NEXT: v_mad_u16 v1, v1, v2, v3
; GFX10-DL-NEXT: global_store_short v0, v1, s[2:3]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
<4 x i8> addrspace(1)* %src2,
i16 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%cv1e0 = zext i8 %v1e0 to i16
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%cv2e0 = zext i8 %v2e0 to i16
%mul1 = mul nuw nsw i16 %cv1e0, %cv2e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%cv1e1 = zext i8 %v1e1 to i16
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%cv2e1 = zext i8 %v2e1 to i16
%mul2 = mul nuw nsw i16 %cv1e1, %cv2e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%cv1e2 = zext i8 %v1e2 to i16
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%cv2e2 = zext i8 %v2e2 to i16
%mul3 = mul nuw nsw i16 %cv1e2, %cv2e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%cv1e3 = zext i8 %v1e3 to i16
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%cv2e3 = zext i8 %v2e3 to i16
%mul4 = mul nuw nsw i16 %cv1e3, %cv2e3
%acc = load i16, i16 addrspace(1)* %dst, align 2
%mad1 = add i16 %mul1, %acc
%mad2 = add i16 %mad1, %mul2
%mad3 = add i16 %mad2, %mul3
%mad4 = add i16 %mad3, %mul4
store i16 %mad4, i16 addrspace(1)* %dst, align 2
ret void
}
define amdgpu_kernel void @udot4_acc8(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot4_acc8:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_ubyte v8, off, s[0:3], 0
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v1, v1, v5, v8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v1, v3, v6, v1
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v4, v7, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_byte v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_acc8:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v4, v[0:1]
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_mov_b32_e32 v3, s1
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_mov_b32_e32 v2, s0
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_ubyte v5, v[2:3]
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_lshrrev_b32_e32 v7, 8, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v1, 16, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v9, 24, v4
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 16, v0
; GFX8-NEXT: v_lshrrev_b32_e32 v8, 8, v0
; GFX8-NEXT: v_lshrrev_b32_e32 v10, 24, v0
; GFX8-NEXT: v_mad_u16 v0, v4, v0, v5
; GFX8-NEXT: v_mad_u16 v0, v7, v8, v0
; GFX8-NEXT: v_mad_u16 v0, v1, v6, v0
; GFX8-NEXT: v_mad_u16 v0, v9, v10, v0
; GFX8-NEXT: flat_store_byte v[2:3], v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_acc8:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-NODL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v0, 16, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v6, 8, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v8, 24, v2
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v7, 8, v3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_legacy_u16 v2, v2, v3, v4
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v5, 16, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v2, v6, v7, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v9, 24, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v0, v5, v2
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v8, v9, v0
; GFX9-NODL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_acc8:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-DL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v0, 16, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v6, 8, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v8, 24, v2
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v7, 8, v3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_legacy_u16 v2, v2, v3, v4
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v5, 16, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v2, v6, v7, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v9, 24, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v0, v5, v2
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v8, v9, v0
; GFX9-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_acc8:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX10-DL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v11, 8, v2
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v3
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
; GFX10-DL-NEXT: v_mad_u16 v4, v2, v3, v4
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v6, 16, v2
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v7, 16, v3
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX10-DL-NEXT: v_mad_u16 v0, v11, v5, v4
; GFX10-DL-NEXT: v_mad_u16 v0, v6, v7, v0
; GFX10-DL-NEXT: v_mad_u16 v0, v2, v3, v0
; GFX10-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
<4 x i8> addrspace(1)* %src2,
i8 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%mul1 = mul nuw nsw i8 %v1e0, %v2e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%mul2 = mul nuw nsw i8 %v1e1, %v2e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%mul3 = mul nuw nsw i8 %v1e2, %v2e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%mul4 = mul nuw nsw i8 %v1e3, %v2e3
%acc = load i8, i8 addrspace(1)* %dst, align 2
%mad1 = add i8 %mul1, %acc
%mad2 = add i8 %mad1, %mul2
%mad3 = add i8 %mad2, %mul3
%mad4 = add i8 %mad3, %mul4
store i8 %mad4, i8 addrspace(1)* %dst, align 2
ret void
}
; TODO: Generate udot4?
define amdgpu_kernel void @udot2_8(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot2_8:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: buffer_load_ubyte v4, off, s[0:3], 0
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: v_bfe_u32 v2, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v3, s4, v0
; GFX7-NEXT: v_bfe_u32 v0, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v1, v1, v3, v4
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_byte v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot2_8:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v4, v[0:1]
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_mov_b32_e32 v3, s1
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_mov_b32_e32 v2, s0
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_ubyte v5, v[2:3]
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_lshrrev_b32_e32 v1, 8, v4
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 8, v0
; GFX8-NEXT: v_mad_u16 v0, v4, v0, v5
; GFX8-NEXT: v_mad_u16 v0, v1, v6, v0
; GFX8-NEXT: flat_store_byte v[2:3], v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot2_8:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-NODL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v0, 8, v2
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v5, 8, v3
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_legacy_u16 v2, v2, v3, v4
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v0, v5, v2
; GFX9-NODL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot2_8:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-DL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v0, 8, v2
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v3
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_legacy_u16 v2, v2, v3, v4
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v0, v5, v2
; GFX9-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot2_8:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX10-DL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v0, 8, v2
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v3
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_mad_u16 v2, v2, v3, v4
; GFX10-DL-NEXT: v_mad_u16 v0, v0, v5, v2
; GFX10-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
<4 x i8> addrspace(1)* %src2,
i8 addrspace(1)* nocapture %dst) {
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%mul1 = mul nuw nsw i8 %v1e0, %v2e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%mul2 = mul nuw nsw i8 %v1e1, %v2e1
%acc = load i8, i8 addrspace(1)* %dst, align 2
%mad1 = add i8 %mul1, %acc
%mad2 = add i8 %mad1, %mul2
store i8 %mad2, i8 addrspace(1)* %dst, align 2
ret void
}
define amdgpu_kernel void @udot4_CommutationInsideMAD(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot4_CommutationInsideMAD:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_ubyte v8, off, s[0:3], 0
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v1, v5, v1, v8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v1, v6, v3, v1
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v7, v4, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v0, v2, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_byte v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_CommutationInsideMAD:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v4, v[0:1]
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: v_mov_b32_e32 v3, s1
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_mov_b32_e32 v2, s0
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_ubyte v5, v[2:3]
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_lshrrev_b32_e32 v7, 8, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v1, 16, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v9, 24, v4
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 16, v0
; GFX8-NEXT: v_lshrrev_b32_e32 v8, 8, v0
; GFX8-NEXT: v_lshrrev_b32_e32 v10, 24, v0
; GFX8-NEXT: v_mad_u16 v0, v0, v4, v5
; GFX8-NEXT: v_mad_u16 v0, v8, v7, v0
; GFX8-NEXT: v_mad_u16 v0, v6, v1, v0
; GFX8-NEXT: v_mad_u16 v0, v10, v9, v0
; GFX8-NEXT: flat_store_byte v[2:3], v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_CommutationInsideMAD:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-NODL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v0, 16, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v6, 8, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v8, 24, v2
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v7, 8, v3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_legacy_u16 v2, v3, v2, v4
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v5, 16, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v2, v7, v6, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v9, 24, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v5, v0, v2
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v9, v8, v0
; GFX9-NODL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_CommutationInsideMAD:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-DL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v0, 16, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v6, 8, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v8, 24, v2
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v7, 8, v3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_legacy_u16 v2, v3, v2, v4
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v5, 16, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v2, v7, v6, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v9, 24, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v5, v0, v2
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v9, v8, v0
; GFX9-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_CommutationInsideMAD:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX10-DL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v11, 8, v2
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v3
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
; GFX10-DL-NEXT: v_mad_u16 v4, v3, v2, v4
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v6, 16, v2
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v7, 16, v3
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX10-DL-NEXT: v_mad_u16 v0, v5, v11, v4
; GFX10-DL-NEXT: v_mad_u16 v0, v7, v6, v0
; GFX10-DL-NEXT: v_mad_u16 v0, v3, v2, v0
; GFX10-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
<4 x i8> addrspace(1)* %src2,
i8 addrspace(1)* nocapture %dst) {
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%mul1 = mul nuw nsw i8 %v2e0, %v1e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%mul2 = mul nuw nsw i8 %v2e1, %v1e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%mul3 = mul nuw nsw i8 %v2e2, %v1e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%mul4 = mul nuw nsw i8 %v2e3, %v1e3
%acc = load i8, i8 addrspace(1)* %dst, align 2
%mad1 = add i8 %acc, %mul1
%mad2 = add i8 %mul2, %mad1
%mad3 = add i8 %mul3, %mad2
%mad4 = add i8 %mul4, %mad3
store i8 %mad4, i8 addrspace(1)* %dst, align 2
ret void
}
; TODO: Support commutation accross the adds.
define amdgpu_kernel void @udot4_CommutationAccrossMADs(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot4_CommutationAccrossMADs:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_ubyte v8, off, s[0:3], 0
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v3, v6, v3, v8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v1, v5, v1, v3
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v7, v4, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v0, v2, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_byte v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_CommutationAccrossMADs:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v4, v[0:1]
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: v_mov_b32_e32 v3, s1
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: v_mov_b32_e32 v2, s0
; GFX8-NEXT: flat_load_ubyte v10, v[2:3]
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 8, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v1, 16, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v8, 24, v4
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_lshrrev_b32_e32 v7, 8, v0
; GFX8-NEXT: v_lshrrev_b32_e32 v5, 16, v0
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_mad_u16 v6, v7, v6, v10
; GFX8-NEXT: v_lshrrev_b32_e32 v9, 24, v0
; GFX8-NEXT: v_mad_u16 v0, v0, v4, v6
; GFX8-NEXT: v_mad_u16 v0, v5, v1, v0
; GFX8-NEXT: v_mad_u16 v0, v9, v8, v0
; GFX8-NEXT: flat_store_byte v[2:3], v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_CommutationAccrossMADs:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-NODL-NEXT: global_load_ubyte v9, v1, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v5, 8, v2
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v6, 8, v3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_legacy_u16 v5, v6, v5, v9
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v0, 16, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v7, 24, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v4, 16, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v2, v3, v2, v5
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v8, 24, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v4, v0, v2
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v8, v7, v0
; GFX9-NODL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_CommutationAccrossMADs:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-DL-NEXT: global_load_ubyte v9, v1, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v2
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v6, 8, v3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_legacy_u16 v5, v6, v5, v9
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v0, 16, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v7, 24, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v4, 16, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v2, v3, v2, v5
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v8, 24, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v4, v0, v2
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v8, v7, v0
; GFX9-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_CommutationAccrossMADs:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: v_mov_b32_e32 v1, 0
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX10-DL-NEXT: global_load_ubyte v4, v1, s[2:3]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v7, 8, v2
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v3
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_mad_u16 v0, v5, v7, v4
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v7, 16, v2
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 16, v3
; GFX10-DL-NEXT: v_mad_u16 v0, v3, v2, v0
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX10-DL-NEXT: v_mad_u16 v0, v5, v7, v0
; GFX10-DL-NEXT: v_mad_u16 v0, v3, v2, v0
; GFX10-DL-NEXT: global_store_byte v1, v0, s[2:3]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
<4 x i8> addrspace(1)* %src2,
i8 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%mul1 = mul nuw nsw i8 %v2e0, %v1e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%mul2 = mul nuw nsw i8 %v2e1, %v1e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%mul3 = mul nuw nsw i8 %v2e2, %v1e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%mul4 = mul nuw nsw i8 %v2e3, %v1e3
%acc = load i8, i8 addrspace(1)* %dst, align 2
%mad1 = add i8 %acc, %mul2
%mad2 = add i8 %mad1, %mul1
%mad3 = add i8 %mad2, %mul3
%mad4 = add i8 %mad3, %mul4
store i8 %mad4, i8 addrspace(1)* %dst, align 2
ret void
}
define amdgpu_kernel void @udot4_multiuse_mul1(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot4_multiuse_mul1:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
; GFX7-NEXT: s_load_dword s4, s[0:1], 0x0
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v8, v1, v5, s4
; GFX7-NEXT: v_mad_u32_u24 v3, v3, v6, v8
; GFX7-NEXT: v_mad_u32_u24 v1, v1, v5, v3
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v4, v7, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_multiuse_mul1:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_movk_i32 s2, 0xff
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v3, v[0:1]
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: s_load_dword s3, s[0:1], 0x0
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_and_b32_e32 v1, s2, v3
; GFX8-NEXT: v_bfe_u32 v4, v3, 8, 8
; GFX8-NEXT: v_bfe_u32 v6, v3, 16, 8
; GFX8-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_and_b32_e32 v2, s2, v0
; GFX8-NEXT: v_bfe_u32 v5, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mad_u32_u24 v8, v1, v2, s3
; GFX8-NEXT: v_mad_u32_u24 v4, v4, v5, v8
; GFX8-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX8-NEXT: v_mad_u32_u24 v1, v1, v2, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX8-NEXT: v_mad_u32_u24 v1, v6, v7, v1
; GFX8-NEXT: v_mad_u32_u24 v2, v3, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: v_mov_b32_e32 v0, s0
; GFX8-NEXT: v_mov_b32_e32 v1, s1
; GFX8-NEXT: flat_store_dword v[0:1], v2
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_multiuse_mul1:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-NODL-NEXT: s_load_dword s1, s[2:3], 0x0
; GFX9-NODL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_and_b32_e32 v3, s0, v1
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
; GFX9-NODL-NEXT: v_and_b32_e32 v4, s0, v2
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v5, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v6, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX9-NODL-NEXT: v_mul_u32_u24_e32 v2, v3, v4
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_u32_u24 v3, v3, v4, s1
; GFX9-NODL-NEXT: v_add3_u32 v2, v5, v3, v2
; GFX9-NODL-NEXT: v_add3_u32 v1, v2, v6, v1
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: global_store_dword v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_multiuse_mul1:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-DL-NEXT: s_load_dword s1, s[2:3], 0x0
; GFX9-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_and_b32_e32 v3, s0, v1
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
; GFX9-DL-NEXT: v_and_b32_e32 v4, s0, v2
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v5, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v6, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX9-DL-NEXT: v_mul_u32_u24_e32 v2, v3, v4
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_u32_u24 v3, v3, v4, s1
; GFX9-DL-NEXT: v_add3_u32 v2, v5, v3, v2
; GFX9-DL-NEXT: v_add3_u32 v1, v2, v6, v1
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: global_store_dword v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_multiuse_mul1:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
; GFX10-DL-NEXT: s_movk_i32 s3, 0xff
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: s_load_dword s2, s[0:1], 0x0
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_and_b32_e32 v0, s3, v1
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
; GFX10-DL-NEXT: v_and_b32_e32 v3, s3, v2
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v7, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX10-DL-NEXT: v_mul_u32_u24_e32 v5, v0, v3
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_mad_u32_u24 v0, v0, v3, s2
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX10-DL-NEXT: v_mov_b32_e32 v2, 0
; GFX10-DL-NEXT: v_add3_u32 v0, v7, v0, v5
; GFX10-DL-NEXT: v_add3_u32 v0, v0, v3, v1
; GFX10-DL-NEXT: global_store_dword v2, v0, s[0:1]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
<4 x i8> addrspace(1)* %src2,
i32 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%cv1e0 = zext i8 %v1e0 to i32
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%cv2e0 = zext i8 %v2e0 to i32
%mul1 = mul nuw nsw i32 %cv1e0, %cv2e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%cv1e1 = zext i8 %v1e1 to i32
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%cv2e1 = zext i8 %v2e1 to i32
%mul2 = mul nuw nsw i32 %cv1e1, %cv2e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%cv1e2 = zext i8 %v1e2 to i32
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%cv2e2 = zext i8 %v2e2 to i32
%mul3 = mul nuw nsw i32 %cv1e2, %cv2e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%cv1e3 = zext i8 %v1e3 to i32
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%cv2e3 = zext i8 %v2e3 to i32
%mul4 = mul nuw nsw i32 %cv1e3, %cv2e3
%acc = load i32, i32 addrspace(1)* %dst, align 4
%add = add i32 %mul1, %acc
%add1 = add i32 %mul2, %add
%add2 = add i32 %add1, %mul1
%add3 = add i32 %add2, %mul3
%add4 = add i32 %add3, %mul4
store i32 %add4, i32 addrspace(1)* %dst, align 4
ret void
}
define amdgpu_kernel void @udot4_multiuse_add1(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: udot4_multiuse_add1:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_load_dword s5, s[0:1], 0x0
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v3, v3, v6, s5
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v1, v1, v5, v3
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v4, v7, v1
; GFX7-NEXT: v_add_i32_e32 v6, vcc, s5, v3
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
; GFX7-NEXT: v_add_i32_e32 v0, vcc, v0, v6
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_multiuse_add1:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_movk_i32 s2, 0xff
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v3, v[0:1]
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: s_load_dword s3, s[0:1], 0x0
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_bfe_u32 v4, v3, 8, 8
; GFX8-NEXT: v_and_b32_e32 v1, s2, v3
; GFX8-NEXT: v_bfe_u32 v6, v3, 16, 8
; GFX8-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_bfe_u32 v5, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mad_u32_u24 v4, v4, v5, s3
; GFX8-NEXT: v_and_b32_e32 v2, s2, v0
; GFX8-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX8-NEXT: v_mad_u32_u24 v1, v1, v2, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX8-NEXT: v_mad_u32_u24 v1, v6, v7, v1
; GFX8-NEXT: v_add_u32_e32 v5, vcc, s3, v4
; GFX8-NEXT: v_mad_u32_u24 v0, v3, v0, v1
; GFX8-NEXT: v_add_u32_e32 v2, vcc, v0, v5
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: v_mov_b32_e32 v0, s0
; GFX8-NEXT: v_mov_b32_e32 v1, s1
; GFX8-NEXT: flat_store_dword v[0:1], v2
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_multiuse_add1:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-NODL-NEXT: s_load_dword s0, s[2:3], 0x0
; GFX9-NODL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_bfe_u32 v4, v1, 8, 8
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
; GFX9-NODL-NEXT: v_bfe_u32 v5, v2, 8, 8
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:BYTE_0
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v6, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_u32_u24 v2, v4, v5, s0
; GFX9-NODL-NEXT: v_add_u32_e32 v4, s0, v2
; GFX9-NODL-NEXT: v_add3_u32 v2, v2, v3, v6
; GFX9-NODL-NEXT: v_add3_u32 v1, v2, v1, v4
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: global_store_dword v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_multiuse_add1:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-DL-NEXT: s_load_dword s0, s[2:3], 0x0
; GFX9-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_bfe_u32 v4, v1, 8, 8
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
; GFX9-DL-NEXT: v_bfe_u32 v5, v2, 8, 8
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:BYTE_0
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v6, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_u32_u24 v2, v4, v5, s0
; GFX9-DL-NEXT: v_add_u32_e32 v4, s0, v2
; GFX9-DL-NEXT: v_add3_u32 v2, v2, v3, v6
; GFX9-DL-NEXT: v_add3_u32 v1, v2, v1, v4
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: global_store_dword v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_multiuse_add1:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: s_load_dword s2, s[0:1], 0x0
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_bfe_u32 v0, v1, 8, 8
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
; GFX10-DL-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v4, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:BYTE_0
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_mad_u32_u24 v6, v0, v3, s2
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX10-DL-NEXT: v_add_nc_u32_e32 v2, s2, v6
; GFX10-DL-NEXT: v_add3_u32 v0, v6, v4, v3
; GFX10-DL-NEXT: v_mov_b32_e32 v3, 0
; GFX10-DL-NEXT: v_add3_u32 v0, v0, v1, v2
; GFX10-DL-NEXT: global_store_dword v3, v0, s[0:1]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
<4 x i8> addrspace(1)* %src2,
i32 addrspace(1)* nocapture %dst) {
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%cv1e0 = zext i8 %v1e0 to i32
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%cv2e0 = zext i8 %v2e0 to i32
%mul1 = mul nuw nsw i32 %cv1e0, %cv2e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%cv1e1 = zext i8 %v1e1 to i32
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%cv2e1 = zext i8 %v2e1 to i32
%mul2 = mul nuw nsw i32 %cv1e1, %cv2e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%cv1e2 = zext i8 %v1e2 to i32
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%cv2e2 = zext i8 %v2e2 to i32
%mul3 = mul nuw nsw i32 %cv1e2, %cv2e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%cv1e3 = zext i8 %v1e3 to i32
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%cv2e3 = zext i8 %v2e3 to i32
%mul4 = mul nuw nsw i32 %cv1e3, %cv2e3
%acc = load i32, i32 addrspace(1)* %dst, align 4
%add1 = add i32 %mul2, %acc
%add = add i32 %add1, %acc
%add2 = add i32 %add1, %mul1
%add3 = add i32 %add2, %mul3
%add4 = add i32 %add3, %mul4
%res = add i32 %add4, %add
store i32 %res, i32 addrspace(1)* %dst, align 4
ret void
}
define amdgpu_kernel void @notdot4_mixedtypes(<4 x i8> addrspace(1)* %src1,
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-LABEL: notdot4_mixedtypes:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: buffer_load_ushort v1, off, s[0:3], 0
; GFX7-NEXT: s_mov_b32 s4, 0xffff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_bfe_i32 v3, v2, 0, 8
; GFX7-NEXT: v_bfe_u32 v4, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_bfe_i32 v6, v0, 0, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 8, 8
; GFX7-NEXT: v_and_b32_e32 v3, s4, v3
; GFX7-NEXT: v_and_b32_e32 v6, s4, v6
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v1, v4, v7, v1
; GFX7-NEXT: v_bfe_u32 v5, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v8, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v1, v3, v6, v1
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v5, v8, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: notdot4_mixedtypes:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; GFX8-NEXT: s_movk_i32 s2, 0xff
; GFX8-NEXT: v_mov_b32_e32 v5, s2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v4, v[0:1]
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v0, v[0:1]
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics tryLatency compares two sched candidates. For the top zone it prefers the one with lesser depth, but only if that depth is greater than the total latency of the instructions we've already scheduled -- otherwise its latency would be hidden and there would be no stall. Unfortunately it only tests the depth of one of the candidates. This can lead to situations where the TopDepthReduce heuristic does not kick in, but a lower priority heuristic chooses the other candidate, whose depth *is* greater than the already scheduled latency, which causes a stall. The fix is to apply the heuristic if the depth of *either* candidate is greater than the already scheduled latency. All this also applies to the BotHeightReduce heuristic in the bottom zone. Differential Revision: https://reviews.llvm.org/D72392
2020-01-07 23:43:46 +08:00
; GFX8-NEXT: v_mov_b32_e32 v3, s1
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v2, s0
; GFX8-NEXT: flat_load_ushort v10, v[2:3]
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_lshrrev_b32_e32 v7, 8, v4
; GFX8-NEXT: v_and_b32_e32 v7, s2, v7
; GFX8-NEXT: v_bfe_i32 v1, v4, 0, 8
; GFX8-NEXT: v_and_b32_sdwa v9, v4, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_lshrrev_b32_e32 v8, 8, v0
; GFX8-NEXT: v_and_b32_e32 v8, s2, v8
; GFX8-NEXT: v_bfe_i32 v6, v0, 0, 8
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_mad_u16 v7, v7, v8, v10
; GFX8-NEXT: v_and_b32_sdwa v5, v0, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX8-NEXT: v_mad_u16 v1, v1, v6, v7
; GFX8-NEXT: v_lshrrev_b32_e32 v4, 24, v4
; GFX8-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX8-NEXT: v_mad_u16 v1, v9, v5, v1
; GFX8-NEXT: v_mad_u16 v0, v4, v0, v1
; GFX8-NEXT: flat_store_short v[2:3], v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: notdot4_mixedtypes:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: v_mov_b32_e32 v1, 0
; GFX9-NODL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-NODL-NEXT: global_load_ushort v9, v1, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v5, 8, v2
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v6, 8, v3
; GFX9-NODL-NEXT: v_and_b32_e32 v5, s0, v5
; GFX9-NODL-NEXT: v_and_b32_e32 v6, s0, v6
; GFX9-NODL-NEXT: v_bfe_i32 v0, v2, 0, 8
; GFX9-NODL-NEXT: v_bfe_i32 v4, v3, 0, 8
[DAGCombine] Prune unnused nodes. Summary: Nodes that have no uses are eventually pruned when they are selected from the worklist. Record nodes newly added to the worklist or DAG and perform pruning after every combine attempt. Reviewers: efriedma, RKSimon, craig.topper, spatel, jyknight Reviewed By: jyknight Subscribers: jdoerfert, jyknight, nemanjai, jvesely, nhaehnle, javed.absar, hiraditya, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58070 llvm-svn: 357283
2019-03-30 01:35:56 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_legacy_u16 v5, v5, v6, v9
; GFX9-NODL-NEXT: v_and_b32_sdwa v7, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-NODL-NEXT: v_and_b32_sdwa v8, v3, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v0, v4, v5
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v7, v8, v0
; GFX9-NODL-NEXT: v_mad_legacy_u16 v0, v2, v3, v0
; GFX9-NODL-NEXT: global_store_short v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: notdot4_mixedtypes:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: v_mov_b32_e32 v1, 0
; GFX9-DL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v2, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v3, v0, s[6:7]
; GFX9-DL-NEXT: global_load_ushort v9, v1, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v2
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v6, 8, v3
; GFX9-DL-NEXT: v_and_b32_e32 v5, s0, v5
; GFX9-DL-NEXT: v_and_b32_e32 v6, s0, v6
; GFX9-DL-NEXT: v_bfe_i32 v0, v2, 0, 8
; GFX9-DL-NEXT: v_bfe_i32 v4, v3, 0, 8
[DAGCombine] Prune unnused nodes. Summary: Nodes that have no uses are eventually pruned when they are selected from the worklist. Record nodes newly added to the worklist or DAG and perform pruning after every combine attempt. Reviewers: efriedma, RKSimon, craig.topper, spatel, jyknight Reviewed By: jyknight Subscribers: jdoerfert, jyknight, nemanjai, jvesely, nhaehnle, javed.absar, hiraditya, jsji, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D58070 llvm-svn: 357283
2019-03-30 01:35:56 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_legacy_u16 v5, v5, v6, v9
; GFX9-DL-NEXT: v_and_b32_sdwa v7, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-DL-NEXT: v_and_b32_sdwa v8, v3, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v0, v4, v5
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v3, 24, v3
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v7, v8, v0
; GFX9-DL-NEXT: v_mad_legacy_u16 v0, v2, v3, v0
; GFX9-DL-NEXT: global_store_short v1, v0, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: notdot4_mixedtypes:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_movk_i32 s0, 0xff
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX10-DL-NEXT: global_load_ushort v3, v0, s[2:3]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v4, 8, v1
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v2
; GFX10-DL-NEXT: v_bfe_i32 v6, v1, 0, 8
; GFX10-DL-NEXT: v_bfe_i32 v9, v2, 0, 8
; GFX10-DL-NEXT: v_and_b32_e32 v4, s0, v4
; GFX10-DL-NEXT: v_and_b32_e32 v5, s0, v5
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_mad_u16 v3, v4, v5, v3
; GFX10-DL-NEXT: v_and_b32_sdwa v4, v1, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX10-DL-NEXT: v_and_b32_sdwa v5, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v1, 24, v1
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX10-DL-NEXT: v_mad_u16 v3, v6, v9, v3
; GFX10-DL-NEXT: v_mad_u16 v3, v4, v5, v3
; GFX10-DL-NEXT: v_mad_u16 v1, v1, v2, v3
; GFX10-DL-NEXT: global_store_short v0, v1, s[2:3]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
<4 x i8> addrspace(1)* %src2,
i16 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match udot4 pattern. Summary: D.u32 = S0.u8[0] * S1.u8[0] + S0.u8[1] * S1.u8[1] + S0.u8[2] * S1.u8[2] + S0.u8[3] * S1.u8[3] + S2.u32 Author: FarhanaAleen Reviewed By: arsenm Subscribers: llvm-commits, AMDGPU Differential Revision: https://reviews.llvm.org/D50921 llvm-svn: 340936
2018-08-30 00:31:18 +08:00
%v1e0 = extractelement <4 x i8> %vec1, i64 0
%cv1e0 = sext i8 %v1e0 to i16
%v2e0 = extractelement <4 x i8> %vec2, i64 0
%cv2e0 = sext i8 %v2e0 to i16
%mul1 = mul nuw nsw i16 %cv1e0, %cv2e0
%v1e1 = extractelement <4 x i8> %vec1, i64 1
%cv1e1 = zext i8 %v1e1 to i16
%v2e1 = extractelement <4 x i8> %vec2, i64 1
%cv2e1 = zext i8 %v2e1 to i16
%mul2 = mul nuw nsw i16 %cv1e1, %cv2e1
%v1e2 = extractelement <4 x i8> %vec1, i64 2
%cv1e2 = zext i8 %v1e2 to i16
%v2e2 = extractelement <4 x i8> %vec2, i64 2
%cv2e2 = zext i8 %v2e2 to i16
%mul3 = mul nuw nsw i16 %cv1e2, %cv2e2
%v1e3 = extractelement <4 x i8> %vec1, i64 3
%cv1e3 = zext i8 %v1e3 to i16
%v2e3 = extractelement <4 x i8> %vec2, i64 3
%cv2e3 = zext i8 %v2e3 to i16
%mul4 = mul nuw nsw i16 %cv1e3, %cv2e3
%acc = load i16, i16 addrspace(1)* %dst, align 2
%add1 = add i16 %mul2, %acc
%add2 = add i16 %add1, %mul1
%add3 = add i16 %add2, %mul3
%add4 = add i16 %add3, %mul4
store i16 %add4, i16 addrspace(1)* %dst, align 2
ret void
}
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; TODO: cleanup s_lshr_b32 and support this pattern.
define amdgpu_kernel void @udot4_acc32_vecMul(<4 x i8> addrspace(1)* %src1,
; GFX7-LABEL: udot4_acc32_vecMul:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_lshrrev_b32_e32 v1, 24, v2
; GFX7-NEXT: v_bfe_u32 v3, v2, 8, 8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_and_b32_e32 v2, s4, v2
; GFX7-NEXT: s_waitcnt vmcnt(0)
; GFX7-NEXT: v_lshrrev_b32_e32 v5, 24, v0
; GFX7-NEXT: v_bfe_u32 v6, v0, 8, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_and_b32_e32 v0, s4, v0
; GFX7-NEXT: s_load_dword s4, s[0:1], 0x0
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, s4
; GFX7-NEXT: v_mad_u32_u24 v0, v3, v6, v0
; GFX7-NEXT: v_mad_u32_u24 v0, v4, v7, v0
; GFX7-NEXT: v_mad_u32_u24 v0, v1, v5, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_acc32_vecMul:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_movk_i32 s2, 0xff
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v3, v[0:1]
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: s_load_dword s3, s[0:1], 0x0
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_lshrrev_b32_e32 v1, 24, v3
; GFX8-NEXT: v_bfe_u32 v4, v3, 16, 8
; GFX8-NEXT: v_lshrrev_b16_e32 v5, 8, v3
; GFX8-NEXT: v_and_b32_e32 v3, s2, v3
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: v_lshrrev_b32_e32 v2, 24, v0
; GFX8-NEXT: v_bfe_u32 v6, v0, 16, 8
; GFX8-NEXT: v_lshrrev_b16_e32 v7, 8, v0
; GFX8-NEXT: v_and_b32_e32 v0, s2, v0
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mad_u32_u24 v0, v3, v0, s3
; GFX8-NEXT: v_mad_u32_u24 v0, v5, v7, v0
; GFX8-NEXT: v_mad_u32_u24 v0, v4, v6, v0
; GFX8-NEXT: v_mad_u32_u24 v2, v1, v2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: v_mov_b32_e32 v0, s0
; GFX8-NEXT: v_mov_b32_e32 v1, s1
; GFX8-NEXT: flat_store_dword v[0:1], v2
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_acc32_vecMul:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-NODL-NEXT: s_load_dword s0, s[2:3], 0x0
; GFX9-NODL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:BYTE_0
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v4, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v5, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX9-NODL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_add3_u32 v2, v3, s0, v4
; GFX9-NODL-NEXT: v_add3_u32 v1, v2, v5, v1
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: global_store_dword v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_acc32_vecMul:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-DL-NEXT: s_load_dword s0, s[2:3], 0x0
; GFX9-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:BYTE_0
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v4, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v5, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX9-DL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_add3_u32 v2, v3, s0, v4
; GFX9-DL-NEXT: v_add3_u32 v1, v2, v5, v1
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: global_store_dword v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_acc32_vecMul:
; GFX10-DL: ; %bb.0: ; %entry
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
; GFX10-DL-NEXT: s_mov_b32 s3, 0xffff
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: s_load_dword s2, s[0:1], 0x0
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_and_b32_sdwa v0, s3, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_1
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
; GFX10-DL-NEXT: v_and_b32_sdwa v3, s3, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_1
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v7, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:BYTE_0
; GFX10-DL-NEXT: v_mul_u32_u24_e32 v0, v0, v3
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v3, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_2 src1_sel:BYTE_2
; GFX10-DL-NEXT: v_mul_u32_u24_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX10-DL-NEXT: v_mov_b32_e32 v2, 0
[AMDGPU] Fix typo in SIInstrInfo::memOpsHaveSameBasePtr Summary: The typo has been present since memOpsHaveSameBasePtr was introduced in r313208. It caused SIInstrInfo::shouldClusterMemOps to cluster more mem ops than it was supposed to. Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D71616
2019-12-18 00:09:02 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_add3_u32 v0, v7, s2, v0
; GFX10-DL-NEXT: v_add3_u32 v0, v0, v3, v1
; GFX10-DL-NEXT: global_store_dword v2, v0, s[0:1]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
<4 x i8> addrspace(1)* %src2,
i32 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
%cvec1 = zext <4 x i8> %vec1 to <4 x i32>
%cvec2 = zext <4 x i8> %vec2 to <4 x i32>
%mul = mul <4 x i32> %cvec1, %cvec2
%mul0 = extractelement <4 x i32> %mul, i64 0
%mul1 = extractelement <4 x i32> %mul, i64 1
%mul2 = extractelement <4 x i32> %mul, i64 2
%mul3 = extractelement <4 x i32> %mul, i64 3
%acc = load i32, i32 addrspace(1)* %dst, align 4
%add1 = add i32 %mul0, %acc
%add2 = add i32 %add1, %mul1
%add3 = add i32 %add2, %mul2
%add4 = add i32 %add3, %mul3
store i32 %add4, i32 addrspace(1)* %dst, align 4
ret void
}
; TODO: This pattern should be recognized.
define amdgpu_kernel void @udot4_acc16_vecMul(<4 x i8> addrspace(1)* %src1,
; GFX7-LABEL: udot4_acc16_vecMul:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_ushort v8, off, s[0:3], 0
; GFX7-NEXT: s_mov_b32 s4, 0xff00
; GFX7-NEXT: s_movk_i32 s5, 0xff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_and_b32_e32 v1, s4, v2
; GFX7-NEXT: v_and_b32_e32 v3, s5, v2
; GFX7-NEXT: v_lshlrev_b32_e32 v1, 8, v1
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v5, s4, v0
; GFX7-NEXT: v_or_b32_e32 v1, v3, v1
; GFX7-NEXT: v_lshlrev_b32_e32 v3, 8, v5
; GFX7-NEXT: v_and_b32_e32 v6, s5, v0
; GFX7-NEXT: v_or_b32_e32 v3, v6, v3
; GFX7-NEXT: v_lshrrev_b32_e32 v5, 16, v1
; GFX7-NEXT: v_lshrrev_b32_e32 v6, 16, v3
; GFX7-NEXT: v_and_b32_e32 v1, s5, v1
; GFX7-NEXT: v_and_b32_e32 v3, s5, v3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v1, v1, v3, v8
; GFX7-NEXT: v_bfe_u32 v4, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v7, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v1, v5, v6, v1
; GFX7-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v0, 24, v0
; GFX7-NEXT: v_mad_u32_u24 v1, v4, v7, v1
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v1
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_acc16_vecMul:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
; GFX8-NEXT: s_movk_i32 s2, 0xff
; GFX8-NEXT: v_mov_b32_e32 v5, s2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v4, v[0:1]
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: v_mov_b32_e32 v3, s1
; GFX8-NEXT: flat_load_dword v0, v[0:1]
; GFX8-NEXT: v_mov_b32_e32 v2, s0
; GFX8-NEXT: flat_load_ushort v10, v[2:3]
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_and_b32_sdwa v9, v4, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX8-NEXT: v_lshrrev_b32_e32 v1, 24, v4
; GFX8-NEXT: v_lshrrev_b16_e32 v6, 8, v4
; GFX8-NEXT: v_and_b32_e32 v4, s2, v4
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_lshrrev_b32_e32 v7, 24, v0
; GFX8-NEXT: v_lshrrev_b16_e32 v8, 8, v0
; GFX8-NEXT: v_and_b32_sdwa v5, v0, v5 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX8-NEXT: v_and_b32_e32 v0, s2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mad_u16 v0, v4, v0, v10
; GFX8-NEXT: v_mad_u16 v0, v6, v8, v0
; GFX8-NEXT: v_mad_u16 v0, v9, v5, v0
; GFX8-NEXT: v_mad_u16 v0, v1, v7, v0
; GFX8-NEXT: flat_store_short v[2:3], v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_acc16_vecMul:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-NODL-NEXT: s_movk_i32 s0, 0xff
; GFX9-NODL-NEXT: v_mov_b32_e32 v4, 0xffff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-NODL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-NODL-NEXT: global_load_ushort v3, v0, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b16_e32 v5, 8, v1
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_and_b32_sdwa v10, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v6, 24, v1
; GFX9-NODL-NEXT: v_and_b32_sdwa v9, v1, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-NODL-NEXT: v_and_b32_sdwa v1, v4, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NODL-NEXT: v_lshrrev_b16_e32 v7, 8, v2
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v8, 24, v2
; GFX9-NODL-NEXT: v_and_b32_sdwa v2, v4, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-NODL-NEXT: v_lshl_or_b32 v2, v7, 16, v2
; GFX9-NODL-NEXT: v_lshl_or_b32 v1, v5, 16, v1
; GFX9-NODL-NEXT: v_and_b32_e32 v10, v4, v10
; GFX9-NODL-NEXT: v_and_b32_e32 v4, v4, v9
AMDGPU: Select global saddr mode from SGPR pointer Use the 64-bit SGPR base with a 0 offset, since it's 1 fewer instruction to materialize the 0 vs. the 64-bit copy.
2020-11-11 00:06:59 +08:00
; GFX9-NODL-NEXT: v_pk_mul_lo_u16 v1, v1, v2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_add_u16_e32 v3, v1, v3
; GFX9-NODL-NEXT: v_lshl_or_b32 v5, v8, 16, v10
; GFX9-NODL-NEXT: v_lshl_or_b32 v4, v6, 16, v4
; GFX9-NODL-NEXT: v_pk_mul_lo_u16 v2, v4, v5
; GFX9-NODL-NEXT: v_add_u16_sdwa v1, v3, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
; GFX9-NODL-NEXT: v_add_u16_e32 v1, v1, v2
; GFX9-NODL-NEXT: v_add_u16_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: global_store_short v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_acc16_vecMul:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX9-DL-NEXT: s_movk_i32 s0, 0xff
; GFX9-DL-NEXT: v_mov_b32_e32 v4, 0xffff
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-DL-NEXT: global_load_ushort v3, v0, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b16_e32 v5, 8, v1
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_and_b32_sdwa v10, v2, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v6, 24, v1
; GFX9-DL-NEXT: v_and_b32_sdwa v9, v1, s0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9-DL-NEXT: v_and_b32_sdwa v1, v4, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-DL-NEXT: v_lshrrev_b16_e32 v7, 8, v2
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v8, 24, v2
; GFX9-DL-NEXT: v_and_b32_sdwa v2, v4, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX9-DL-NEXT: v_lshl_or_b32 v2, v7, 16, v2
; GFX9-DL-NEXT: v_lshl_or_b32 v1, v5, 16, v1
; GFX9-DL-NEXT: v_and_b32_e32 v10, v4, v10
; GFX9-DL-NEXT: v_and_b32_e32 v4, v4, v9
AMDGPU: Select global saddr mode from SGPR pointer Use the 64-bit SGPR base with a 0 offset, since it's 1 fewer instruction to materialize the 0 vs. the 64-bit copy.
2020-11-11 00:06:59 +08:00
; GFX9-DL-NEXT: v_pk_mul_lo_u16 v1, v1, v2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_add_u16_e32 v3, v1, v3
; GFX9-DL-NEXT: v_lshl_or_b32 v5, v8, 16, v10
; GFX9-DL-NEXT: v_lshl_or_b32 v4, v6, 16, v4
; GFX9-DL-NEXT: v_pk_mul_lo_u16 v2, v4, v5
; GFX9-DL-NEXT: v_add_u16_sdwa v1, v3, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
; GFX9-DL-NEXT: v_add_u16_e32 v1, v1, v2
; GFX9-DL-NEXT: v_add_u16_sdwa v1, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: global_store_short v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_acc16_vecMul:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
; GFX10-DL-NEXT: v_mov_b32_e32 v4, 0xffff
; GFX10-DL-NEXT: s_movk_i32 s2, 0xff
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX10-DL-NEXT: global_load_ushort v3, v0, s[0:1]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_lshrrev_b16 v5, 8, v1
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_and_b32_sdwa v8, v4, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_lshrrev_b16 v6, 8, v2
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_and_b32_sdwa v7, v4, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:BYTE_0
; GFX10-DL-NEXT: v_and_b32_sdwa v9, v1, s2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX10-DL-NEXT: v_and_b32_sdwa v10, v2, s2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX10-DL-NEXT: v_lshl_or_b32 v5, v5, 16, v8
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v1, 24, v1
; GFX10-DL-NEXT: v_lshl_or_b32 v6, v6, 16, v7
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v2, 24, v2
; GFX10-DL-NEXT: v_and_b32_e32 v7, v4, v10
; GFX10-DL-NEXT: v_and_b32_e32 v4, v4, v9
; GFX10-DL-NEXT: v_pk_mul_lo_u16 v5, v5, v6
; GFX10-DL-NEXT: v_lshl_or_b32 v2, v2, 16, v7
; GFX10-DL-NEXT: v_lshl_or_b32 v1, v1, 16, v4
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v4, 16, v5
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_add_nc_u16 v3, v5, v3
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_pk_mul_lo_u16 v1, v1, v2
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_add_nc_u16 v2, v3, v4
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v3, 16, v1
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_add_nc_u16 v1, v2, v1
; GFX10-DL-NEXT: v_add_nc_u16 v1, v1, v3
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: global_store_short v0, v1, s[0:1]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
<4 x i8> addrspace(1)* %src2,
i16 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
%cvec1 = zext <4 x i8> %vec1 to <4 x i16>
%cvec2 = zext <4 x i8> %vec2 to <4 x i16>
%mul = mul <4 x i16> %cvec1, %cvec2
%mul0 = extractelement <4 x i16> %mul, i64 0
%mul1 = extractelement <4 x i16> %mul, i64 1
%mul2 = extractelement <4 x i16> %mul, i64 2
%mul3 = extractelement <4 x i16> %mul, i64 3
%acc = load i16, i16 addrspace(1)* %dst, align 4
%add1 = add i16 %mul0, %acc
%add2 = add i16 %add1, %mul1
%add3 = add i16 %add2, %mul2
%add4 = add i16 %add3, %mul3
store i16 %add4, i16 addrspace(1)* %dst, align 4
ret void
}
; TODO: Support this pattern.
define amdgpu_kernel void @udot4_acc8_vecMul(<4 x i8> addrspace(1)* %src1,
; GFX7-LABEL: udot4_acc8_vecMul:
; GFX7: ; %bb.0: ; %entry
; GFX7-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GFX7-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GFX7-NEXT: s_mov_b32 s3, 0xf000
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b32 s10, 0
; GFX7-NEXT: s_mov_b32 s11, s3
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: s_mov_b64 s[8:9], s[4:5]
; GFX7-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX7-NEXT: v_mov_b32_e32 v1, 0
; GFX7-NEXT: buffer_load_dword v2, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b64 s[8:9], s[6:7]
; GFX7-NEXT: buffer_load_dword v0, v[0:1], s[8:11], 0 addr64
; GFX7-NEXT: s_mov_b32 s2, -1
; GFX7-NEXT: buffer_load_ubyte v8, off, s[0:3], 0
; GFX7-NEXT: s_movk_i32 s4, 0xff
; GFX7-NEXT: s_waitcnt vmcnt(2)
; GFX7-NEXT: v_and_b32_e32 v3, s4, v2
; GFX7-NEXT: v_bfe_u32 v4, v2, 8, 8
; GFX7-NEXT: s_waitcnt vmcnt(1)
; GFX7-NEXT: v_and_b32_e32 v6, s4, v0
; GFX7-NEXT: v_bfe_u32 v7, v0, 8, 8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX7-NEXT: v_mad_u32_u24 v3, v3, v6, v8
; GFX7-NEXT: v_lshrrev_b32_e32 v1, 24, v2
; GFX7-NEXT: v_lshrrev_b32_e32 v5, 24, v0
; GFX7-NEXT: v_bfe_u32 v2, v2, 16, 8
; GFX7-NEXT: v_bfe_u32 v0, v0, 16, 8
; GFX7-NEXT: v_mad_u32_u24 v3, v4, v7, v3
; GFX7-NEXT: v_mad_u32_u24 v0, v2, v0, v3
; GFX7-NEXT: v_mad_u32_u24 v0, v1, v5, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX7-NEXT: buffer_store_byte v0, off, s[0:3], 0
; GFX7-NEXT: s_endpgm
;
; GFX8-LABEL: udot4_acc8_vecMul:
; GFX8: ; %bb.0: ; %entry
; GFX8-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX8-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_lshlrev_b32_e32 v2, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mov_b32_e32 v1, s5
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s4, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v3, v[0:1]
; GFX8-NEXT: v_mov_b32_e32 v1, s7
; GFX8-NEXT: v_add_u32_e32 v0, vcc, s6, v2
; GFX8-NEXT: v_addc_u32_e32 v1, vcc, 0, v1, vcc
; GFX8-NEXT: flat_load_dword v2, v[0:1]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: v_mov_b32_e32 v0, s0
; GFX8-NEXT: v_mov_b32_e32 v1, s1
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: flat_load_ubyte v4, v[0:1]
; GFX8-NEXT: s_waitcnt vmcnt(2)
; GFX8-NEXT: v_lshrrev_b32_e32 v5, 16, v3
; GFX8-NEXT: s_waitcnt vmcnt(1)
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 16, v2
; GFX8-NEXT: v_mul_lo_u16_sdwa v7, v3, v2 dst_sel:BYTE_1 dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX8-NEXT: v_mul_lo_u16_e32 v9, v5, v6
; GFX8-NEXT: v_or_b32_sdwa v7, v9, v7 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX8-NEXT: v_mul_lo_u16_sdwa v8, v3, v2 dst_sel:BYTE_1 dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX8-NEXT: v_lshlrev_b32_e32 v9, 16, v7
; GFX8-NEXT: v_or_b32_e32 v8, v8, v9
; GFX8-NEXT: v_lshrrev_b32_e32 v8, 8, v8
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX8-NEXT: v_mad_u16 v2, v3, v2, v4
; GFX8-NEXT: v_add_u16_e32 v2, v2, v8
; GFX8-NEXT: v_lshrrev_b32_e32 v7, 8, v7
; GFX8-NEXT: v_mad_u16 v2, v5, v6, v2
; GFX8-NEXT: v_add_u16_e32 v2, v2, v7
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX8-NEXT: flat_store_byte v[0:1], v2
; GFX8-NEXT: s_endpgm
;
; GFX9-NODL-LABEL: udot4_acc8_vecMul:
; GFX9-NODL: ; %bb.0: ; %entry
; GFX9-NODL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
[AMDGPU] Remove dubious logic in bidirectional list scheduler Summary: pickNodeBidirectional tried to compare the best top candidate and the best bottom candidate by examining TopCand.Reason and BotCand.Reason. This is unsound because, after calling pickNodeFromQueue, Cand.Reason does not reflect the most important reason why Cand was chosen. Rather it reflects the most recent reason why it beat some other potential candidate, which could have been for some low priority tie breaker reason. I have seen this cause problems where TopCand is a good candidate, but because TopCand.Reason is ORDER (which is very low priority) it is repeatedly ignored in favour of a mediocre BotCand. This is not how bidirectional scheduling is supposed to work. To fix this I changed the code to always compare TopCand and BotCand directly, like the generic implementation of pickNodeBidirectional does. This removes some uncommented AMDGPU-specific logic; if this logic turns out to be important then perhaps it could be moved into an override of tryCandidate instead. Graphics shader benchmarking on gfx10 shows a lot more positive than negative effects from this change. Reviewers: arsenm, tstellar, rampitec, kzhuravl, vpykhtin, dstuttard, tpr, atrick, MatzeB Subscribers: jvesely, wdng, nhaehnle, yaxunl, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68338
2019-10-07 22:33:59 +08:00
; GFX9-NODL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-NODL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-NODL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-NODL-NEXT: global_load_ubyte v3, v0, s[2:3]
; GFX9-NODL-NEXT: s_waitcnt vmcnt(2)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v4, 16, v1
; GFX9-NODL-NEXT: s_waitcnt vmcnt(1)
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v5, 16, v2
; GFX9-NODL-NEXT: v_mul_lo_u16_sdwa v6, v1, v2 dst_sel:BYTE_1 dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX9-NODL-NEXT: v_mul_lo_u16_e32 v8, v4, v5
; GFX9-NODL-NEXT: v_or_b32_sdwa v6, v8, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-NODL-NEXT: v_mul_lo_u16_sdwa v7, v1, v2 dst_sel:BYTE_1 dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX9-NODL-NEXT: v_lshlrev_b32_e32 v8, 16, v6
; GFX9-NODL-NEXT: v_or_b32_e32 v7, v7, v8
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v7, 8, v7
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-NODL-NEXT: v_mad_legacy_u16 v1, v1, v2, v3
; GFX9-NODL-NEXT: v_add_u16_e32 v1, v1, v7
; GFX9-NODL-NEXT: v_lshrrev_b32_e32 v6, 8, v6
; GFX9-NODL-NEXT: v_mad_legacy_u16 v1, v4, v5, v1
; GFX9-NODL-NEXT: v_add_u16_e32 v1, v1, v6
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-NODL-NEXT: global_store_byte v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-NODL-NEXT: s_endpgm
;
; GFX9-DL-LABEL: udot4_acc8_vecMul:
; GFX9-DL: ; %bb.0: ; %entry
; GFX9-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x34
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX9-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX9-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX9-DL-NEXT: global_load_ubyte v3, v0, s[2:3]
; GFX9-DL-NEXT: s_waitcnt vmcnt(2)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v4, 16, v1
; GFX9-DL-NEXT: s_waitcnt vmcnt(1)
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v5, 16, v2
; GFX9-DL-NEXT: v_mul_lo_u16_sdwa v6, v1, v2 dst_sel:BYTE_1 dst_unused:UNUSED_PAD src0_sel:BYTE_3 src1_sel:BYTE_3
; GFX9-DL-NEXT: v_mul_lo_u16_e32 v8, v4, v5
; GFX9-DL-NEXT: v_or_b32_sdwa v6, v8, v6 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX9-DL-NEXT: v_mul_lo_u16_sdwa v7, v1, v2 dst_sel:BYTE_1 dst_unused:UNUSED_PAD src0_sel:BYTE_1 src1_sel:BYTE_1
; GFX9-DL-NEXT: v_lshlrev_b32_e32 v8, 16, v6
; GFX9-DL-NEXT: v_or_b32_e32 v7, v7, v8
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v7, 8, v7
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX9-DL-NEXT: v_mad_legacy_u16 v1, v1, v2, v3
; GFX9-DL-NEXT: v_add_u16_e32 v1, v1, v7
; GFX9-DL-NEXT: v_lshrrev_b32_e32 v6, 8, v6
; GFX9-DL-NEXT: v_mad_legacy_u16 v1, v4, v5, v1
; GFX9-DL-NEXT: v_add_u16_e32 v1, v1, v6
[AMDGPU] Update subtarget features for new target ID support Support for XNACK and SRAMECC is not static on some GPUs. We must be able to differentiate between different scenarios for these dynamic subtarget features. The possible settings are: - Unsupported: The GPU has no support for XNACK/SRAMECC. - Any: Preference is unspecified. Use conservative settings that can run anywhere. - Off: Request support for XNACK/SRAMECC Off - On: Request support for XNACK/SRAMECC On GCNSubtarget will track the four options based on the following criteria. If the subtarget does not support XNACK/SRAMECC we say the setting is "Unsupported". If no subtarget features for XNACK/SRAMECC are requested we must support "Any" mode. If the subtarget features XNACK/SRAMECC exist in the feature string when initializing the subtarget, the settings are "On/Off". The defaults are updated to be conservatively correct, meaning if no setting for XNACK or SRAMECC is explicitly requested, defaults will be used which generate code that can be run anywhere. This corresponds to the "Any" setting. Differential Revision: https://reviews.llvm.org/D85882
2020-12-01 01:06:35 +08:00
; GFX9-DL-NEXT: global_store_byte v0, v1, s[2:3]
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
; GFX9-DL-NEXT: s_endpgm
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
;
; GFX10-DL-LABEL: udot4_acc8_vecMul:
; GFX10-DL: ; %bb.0: ; %entry
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x24
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v0, 2, v0
; GFX10-DL-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x34
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt lgkmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: s_clause 0x1
; GFX10-DL-NEXT: global_load_dword v1, v0, s[4:5]
; GFX10-DL-NEXT: global_load_dword v2, v0, s[6:7]
; GFX10-DL-NEXT: v_mov_b32_e32 v0, 0
; GFX10-DL-NEXT: global_load_ubyte v3, v0, s[0:1]
; GFX10-DL-NEXT: s_waitcnt vmcnt(2)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v4, 24, v1
; GFX10-DL-NEXT: s_waitcnt vmcnt(1)
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 24, v2
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v6, 16, v1
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v7, 16, v2
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_lshrrev_b16 v8, 8, v2
; GFX10-DL-NEXT: v_mul_lo_u16 v4, v4, v5
; GFX10-DL-NEXT: v_lshrrev_b16 v5, 8, v1
; GFX10-DL-NEXT: v_mul_lo_u16 v9, v6, v7
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_waitcnt vmcnt(0)
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_mad_u16 v3, v1, v2, v3
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_lshlrev_b16 v4, 8, v4
; GFX10-DL-NEXT: v_mul_lo_u16 v5, v5, v8
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_or_b32_sdwa v4, v9, v4 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_lshlrev_b16 v5, 8, v5
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_lshlrev_b32_e32 v8, 16, v4
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v2, 8, v4
; GFX10-DL-NEXT: v_or_b32_sdwa v5, v5, v8 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX10-DL-NEXT: v_lshrrev_b32_e32 v5, 8, v5
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_add_nc_u16 v1, v3, v5
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: v_mad_u16 v1, v6, v7, v1
[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 suffices Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-04-01 19:21:00 +08:00
; GFX10-DL-NEXT: v_add_nc_u16 v1, v1, v2
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
; GFX10-DL-NEXT: global_store_byte v0, v1, s[0:1]
[AMDGPU] gfx10 tests. NFC. llvm-svn: 363946
2019-06-21 00:29:40 +08:00
; GFX10-DL-NEXT: s_endpgm
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
<4 x i8> addrspace(1)* %src2,
i8 addrspace(1)* nocapture %dst) {
entry:
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
%idx = call i32 @llvm.amdgcn.workitem.id.x()
%gep1 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src1, i32 %idx
%vec1 = load <4 x i8>, <4 x i8> addrspace(1)* %gep1
%gep2 = getelementptr <4 x i8>, <4 x i8> addrspace(1)* %src2, i32 %idx
%vec2 = load <4 x i8>, <4 x i8> addrspace(1)* %gep2
[AMDGPU] Match signed dot4/8 pattern. Summary: This patch matches signed dot4 and dot8 pattern. Author: FarhanaAleen Reviewed By: msearles Differential Revision: https://reviews.llvm.org/D52520 llvm-svn: 343798
2018-10-05 00:57:37 +08:00
%mul = mul <4 x i8> %vec1, %vec2
%mul0 = extractelement <4 x i8> %mul, i64 0
%mul1 = extractelement <4 x i8> %mul, i64 1
%mul2 = extractelement <4 x i8> %mul, i64 2
%mul3 = extractelement <4 x i8> %mul, i64 3
%acc = load i8, i8 addrspace(1)* %dst, align 4
%add1 = add i8 %mul0, %acc
%add2 = add i8 %add1, %mul1
%add3 = add i8 %add2, %mul2
%add4 = add i8 %add3, %mul3
store i8 %add4, i8 addrspace(1)* %dst, align 4
ret void
}
[AMDGPU] Use divergent addresses for vector loads Change some test cases to use divergent addresses for vector loads, which should be the common case in real world code. Using uniform addresses causes poor instruction selection for the surrounding code which has to be fixed up post-register-allocation, and this causes a lot of testsuite churn for a forthcoming patch to stop selecting 24-bit vector multiply instructions for uniform multiplies. This shows up some problems in the idot tests where we fail to select v_dot instructions because the patterns only match MUL_[UI]24 ISD nodes, but the DAG contains i16 mul nodes instead. Differential Revision: https://reviews.llvm.org/D97062
2021-02-19 23:04:03 +08:00
declare i32 @llvm.amdgcn.workitem.id.x()