maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/Thumb2/mve-vld2.ll

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

687 lines
25 KiB
LLVM
Raw Normal View History

[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[ARM] Replace arm vendor with none. NFC
2020-04-22 23:33:11 +08:00
; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp,+fp64 -verify-machineinstrs %s -o - | FileCheck %s
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; i32
define void @vld2_v2i32(<4 x i32> *%src, <2 x i32> *%dst) {
; CHECK-LABEL: vld2_v2i32:
; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vldrw.u32 q0, [r0]
; CHECK-NEXT: vrev64.32 q1, q0
; CHECK-NEXT: vmov r2, s2
; CHECK-NEXT: vmov r0, s6
; CHECK-NEXT: vmov r3, s0
; CHECK-NEXT: add r0, r2
; CHECK-NEXT: vmov r2, s4
; CHECK-NEXT: add r2, r3
; CHECK-NEXT: strd r2, r0, [r1]
; CHECK-NEXT: bx lr
entry:
%l1 = load <4 x i32>, <4 x i32>* %src, align 4
%s1 = shufflevector <4 x i32> %l1, <4 x i32> undef, <2 x i32> <i32 0, i32 2>
%s2 = shufflevector <4 x i32> %l1, <4 x i32> undef, <2 x i32> <i32 1, i32 3>
%a = add <2 x i32> %s1, %s2
store <2 x i32> %a, <2 x i32> *%dst
ret void
}
define void @vld2_v4i32(<8 x i32> *%src, <4 x i32> *%dst) {
; CHECK-LABEL: vld2_v4i32:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.32 {q0, q1}, [r0]
; CHECK-NEXT: vld21.32 {q0, q1}, [r0]
; CHECK-NEXT: vadd.i32 q0, q0, q1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
%l1 = load <8 x i32>, <8 x i32>* %src, align 4
%s1 = shufflevector <8 x i32> %l1, <8 x i32> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x i32> %l1, <8 x i32> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = add <4 x i32> %s1, %s2
store <4 x i32> %a, <4 x i32> *%dst
ret void
}
define void @vld2_v8i32(<16 x i32> *%src, <8 x i32> *%dst) {
; CHECK-LABEL: vld2_v8i32:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.32 {q0, q1}, [r0]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q0, q1}, [r0]!
; CHECK-NEXT: vld20.32 {q2, q3}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.i32 q0, q0, q1
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q2, q3}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vadd.i32 q1, q2, q3
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vstrw.32 q1, [r1, #16]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: bx lr
entry:
%l1 = load <16 x i32>, <16 x i32>* %src, align 4
%s1 = shufflevector <16 x i32> %l1, <16 x i32> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
%s2 = shufflevector <16 x i32> %l1, <16 x i32> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15>
%a = add <8 x i32> %s1, %s2
store <8 x i32> %a, <8 x i32> *%dst
ret void
}
define void @vld2_v16i32(<32 x i32> *%src, <16 x i32> *%dst) {
; CHECK-LABEL: vld2_v16i32:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: .vsave {d8, d9, d10, d11, d12, d13}
; CHECK-NEXT: vpush {d8, d9, d10, d11, d12, d13}
; CHECK-NEXT: vld20.32 {q0, q1}, [r0]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: add.w r2, r0, #96
; CHECK-NEXT: add.w r3, r0, #64
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vld20.32 {q3, q4}, [r2]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q0, q1}, [r0]!
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vld21.32 {q3, q4}, [r2]
; CHECK-NEXT: vld20.32 {q5, q6}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.i32 q0, q0, q1
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld20.32 {q1, q2}, [r3]
[ARM] Create MQQPR and MQQQQPR register classes Similar to the MQPR register class as the MVE equivalent to QPR, this adds MQQPR and MQQQQPR register classes for the MVE equivalents of QQPR and QQQQPR registers. The MVE MQPR seemed have worked out quite well, and adding MQQPR and MQQQQPR allows us to a little more accurately specify the number of registers, calculating register pressure limits a little better. Differential Revision: https://reviews.llvm.org/D107463
2021-08-17 05:58:12 +08:00
; CHECK-NEXT: vadd.i32 q3, q3, q4
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vld21.32 {q5, q6}, [r0]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q1, q2}, [r3]
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vstrw.32 q3, [r1, #48]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.i32 q5, q5, q6
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.i32 q1, q1, q2
; CHECK-NEXT: vstrw.32 q5, [r1, #16]
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vstrw.32 q1, [r1, #32]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vpop {d8, d9, d10, d11, d12, d13}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: bx lr
entry:
%l1 = load <32 x i32>, <32 x i32>* %src, align 4
%s1 = shufflevector <32 x i32> %l1, <32 x i32> undef, <16 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14, i32 16, i32 18, i32 20, i32 22, i32 24, i32 26, i32 28, i32 30>
%s2 = shufflevector <32 x i32> %l1, <32 x i32> undef, <16 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15, i32 17, i32 19, i32 21, i32 23, i32 25, i32 27, i32 29, i32 31>
%a = add <16 x i32> %s1, %s2
store <16 x i32> %a, <16 x i32> *%dst
ret void
}
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
define void @vld2_v4i32_align1(<8 x i32> *%src, <4 x i32> *%dst) {
; CHECK-LABEL: vld2_v4i32_align1:
; CHECK: @ %bb.0: @ %entry
[ARM] Add alignment checks for MVE VLDn The MVE VLD2/4 and VST2/4 instructions require the pointer to be aligned to at least the size of the element type. This adds a check for that into the ARM lowerInterleavedStore and lowerInterleavedLoad functions, not creating the intrinsics if they are invalid for the alignment of the load/store. Unfortunately this is one of those bug fixes that does effect some useful codegen, as we were able to sometimes do some nice lowering of q15 types. But they can cause problem with low aligned pointers. Differential Revision: https://reviews.llvm.org/D95319
2021-01-28 21:10:08 +08:00
; CHECK-NEXT: vldrb.u8 q0, [r0, #16]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vldrb.u8 q1, [r0]
[ARM] Add alignment checks for MVE VLDn The MVE VLD2/4 and VST2/4 instructions require the pointer to be aligned to at least the size of the element type. This adds a check for that into the ARM lowerInterleavedStore and lowerInterleavedLoad functions, not creating the intrinsics if they are invalid for the alignment of the load/store. Unfortunately this is one of those bug fixes that does effect some useful codegen, as we were able to sometimes do some nice lowering of q15 types. But they can cause problem with low aligned pointers. Differential Revision: https://reviews.llvm.org/D95319
2021-01-28 21:10:08 +08:00
; CHECK-NEXT: vmov.f32 s8, s5
; CHECK-NEXT: vmov.f32 s9, s7
; CHECK-NEXT: vmov.f32 s5, s6
; CHECK-NEXT: vmov.f32 s10, s1
; CHECK-NEXT: vmov.f32 s11, s3
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov.f32 s6, s0
[ARM] Add alignment checks for MVE VLDn The MVE VLD2/4 and VST2/4 instructions require the pointer to be aligned to at least the size of the element type. This adds a check for that into the ARM lowerInterleavedStore and lowerInterleavedLoad functions, not creating the intrinsics if they are invalid for the alignment of the load/store. Unfortunately this is one of those bug fixes that does effect some useful codegen, as we were able to sometimes do some nice lowering of q15 types. But they can cause problem with low aligned pointers. Differential Revision: https://reviews.llvm.org/D95319
2021-01-28 21:10:08 +08:00
; CHECK-NEXT: vmov.f32 s7, s2
; CHECK-NEXT: vadd.i32 q0, q1, q2
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
%l1 = load <8 x i32>, <8 x i32>* %src, align 1
%s1 = shufflevector <8 x i32> %l1, <8 x i32> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x i32> %l1, <8 x i32> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = add <4 x i32> %s1, %s2
store <4 x i32> %a, <4 x i32> *%dst
ret void
}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; i16
define void @vld2_v2i16(<4 x i16> *%src, <2 x i16> *%dst) {
; CHECK-LABEL: vld2_v2i16:
; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vldrh.u32 q0, [r0]
; CHECK-NEXT: vrev64.32 q1, q0
; CHECK-NEXT: vmov r2, s2
; CHECK-NEXT: vmov r0, s6
; CHECK-NEXT: add r0, r2
; CHECK-NEXT: strh r0, [r1, #2]
; CHECK-NEXT: vmov r0, s4
; CHECK-NEXT: vmov r2, s0
; CHECK-NEXT: add r0, r2
; CHECK-NEXT: strh r0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <4 x i16>, <4 x i16>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <4 x i16> %l1, <4 x i16> undef, <2 x i32> <i32 0, i32 2>
%s2 = shufflevector <4 x i16> %l1, <4 x i16> undef, <2 x i32> <i32 1, i32 3>
%a = add <2 x i16> %s1, %s2
store <2 x i16> %a, <2 x i16> *%dst
ret void
}
define void @vld2_v4i16(<8 x i16> *%src, <4 x i16> *%dst) {
; CHECK-LABEL: vld2_v4i16:
; CHECK: @ %bb.0: @ %entry
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: vldrh.u16 q0, [r0]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vrev32.16 q1, q0
; CHECK-NEXT: vadd.i32 q0, q0, q1
; CHECK-NEXT: vstrh.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <8 x i16>, <8 x i16>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <8 x i16> %l1, <8 x i16> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x i16> %l1, <8 x i16> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = add <4 x i16> %s1, %s2
store <4 x i16> %a, <4 x i16> *%dst
ret void
}
define void @vld2_v8i16(<16 x i16> *%src, <8 x i16> *%dst) {
; CHECK-LABEL: vld2_v8i16:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.16 {q0, q1}, [r0]
; CHECK-NEXT: vld21.16 {q0, q1}, [r0]
; CHECK-NEXT: vadd.i16 q0, q0, q1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <16 x i16>, <16 x i16>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <16 x i16> %l1, <16 x i16> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
%s2 = shufflevector <16 x i16> %l1, <16 x i16> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15>
%a = add <8 x i16> %s1, %s2
store <8 x i16> %a, <8 x i16> *%dst
ret void
}
define void @vld2_v16i16(<32 x i16> *%src, <16 x i16> *%dst) {
; CHECK-LABEL: vld2_v16i16:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.16 {q0, q1}, [r0]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.16 {q0, q1}, [r0]!
; CHECK-NEXT: vld20.16 {q2, q3}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.i16 q0, q0, q1
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.16 {q2, q3}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vadd.i16 q1, q2, q3
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vstrw.32 q1, [r1, #16]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <32 x i16>, <32 x i16>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <32 x i16> %l1, <32 x i16> undef, <16 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14, i32 16, i32 18, i32 20, i32 22, i32 24, i32 26, i32 28, i32 30>
%s2 = shufflevector <32 x i16> %l1, <32 x i16> undef, <16 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15, i32 17, i32 19, i32 21, i32 23, i32 25, i32 27, i32 29, i32 31>
%a = add <16 x i16> %s1, %s2
store <16 x i16> %a, <16 x i16> *%dst
ret void
}
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
define void @vld2_v8i16_align1(<16 x i16> *%src, <8 x i16> *%dst) {
; CHECK-LABEL: vld2_v8i16_align1:
; CHECK: @ %bb.0: @ %entry
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vldrb.u8 q0, [r0]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vldrb.u8 q2, [r0, #16]
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s4, s0
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s6, s1
; CHECK-NEXT: vins.f16 s4, s6
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s5, s2
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s6, s3
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s12, s9
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vins.f16 s5, s6
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s6, s8
; CHECK-NEXT: vins.f16 s6, s12
; CHECK-NEXT: vmovx.f16 s7, s10
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s12, s11
; CHECK-NEXT: vins.f16 s2, s3
; CHECK-NEXT: vins.f16 s10, s11
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vins.f16 s8, s9
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vins.f16 s0, s1
; CHECK-NEXT: vmov.f32 s1, s2
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vins.f16 s7, s12
; CHECK-NEXT: vmov.f32 s2, s8
; CHECK-NEXT: vmov.f32 s3, s10
; CHECK-NEXT: vadd.i16 q0, q0, q1
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
%l1 = load <16 x i16>, <16 x i16>* %src, align 1
%s1 = shufflevector <16 x i16> %l1, <16 x i16> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
%s2 = shufflevector <16 x i16> %l1, <16 x i16> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15>
%a = add <8 x i16> %s1, %s2
store <8 x i16> %a, <8 x i16> *%dst
ret void
}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; i8
define void @vld2_v2i8(<4 x i8> *%src, <2 x i8> *%dst) {
; CHECK-LABEL: vld2_v2i8:
; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vldrb.u32 q0, [r0]
; CHECK-NEXT: vrev64.32 q1, q0
; CHECK-NEXT: vmov r2, s2
; CHECK-NEXT: vmov r0, s6
; CHECK-NEXT: add r0, r2
; CHECK-NEXT: strb r0, [r1, #1]
; CHECK-NEXT: vmov r0, s4
; CHECK-NEXT: vmov r2, s0
; CHECK-NEXT: add r0, r2
; CHECK-NEXT: strb r0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <4 x i8>, <4 x i8>* %src, align 1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <4 x i8> %l1, <4 x i8> undef, <2 x i32> <i32 0, i32 2>
%s2 = shufflevector <4 x i8> %l1, <4 x i8> undef, <2 x i32> <i32 1, i32 3>
%a = add <2 x i8> %s1, %s2
store <2 x i8> %a, <2 x i8> *%dst
ret void
}
define void @vld2_v4i8(<8 x i8> *%src, <4 x i8> *%dst) {
; CHECK-LABEL: vld2_v4i8:
; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vldrb.u16 q0, [r0]
; CHECK-NEXT: vrev32.16 q1, q0
; CHECK-NEXT: vadd.i32 q0, q0, q1
; CHECK-NEXT: vstrb.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <8 x i8>, <8 x i8>* %src, align 1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <8 x i8> %l1, <8 x i8> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x i8> %l1, <8 x i8> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = add <4 x i8> %s1, %s2
store <4 x i8> %a, <4 x i8> *%dst
ret void
}
define void @vld2_v8i8(<16 x i8> *%src, <8 x i8> *%dst) {
; CHECK-LABEL: vld2_v8i8:
; CHECK: @ %bb.0: @ %entry
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: vldrb.u8 q0, [r0]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vrev16.8 q1, q0
; CHECK-NEXT: vadd.i16 q0, q0, q1
; CHECK-NEXT: vstrb.16 q0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <16 x i8>, <16 x i8>* %src, align 1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <16 x i8> %l1, <16 x i8> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
%s2 = shufflevector <16 x i8> %l1, <16 x i8> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15>
%a = add <8 x i8> %s1, %s2
store <8 x i8> %a, <8 x i8> *%dst
ret void
}
define void @vld2_v16i8(<32 x i8> *%src, <16 x i8> *%dst) {
; CHECK-LABEL: vld2_v16i8:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.8 {q0, q1}, [r0]
; CHECK-NEXT: vld21.8 {q0, q1}, [r0]
; CHECK-NEXT: vadd.i8 q0, q0, q1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <32 x i8>, <32 x i8>* %src, align 1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <32 x i8> %l1, <32 x i8> undef, <16 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14, i32 16, i32 18, i32 20, i32 22, i32 24, i32 26, i32 28, i32 30>
%s2 = shufflevector <32 x i8> %l1, <32 x i8> undef, <16 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15, i32 17, i32 19, i32 21, i32 23, i32 25, i32 27, i32 29, i32 31>
%a = add <16 x i8> %s1, %s2
store <16 x i8> %a, <16 x i8> *%dst
ret void
}
; i64
define void @vld2_v2i64(<4 x i64> *%src, <2 x i64> *%dst) {
; CHECK-LABEL: vld2_v2i64:
; CHECK: @ %bb.0: @ %entry
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: .save {r4, r5, r6, lr}
; CHECK-NEXT: push {r4, r5, r6, lr}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vldrw.u32 q0, [r0]
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vldrw.u32 q1, [r0, #16]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov.f32 s8, s2
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov.f32 s9, s3
; CHECK-NEXT: vmov.f32 s2, s4
; CHECK-NEXT: vmov.f32 s3, s5
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov lr, r12, d3
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov r5, r6, d0
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov r0, r4, d4
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov r3, r2, d1
; CHECK-NEXT: adds.w r3, r3, lr
; CHECK-NEXT: adc.w r2, r2, r12
; CHECK-NEXT: adds r0, r0, r5
; CHECK-NEXT: adcs r6, r4
; CHECK-NEXT: vmov q0[2], q0[0], r0, r3
; CHECK-NEXT: vmov q0[3], q0[1], r6, r2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: pop {r4, r5, r6, pc}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <4 x i64>, <4 x i64>* %src, align 8
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <4 x i64> %l1, <4 x i64> undef, <2 x i32> <i32 0, i32 2>
%s2 = shufflevector <4 x i64> %l1, <4 x i64> undef, <2 x i32> <i32 1, i32 3>
%a = add <2 x i64> %s1, %s2
store <2 x i64> %a, <2 x i64> *%dst
ret void
}
define void @vld2_v4i64(<8 x i64> *%src, <4 x i64> *%dst) {
; CHECK-LABEL: vld2_v4i64:
; CHECK: @ %bb.0: @ %entry
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: .save {r4, r5, r6, r7, r8, lr}
; CHECK-NEXT: push.w {r4, r5, r6, r7, r8, lr}
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: .vsave {d8}
; CHECK-NEXT: vpush {d8}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vldrw.u32 q0, [r0]
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vldrw.u32 q2, [r0, #16]
; CHECK-NEXT: vldrw.u32 q3, [r0, #48]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov.f32 s4, s2
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov.f32 s5, s3
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov lr, r12, d5
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov.f32 s2, s8
; CHECK-NEXT: vmov.f32 s3, s9
; CHECK-NEXT: vldrw.u32 q2, [r0, #32]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov.f32 s16, s10
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov.f32 s17, s11
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov r5, r6, d4
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov r2, r3, d1
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov.f32 s2, s12
; CHECK-NEXT: vmov.f32 s3, s13
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov r0, r7, d8
; CHECK-NEXT: adds.w lr, lr, r2
; CHECK-NEXT: adc.w r12, r12, r3
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov r3, r4, d7
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: adds r0, r0, r5
; CHECK-NEXT: adc.w r8, r6, r7
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov r6, r5, d1
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov r2, r7, d0
; CHECK-NEXT: adds r3, r3, r6
; CHECK-NEXT: adc.w r6, r5, r4
; CHECK-NEXT: vmov r5, r4, d2
; CHECK-NEXT: vmov q1[2], q1[0], r0, r3
; CHECK-NEXT: vmov q1[3], q1[1], r8, r6
; CHECK-NEXT: vstrw.32 q1, [r1, #16]
; CHECK-NEXT: adds r2, r2, r5
; CHECK-NEXT: vmov q0[2], q0[0], r2, lr
; CHECK-NEXT: adc.w r0, r7, r4
; CHECK-NEXT: vmov q0[3], q0[1], r0, r12
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vpop {d8}
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: pop.w {r4, r5, r6, r7, r8, pc}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <8 x i64>, <8 x i64>* %src, align 8
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <8 x i64> %l1, <8 x i64> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x i64> %l1, <8 x i64> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = add <4 x i64> %s1, %s2
store <4 x i64> %a, <4 x i64> *%dst
ret void
}
; f32
define void @vld2_v2f32(<4 x float> *%src, <2 x float> *%dst) {
; CHECK-LABEL: vld2_v2f32:
; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vldrw.u32 q0, [r0]
; CHECK-NEXT: vmov.f32 s4, s1
; CHECK-NEXT: vmov.f32 s5, s3
; CHECK-NEXT: vmov.f32 s1, s2
; CHECK-NEXT: vadd.f32 q0, q0, q1
; CHECK-NEXT: vstmia r1, {s0, s1}
; CHECK-NEXT: bx lr
entry:
%l1 = load <4 x float>, <4 x float>* %src, align 4
%s1 = shufflevector <4 x float> %l1, <4 x float> undef, <2 x i32> <i32 0, i32 2>
%s2 = shufflevector <4 x float> %l1, <4 x float> undef, <2 x i32> <i32 1, i32 3>
%a = fadd <2 x float> %s1, %s2
store <2 x float> %a, <2 x float> *%dst
ret void
}
define void @vld2_v4f32(<8 x float> *%src, <4 x float> *%dst) {
; CHECK-LABEL: vld2_v4f32:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.32 {q0, q1}, [r0]
; CHECK-NEXT: vld21.32 {q0, q1}, [r0]
; CHECK-NEXT: vadd.f32 q0, q0, q1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
%l1 = load <8 x float>, <8 x float>* %src, align 4
%s1 = shufflevector <8 x float> %l1, <8 x float> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x float> %l1, <8 x float> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = fadd <4 x float> %s1, %s2
store <4 x float> %a, <4 x float> *%dst
ret void
}
define void @vld2_v8f32(<16 x float> *%src, <8 x float> *%dst) {
; CHECK-LABEL: vld2_v8f32:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.32 {q0, q1}, [r0]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q0, q1}, [r0]!
; CHECK-NEXT: vld20.32 {q2, q3}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.f32 q0, q0, q1
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q2, q3}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vadd.f32 q1, q2, q3
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vstrw.32 q1, [r1, #16]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: bx lr
entry:
%l1 = load <16 x float>, <16 x float>* %src, align 4
%s1 = shufflevector <16 x float> %l1, <16 x float> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
%s2 = shufflevector <16 x float> %l1, <16 x float> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15>
%a = fadd <8 x float> %s1, %s2
store <8 x float> %a, <8 x float> *%dst
ret void
}
define void @vld2_v16f32(<32 x float> *%src, <16 x float> *%dst) {
; CHECK-LABEL: vld2_v16f32:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: .vsave {d8, d9, d10, d11, d12, d13}
; CHECK-NEXT: vpush {d8, d9, d10, d11, d12, d13}
; CHECK-NEXT: vld20.32 {q0, q1}, [r0]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: add.w r2, r0, #96
; CHECK-NEXT: add.w r3, r0, #64
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vld20.32 {q3, q4}, [r2]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q0, q1}, [r0]!
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vld21.32 {q3, q4}, [r2]
; CHECK-NEXT: vld20.32 {q5, q6}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.f32 q0, q0, q1
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld20.32 {q1, q2}, [r3]
[ARM] Create MQQPR and MQQQQPR register classes Similar to the MQPR register class as the MVE equivalent to QPR, this adds MQQPR and MQQQQPR register classes for the MVE equivalents of QQPR and QQQQPR registers. The MVE MQPR seemed have worked out quite well, and adding MQQPR and MQQQQPR allows us to a little more accurately specify the number of registers, calculating register pressure limits a little better. Differential Revision: https://reviews.llvm.org/D107463
2021-08-17 05:58:12 +08:00
; CHECK-NEXT: vadd.f32 q3, q3, q4
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vld21.32 {q5, q6}, [r0]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.32 {q1, q2}, [r3]
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vstrw.32 q3, [r1, #48]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.f32 q5, q5, q6
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.f32 q1, q1, q2
; CHECK-NEXT: vstrw.32 q5, [r1, #16]
[ARM] Transfer memory operands for VLDn We create MMO's for the VLDn/VSTn intrinsics in ARMTargetLowering:: getTgtMemIntrinsic, but they do not currently make it ll the way through ISel. This changes that in the various places it needs changing, making sure that the MMO is propagate through to the final instruction. This can help in scheduling, not treating the VLD2/VST2 as a scheduling barrier. Differential Revision: https://reviews.llvm.org/D101096
2021-05-03 07:04:21 +08:00
; CHECK-NEXT: vstrw.32 q1, [r1, #32]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vpop {d8, d9, d10, d11, d12, d13}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: bx lr
entry:
%l1 = load <32 x float>, <32 x float>* %src, align 4
%s1 = shufflevector <32 x float> %l1, <32 x float> undef, <16 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14, i32 16, i32 18, i32 20, i32 22, i32 24, i32 26, i32 28, i32 30>
%s2 = shufflevector <32 x float> %l1, <32 x float> undef, <16 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15, i32 17, i32 19, i32 21, i32 23, i32 25, i32 27, i32 29, i32 31>
%a = fadd <16 x float> %s1, %s2
store <16 x float> %a, <16 x float> *%dst
ret void
}
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
define void @vld2_v4f32_align1(<8 x float> *%src, <4 x float> *%dst) {
; CHECK-LABEL: vld2_v4f32_align1:
; CHECK: @ %bb.0: @ %entry
[ARM] Add alignment checks for MVE VLDn The MVE VLD2/4 and VST2/4 instructions require the pointer to be aligned to at least the size of the element type. This adds a check for that into the ARM lowerInterleavedStore and lowerInterleavedLoad functions, not creating the intrinsics if they are invalid for the alignment of the load/store. Unfortunately this is one of those bug fixes that does effect some useful codegen, as we were able to sometimes do some nice lowering of q15 types. But they can cause problem with low aligned pointers. Differential Revision: https://reviews.llvm.org/D95319
2021-01-28 21:10:08 +08:00
; CHECK-NEXT: vldrb.u8 q0, [r0, #16]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vldrb.u8 q1, [r0]
[ARM] Add alignment checks for MVE VLDn The MVE VLD2/4 and VST2/4 instructions require the pointer to be aligned to at least the size of the element type. This adds a check for that into the ARM lowerInterleavedStore and lowerInterleavedLoad functions, not creating the intrinsics if they are invalid for the alignment of the load/store. Unfortunately this is one of those bug fixes that does effect some useful codegen, as we were able to sometimes do some nice lowering of q15 types. But they can cause problem with low aligned pointers. Differential Revision: https://reviews.llvm.org/D95319
2021-01-28 21:10:08 +08:00
; CHECK-NEXT: vmov.f32 s8, s5
; CHECK-NEXT: vmov.f32 s9, s7
; CHECK-NEXT: vmov.f32 s5, s6
; CHECK-NEXT: vmov.f32 s10, s1
; CHECK-NEXT: vmov.f32 s11, s3
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov.f32 s6, s0
[ARM] Add alignment checks for MVE VLDn The MVE VLD2/4 and VST2/4 instructions require the pointer to be aligned to at least the size of the element type. This adds a check for that into the ARM lowerInterleavedStore and lowerInterleavedLoad functions, not creating the intrinsics if they are invalid for the alignment of the load/store. Unfortunately this is one of those bug fixes that does effect some useful codegen, as we were able to sometimes do some nice lowering of q15 types. But they can cause problem with low aligned pointers. Differential Revision: https://reviews.llvm.org/D95319
2021-01-28 21:10:08 +08:00
; CHECK-NEXT: vmov.f32 s7, s2
; CHECK-NEXT: vadd.f32 q0, q1, q2
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
%l1 = load <8 x float>, <8 x float>* %src, align 1
%s1 = shufflevector <8 x float> %l1, <8 x float> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x float> %l1, <8 x float> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = fadd <4 x float> %s1, %s2
store <4 x float> %a, <4 x float> *%dst
ret void
}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; f16
define void @vld2_v2f16(<4 x half> *%src, <2 x half> *%dst) {
; CHECK-LABEL: vld2_v2f16:
; CHECK: @ %bb.0: @ %entry
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: ldr r2, [r0]
; CHECK-NEXT: ldr r0, [r0, #4]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vmov.32 q0[0], r2
; CHECK-NEXT: vmov.32 q0[1], r0
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s4, s0
; CHECK-NEXT: vmovx.f16 s2, s1
; CHECK-NEXT: vins.f16 s4, s2
[ARM] Select VINS from vector inserts This patch adds tablegen patterns for pairs of i16/f16 insert/extracts. If we are inserting into two adjacent vector lanes (0 and 1 for example), we can use either a vmov;vins or vmovx;vins to insert the pair together, avoiding a round-trip from GRP registers. This is quite a large patterns with a number of EXTRACT_SUBREG/INSERT_SUBREG/ COPY_TO_REGCLASS nodes, but hopefully as most of those become copies all that will be cleaned up by further optimizations. The VINS pattern was also adjusted to allow it to represent that it is inserting into the top half of an existing register. Differential Revision: https://reviews.llvm.org/D95381
2021-02-02 21:50:02 +08:00
; CHECK-NEXT: vins.f16 s0, s1
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vadd.f16 q0, q0, q1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vmov r0, s0
; CHECK-NEXT: str r0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <4 x half>, <4 x half>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <4 x half> %l1, <4 x half> undef, <2 x i32> <i32 0, i32 2>
%s2 = shufflevector <4 x half> %l1, <4 x half> undef, <2 x i32> <i32 1, i32 3>
%a = fadd <2 x half> %s1, %s2
store <2 x half> %a, <2 x half> *%dst
ret void
}
define void @vld2_v4f16(<8 x half> *%src, <4 x half> *%dst) {
; CHECK-LABEL: vld2_v4f16:
; CHECK: @ %bb.0: @ %entry
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: vldrh.u16 q0, [r0]
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s4, s0
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s6, s1
; CHECK-NEXT: vins.f16 s4, s6
[ARM] Select VINS from vector inserts This patch adds tablegen patterns for pairs of i16/f16 insert/extracts. If we are inserting into two adjacent vector lanes (0 and 1 for example), we can use either a vmov;vins or vmovx;vins to insert the pair together, avoiding a round-trip from GRP registers. This is quite a large patterns with a number of EXTRACT_SUBREG/INSERT_SUBREG/ COPY_TO_REGCLASS nodes, but hopefully as most of those become copies all that will be cleaned up by further optimizations. The VINS pattern was also adjusted to allow it to represent that it is inserting into the top half of an existing register. Differential Revision: https://reviews.llvm.org/D95381
2021-02-02 21:50:02 +08:00
; CHECK-NEXT: vmovx.f16 s5, s2
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s6, s3
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vins.f16 s2, s3
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vins.f16 s0, s1
; CHECK-NEXT: vins.f16 s5, s6
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmov.f32 s1, s2
[ARM] Select VINS from vector inserts This patch adds tablegen patterns for pairs of i16/f16 insert/extracts. If we are inserting into two adjacent vector lanes (0 and 1 for example), we can use either a vmov;vins or vmovx;vins to insert the pair together, avoiding a round-trip from GRP registers. This is quite a large patterns with a number of EXTRACT_SUBREG/INSERT_SUBREG/ COPY_TO_REGCLASS nodes, but hopefully as most of those become copies all that will be cleaned up by further optimizations. The VINS pattern was also adjusted to allow it to represent that it is inserting into the top half of an existing register. Differential Revision: https://reviews.llvm.org/D95381
2021-02-02 21:50:02 +08:00
; CHECK-NEXT: vadd.f16 q0, q0, q1
[ARM] Create VMOVRRD from adjacent vector extracts This adds a combine for extract(x, n); extract(x, n+1) -> VMOVRRD(extract x, n/2). This allows two vector lanes to be moved at the same time in a single instruction, and thanks to the other VMOVRRD folds we have added recently can help reduce the amount of executed instructions. Floating point types are very similar, but will include a bitcast to an integer type. This also adds a shouldRewriteCopySrc, to prevent copy propagation from DPR to SPR, which can break as not all DPR regs can be extracted from directly. Otherwise the machine verifier is unhappy. Differential Revision: https://reviews.llvm.org/D100244
2021-04-20 22:15:43 +08:00
; CHECK-NEXT: vmov r0, r2, d0
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: strd r0, r2, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <8 x half>, <8 x half>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <8 x half> %l1, <8 x half> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x half> %l1, <8 x half> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = fadd <4 x half> %s1, %s2
store <4 x half> %a, <4 x half> *%dst
ret void
}
define void @vld2_v8f16(<16 x half> *%src, <8 x half> *%dst) {
; CHECK-LABEL: vld2_v8f16:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vld20.16 {q0, q1}, [r0]
; CHECK-NEXT: vld21.16 {q0, q1}, [r0]
; CHECK-NEXT: vadd.f16 q0, q0, q1
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <16 x half>, <16 x half>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <16 x half> %l1, <16 x half> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
%s2 = shufflevector <16 x half> %l1, <16 x half> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15>
%a = fadd <8 x half> %s1, %s2
store <8 x half> %a, <8 x half> *%dst
ret void
}
define void @vld2_v16f16(<32 x half> *%src, <16 x half> *%dst) {
; CHECK-LABEL: vld2_v16f16:
; CHECK: @ %bb.0: @ %entry
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld20.16 {q0, q1}, [r0]
; CHECK-NEXT: vld21.16 {q0, q1}, [r0]!
; CHECK-NEXT: vld20.16 {q2, q3}, [r0]
[ARM] MVE interleaving load and stores. Now that we have the intrinsics, we can add VLD2/4 and VST2/4 lowering for MVE. This works the same way as Neon, recognising the load/shuffles combination and converting them into intrinsics in a pre-isel pass, which just calls getMaxSupportedInterleaveFactor, lowerInterleavedLoad and lowerInterleavedStore. The main difference to Neon is that we do not have a VLD3 instruction. Otherwise most of the code works very similarly, with just some minor differences in the form of the intrinsics to work around. VLD3 is disabled by making isLegalInterleavedAccessType return false for those cases. We may need some other future adjustments, such as VLD4 take up half the available registers so should maybe cost more. This patch should get the basics in though. Differential Revision: https://reviews.llvm.org/D69392
2019-11-20 02:37:21 +08:00
; CHECK-NEXT: vadd.f16 q0, q0, q1
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vld21.16 {q2, q3}, [r0]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
[ARM] MVE VLDn postinc This adds Post inc variants of the VLD2/4 and VST2/4 instructions in MVE. It uses the same mechanism/nodes as Neon, transforming the intrinsic+add pair into a ARMISD::VLD2_UPD, which gets selected to a post-inc instruction. The code to do that is mostly taken from the existing Neon code, but simplified as less variants are needed. It also fills in some getTgtMemIntrinsic for the arm.mve.vld2/4 instrinsics, which allow the nodes to have MMO's, calculated as the full length to the memory being loaded/stored. Differential Revision: https://reviews.llvm.org/D71194
2020-01-19 02:25:45 +08:00
; CHECK-NEXT: vadd.f16 q2, q2, q3
; CHECK-NEXT: vstrw.32 q2, [r1, #16]
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <32 x half>, <32 x half>* %src, align 2
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <32 x half> %l1, <32 x half> undef, <16 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14, i32 16, i32 18, i32 20, i32 22, i32 24, i32 26, i32 28, i32 30>
%s2 = shufflevector <32 x half> %l1, <32 x half> undef, <16 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15, i32 17, i32 19, i32 21, i32 23, i32 25, i32 27, i32 29, i32 31>
%a = fadd <16 x half> %s1, %s2
store <16 x half> %a, <16 x half> *%dst
ret void
}
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
define void @vld2_v8f16_align1(<16 x half> *%src, <8 x half> *%dst) {
; CHECK-LABEL: vld2_v8f16_align1:
; CHECK: @ %bb.0: @ %entry
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vldrb.u8 q0, [r0]
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vldrb.u8 q2, [r0, #16]
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s4, s0
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s6, s1
; CHECK-NEXT: vins.f16 s4, s6
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s5, s2
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s6, s3
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s12, s9
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vins.f16 s5, s6
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmovx.f16 s6, s8
; CHECK-NEXT: vins.f16 s6, s12
; CHECK-NEXT: vmovx.f16 s7, s10
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmovx.f16 s12, s11
; CHECK-NEXT: vins.f16 s2, s3
; CHECK-NEXT: vins.f16 s10, s11
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vins.f16 s8, s9
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vins.f16 s0, s1
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmov.f32 s1, s2
; CHECK-NEXT: vins.f16 s7, s12
[ARM] Enable subreg liveness This enables subreg liveness in the arm backend when MVE is present, which allows the register allocator to detect when subregister are alive/dead, compared to only acting on full registers. This can helps produce better code on MVE with the way MQPR registers are made up of SPR registers, but is especially helpful for MQQPR and MQQQQPR registers, where there are very few "registers" available and being able to split them up into subregs can help produce much better code. Differential Revision: https://reviews.llvm.org/D107642
2021-08-17 21:10:33 +08:00
; CHECK-NEXT: vmov.f32 s2, s8
[ARM] Move double vector insert patterns using vins to DAG combine This removes the existing patterns for inserting two lanes into an f16/i16 vector register using VINS, instead using a DAG combine to pattern match the same code sequences. The tablegen patterns were already on the large side (foreach LANE = [0, 2, 4, 6]) and were not handling all the cases they could. Moving that to a DAG combine, whilst not less code, allows us to better control and expand the selection of VINSs. Additionally this allows us to remove the AddedComplexity on VCVTT. The extra trick that this has learned in the process is to move two adjacent lanes using a single f32 vmov, allowing some extra inefficiencies to be removed. Differenial Revision: https://reviews.llvm.org/D96876
2021-02-22 17:29:47 +08:00
; CHECK-NEXT: vmov.f32 s3, s10
; CHECK-NEXT: vadd.f16 q0, q0, q1
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
%l1 = load <16 x half>, <16 x half>* %src, align 1
%s1 = shufflevector <16 x half> %l1, <16 x half> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
%s2 = shufflevector <16 x half> %l1, <16 x half> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 9, i32 11, i32 13, i32 15>
%a = fadd <8 x half> %s1, %s2
store <8 x half> %a, <8 x half> *%dst
ret void
}
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
; f64
define void @vld2_v2f64(<4 x double> *%src, <2 x double> *%dst) {
; CHECK-LABEL: vld2_v2f64:
; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vldrw.u32 q0, [r0, #16]
; CHECK-NEXT: vldrw.u32 q1, [r0]
; CHECK-NEXT: vadd.f64 d1, d0, d1
; CHECK-NEXT: vadd.f64 d0, d2, d3
; CHECK-NEXT: vstrw.32 q0, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <4 x double>, <4 x double>* %src, align 8
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <4 x double> %l1, <4 x double> undef, <2 x i32> <i32 0, i32 2>
%s2 = shufflevector <4 x double> %l1, <4 x double> undef, <2 x i32> <i32 1, i32 3>
%a = fadd <2 x double> %s1, %s2
store <2 x double> %a, <2 x double> *%dst
ret void
}
define void @vld2_v4f64(<8 x double> *%src, <4 x double> *%dst) {
; CHECK-LABEL: vld2_v4f64:
; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vldrw.u32 q0, [r0, #48]
; CHECK-NEXT: vldrw.u32 q1, [r0, #32]
; CHECK-NEXT: vldrw.u32 q2, [r0]
; CHECK-NEXT: vadd.f64 d1, d0, d1
; CHECK-NEXT: vadd.f64 d0, d2, d3
; CHECK-NEXT: vldrw.u32 q1, [r0, #16]
; CHECK-NEXT: vadd.f64 d3, d2, d3
; CHECK-NEXT: vstrw.32 q0, [r1, #16]
; CHECK-NEXT: vadd.f64 d2, d4, d5
; CHECK-NEXT: vstrw.32 q1, [r1]
; CHECK-NEXT: bx lr
entry:
[ARM] Extra MVE unaligned VLDn tests. NFC
2021-01-25 05:39:00 +08:00
%l1 = load <8 x double>, <8 x double>* %src, align 8
[ARM] Add and update a lot of VLDn tests. NFC
2019-11-20 02:17:46 +08:00
%s1 = shufflevector <8 x double> %l1, <8 x double> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%s2 = shufflevector <8 x double> %l1, <8 x double> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%a = fadd <4 x double> %s1, %s2
store <4 x double> %a, <4 x double> *%dst
ret void
}