2019-09-03 03:03:35 +08:00
|
|
|
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
|
2020-11-12 16:37:46 +08:00
|
|
|
; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-LE
|
|
|
|
|
; RUN: llc -mtriple=thumbebv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-BE
|
2019-09-03 03:03:35 +08:00
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc <4 x i32> @bitcast_to_v4i1(i4 %b, <4 x i32> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_to_v4i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .pad #4
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #4
|
|
|
|
|
; CHECK-LE-NEXT: and r0, r0, #15
|
|
|
|
|
; CHECK-LE-NEXT: vmov.i8 q1, #0x0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.i8 q2, #0xff
|
|
|
|
|
; CHECK-LE-NEXT: vmsr p0, r0
|
|
|
|
|
; CHECK-LE-NEXT: vpsel q1, q2, q1
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q1[2]
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-LE-NEXT: vmov.u8 r1, q1[0]
|
|
|
|
|
; CHECK-LE-NEXT: vmov q2[2], q2[0], r1, r0
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q1[3]
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-LE-NEXT: vmov.u8 r1, q1[1]
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: vmov.i32 q1, #0x0
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-LE-NEXT: vmov q2[3], q2[1], r1, r0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: vcmp.i32 ne, q2, zr
|
|
|
|
|
; CHECK-LE-NEXT: vpsel q0, q0, q1
|
|
|
|
|
; CHECK-LE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-LE-NEXT: bx lr
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_to_v4i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .pad #4
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #4
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: rbit r0, r0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.i8 q1, #0x0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.i8 q2, #0xff
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: lsrs r0, r0, #28
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vmsr p0, r0
|
|
|
|
|
; CHECK-BE-NEXT: vpsel q1, q2, q1
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q1[2]
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.u8 r1, q1[0]
|
|
|
|
|
; CHECK-BE-NEXT: vmov q2[2], q2[0], r1, r0
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q1[3]
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.u8 r1, q1[1]
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vrev64.32 q1, q0
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-BE-NEXT: vmov q2[3], q2[1], r1, r0
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.i32 q0, #0x0
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-BE-NEXT: vcmp.i32 ne, q2, zr
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vpsel q1, q1, q0
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.32 q0, q1
|
|
|
|
|
; CHECK-BE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-BE-NEXT: bx lr
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = bitcast i4 %b to <4 x i1>
|
|
|
|
|
%s = select <4 x i1> %c, <4 x i32> %a, <4 x i32> zeroinitializer
|
|
|
|
|
ret <4 x i32> %s
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc <8 x i16> @bitcast_to_v8i1(i8 %b, <8 x i16> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_to_v8i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .pad #8
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: uxtb r0, r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.i8 q1, #0x0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.i8 q2, #0xff
|
|
|
|
|
; CHECK-LE-NEXT: vmsr p0, r0
|
|
|
|
|
; CHECK-LE-NEXT: vpsel q2, q2, q1
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[0]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[0], r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[1]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[1], r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[2]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[2], r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[3]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[3], r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[4]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[4], r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[5]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[5], r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[6]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[6], r0
|
|
|
|
|
; CHECK-LE-NEXT: vmov.u8 r0, q2[7]
|
|
|
|
|
; CHECK-LE-NEXT: vmov.16 q1[7], r0
|
|
|
|
|
; CHECK-LE-NEXT: vcmp.i16 ne, q1, zr
|
|
|
|
|
; CHECK-LE-NEXT: vmov.i32 q1, #0x0
|
|
|
|
|
; CHECK-LE-NEXT: vpsel q0, q0, q1
|
|
|
|
|
; CHECK-LE-NEXT: add sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: bx lr
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_to_v8i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .pad #8
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: uxtb r0, r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.i8 q1, #0x0
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: rbit r0, r0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.i8 q2, #0xff
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: lsrs r0, r0, #24
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vmsr p0, r0
|
|
|
|
|
; CHECK-BE-NEXT: vpsel q2, q2, q1
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[0]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[0], r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[1]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[1], r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[2]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[2], r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[3]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[3], r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[4]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[4], r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[5]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[5], r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[6]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[6], r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.u8 r0, q2[7]
|
|
|
|
|
; CHECK-BE-NEXT: vmov.16 q1[7], r0
|
|
|
|
|
; CHECK-BE-NEXT: vcmp.i16 ne, q1, zr
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.16 q1, q0
|
|
|
|
|
; CHECK-BE-NEXT: vmov.i32 q0, #0x0
|
|
|
|
|
; CHECK-BE-NEXT: vrev32.16 q0, q0
|
|
|
|
|
; CHECK-BE-NEXT: vpsel q1, q1, q0
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.16 q0, q1
|
|
|
|
|
; CHECK-BE-NEXT: add sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: bx lr
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = bitcast i8 %b to <8 x i1>
|
|
|
|
|
%s = select <8 x i1> %c, <8 x i16> %a, <8 x i16> zeroinitializer
|
|
|
|
|
ret <8 x i16> %s
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc <16 x i8> @bitcast_to_v16i1(i16 %b, <16 x i8> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_to_v16i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .save {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-LE-NEXT: push {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-LE-NEXT: .setfp r7, sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: add r7, sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: .pad #16
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #16
|
|
|
|
|
; CHECK-LE-NEXT: mov r4, sp
|
|
|
|
|
; CHECK-LE-NEXT: bfc r4, #0, #4
|
|
|
|
|
; CHECK-LE-NEXT: mov sp, r4
|
|
|
|
|
; CHECK-LE-NEXT: sub.w r4, r7, #8
|
|
|
|
|
; CHECK-LE-NEXT: vmsr p0, r0
|
2020-12-01 17:04:36 +08:00
|
|
|
; CHECK-LE-NEXT: vmov.i32 q1, #0x0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: vpsel q0, q0, q1
|
|
|
|
|
; CHECK-LE-NEXT: mov sp, r4
|
|
|
|
|
; CHECK-LE-NEXT: pop {r4, r6, r7, pc}
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_to_v16i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .save {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-BE-NEXT: push {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-BE-NEXT: .setfp r7, sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: add r7, sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: .pad #16
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #16
|
|
|
|
|
; CHECK-BE-NEXT: mov r4, sp
|
|
|
|
|
; CHECK-BE-NEXT: bfc r4, #0, #4
|
|
|
|
|
; CHECK-BE-NEXT: mov sp, r4
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: uxth r0, r0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vrev64.8 q1, q0
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: rbit r0, r0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vmov.i32 q0, #0x0
|
|
|
|
|
; CHECK-BE-NEXT: sub.w r4, r7, #8
|
|
|
|
|
; CHECK-BE-NEXT: vrev32.8 q0, q0
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: lsrs r0, r0, #16
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vmsr p0, r0
|
|
|
|
|
; CHECK-BE-NEXT: vpsel q1, q1, q0
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.8 q0, q1
|
|
|
|
|
; CHECK-BE-NEXT: mov sp, r4
|
|
|
|
|
; CHECK-BE-NEXT: pop {r4, r6, r7, pc}
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = bitcast i16 %b to <16 x i1>
|
|
|
|
|
%s = select <16 x i1> %c, <16 x i8> %a, <16 x i8> zeroinitializer
|
|
|
|
|
ret <16 x i8> %s
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc <2 x i64> @bitcast_to_v2i1(i2 %b, <2 x i64> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_to_v2i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .pad #4
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #4
|
2020-05-03 05:09:18 +08:00
|
|
|
; CHECK-LE-NEXT: and r1, r0, #2
|
|
|
|
|
; CHECK-LE-NEXT: and r0, r0, #1
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-LE-NEXT: movs r2, #0
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-LE-NEXT: rsbs r0, r0, #0
|
2020-05-03 05:09:18 +08:00
|
|
|
; CHECK-LE-NEXT: sub.w r1, r2, r1, lsr #1
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-LE-NEXT: vmov q1[2], q1[0], r0, r1
|
|
|
|
|
; CHECK-LE-NEXT: vmov q1[3], q1[1], r0, r1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: vand q0, q0, q1
|
|
|
|
|
; CHECK-LE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-LE-NEXT: bx lr
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_to_v2i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .pad #4
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #4
|
2020-05-03 05:09:18 +08:00
|
|
|
; CHECK-BE-NEXT: and r1, r0, #2
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-BE-NEXT: and r0, r0, #1
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-BE-NEXT: movs r2, #0
|
2020-12-18 21:33:40 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r0, r0, #0
|
[ARM] Match dual lane vmovs from insert_vector_elt
MVE has a dual lane vector move instruction, capable of moving two
general purpose registers into lanes of a vector register. They look
like one of:
vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1
They only accept these lane indices though (and only insert into an
i32), either moving lanes 1 and 3, or 0 and 2.
This patch adds some tablegen patterns for them, selecting from vector
inserts elements. Because the insert_elements are know to be
canonicalized to ascending order there are several patterns that we need
to select. These lane indices are:
3 2 1 0 -> vmovqrr 31; vmovqrr 20
3 2 1 -> vmovqrr 31; vmov 2
3 1 -> vmovqrr 31
2 1 0 -> vmovqrr 20; vmov 1
2 0 -> vmovqrr 20
With the top one being the most common. All other potential patterns of
lane indices will be matched by a combination of these and the
individual vmov pattern already present. This does mean that we are
selecting several machine instructions at once due to the need to
re-arrange the inserts, but in this case there is nothing else that will
attempt to match an insert_vector_elt node.
This is a recommit of 6cc3d80a84884a79967fffa4596c14001b8ba8a3 after
fixing the backward instruction definitions.
2020-12-19 00:13:08 +08:00
|
|
|
; CHECK-BE-NEXT: sub.w r1, r2, r1, lsr #1
|
|
|
|
|
; CHECK-BE-NEXT: vmov q1[2], q1[0], r1, r0
|
|
|
|
|
; CHECK-BE-NEXT: vmov q1[3], q1[1], r1, r0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: vrev64.32 q2, q1
|
|
|
|
|
; CHECK-BE-NEXT: vand q0, q0, q2
|
|
|
|
|
; CHECK-BE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-BE-NEXT: bx lr
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = bitcast i2 %b to <2 x i1>
|
|
|
|
|
%s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> zeroinitializer
|
|
|
|
|
ret <2 x i64> %s
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc i4 @bitcast_from_v4i1(<4 x i32> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_from_v4i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .pad #4
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #4
|
|
|
|
|
; CHECK-LE-NEXT: vcmp.i32 eq, q0, zr
|
|
|
|
|
; CHECK-LE-NEXT: vmrs r1, p0
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-LE-NEXT: and r0, r1, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r0, #0
|
|
|
|
|
; CHECK-LE-NEXT: movs r0, #0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #0, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #4, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #1, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #8, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r1, r1, #12, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #2, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r1, r1, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r1, #3, #1
|
|
|
|
|
; CHECK-LE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-LE-NEXT: bx lr
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_from_v4i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .pad #4
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #4
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.32 q1, q0
|
|
|
|
|
; CHECK-BE-NEXT: vcmp.i32 eq, q1, zr
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: vmrs r1, p0
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r0, r1, #12, #1
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r0, #0
|
|
|
|
|
; CHECK-BE-NEXT: movs r0, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #0, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #8, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #1, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #4, #1
|
|
|
|
|
; CHECK-BE-NEXT: and r1, r1, #1
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #2, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r1, r1, #0
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: bfi r0, r1, #3, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-BE-NEXT: bx lr
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = icmp eq <4 x i32> %a, zeroinitializer
|
|
|
|
|
%b = bitcast <4 x i1> %c to i4
|
|
|
|
|
ret i4 %b
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc i8 @bitcast_from_v8i1(<8 x i16> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_from_v8i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .pad #8
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: vcmp.i16 eq, q0, zr
|
|
|
|
|
; CHECK-LE-NEXT: vmrs r1, p0
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-LE-NEXT: and r0, r1, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r0, #0
|
|
|
|
|
; CHECK-LE-NEXT: movs r0, #0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #0, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #2, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #1, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #4, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #2, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #6, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #3, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #8, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #4, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #10, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #5, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r2, r1, #12, #1
|
|
|
|
|
; CHECK-LE-NEXT: ubfx r1, r1, #14, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r2, #6, #1
|
|
|
|
|
; CHECK-LE-NEXT: rsbs r1, r1, #0
|
|
|
|
|
; CHECK-LE-NEXT: bfi r0, r1, #7, #1
|
|
|
|
|
; CHECK-LE-NEXT: uxtb r0, r0
|
|
|
|
|
; CHECK-LE-NEXT: add sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: bx lr
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_from_v8i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .pad #8
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.16 q1, q0
|
|
|
|
|
; CHECK-BE-NEXT: vcmp.i16 eq, q1, zr
|
|
|
|
|
; CHECK-BE-NEXT: vmrs r1, p0
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r0, r1, #14, #1
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r0, #0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: movs r0, #0
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #0, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #12, #1
|
2019-09-17 01:29:07 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #1, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #10, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #2, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #8, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #3, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #6, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #4, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #4, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #5, #1
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: ubfx r2, r1, #2, #1
|
|
|
|
|
; CHECK-BE-NEXT: and r1, r1, #1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: rsbs r2, r2, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r2, #6, #1
|
|
|
|
|
; CHECK-BE-NEXT: rsbs r1, r1, #0
|
|
|
|
|
; CHECK-BE-NEXT: bfi r0, r1, #7, #1
|
|
|
|
|
; CHECK-BE-NEXT: uxtb r0, r0
|
|
|
|
|
; CHECK-BE-NEXT: add sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: bx lr
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = icmp eq <8 x i16> %a, zeroinitializer
|
|
|
|
|
%b = bitcast <8 x i1> %c to i8
|
|
|
|
|
ret i8 %b
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc i16 @bitcast_from_v16i1(<16 x i8> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_from_v16i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .save {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-LE-NEXT: push {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-LE-NEXT: .setfp r7, sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: add r7, sp, #8
|
|
|
|
|
; CHECK-LE-NEXT: .pad #16
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #16
|
|
|
|
|
; CHECK-LE-NEXT: mov r4, sp
|
|
|
|
|
; CHECK-LE-NEXT: bfc r4, #0, #4
|
|
|
|
|
; CHECK-LE-NEXT: mov sp, r4
|
|
|
|
|
; CHECK-LE-NEXT: vcmp.i8 eq, q0, zr
|
|
|
|
|
; CHECK-LE-NEXT: sub.w r4, r7, #8
|
|
|
|
|
; CHECK-LE-NEXT: vmrs r0, p0
|
|
|
|
|
; CHECK-LE-NEXT: uxth r0, r0
|
|
|
|
|
; CHECK-LE-NEXT: mov sp, r4
|
|
|
|
|
; CHECK-LE-NEXT: pop {r4, r6, r7, pc}
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_from_v16i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .save {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-BE-NEXT: push {r4, r6, r7, lr}
|
|
|
|
|
; CHECK-BE-NEXT: .setfp r7, sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: add r7, sp, #8
|
|
|
|
|
; CHECK-BE-NEXT: .pad #16
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #16
|
|
|
|
|
; CHECK-BE-NEXT: mov r4, sp
|
|
|
|
|
; CHECK-BE-NEXT: bfc r4, #0, #4
|
|
|
|
|
; CHECK-BE-NEXT: mov sp, r4
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.8 q1, q0
|
|
|
|
|
; CHECK-BE-NEXT: sub.w r4, r7, #8
|
|
|
|
|
; CHECK-BE-NEXT: vcmp.i8 eq, q1, zr
|
|
|
|
|
; CHECK-BE-NEXT: vmrs r0, p0
|
2021-02-08 19:37:17 +08:00
|
|
|
; CHECK-BE-NEXT: rbit r0, r0
|
|
|
|
|
; CHECK-BE-NEXT: lsrs r0, r0, #16
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: mov sp, r4
|
|
|
|
|
; CHECK-BE-NEXT: pop {r4, r6, r7, pc}
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = icmp eq <16 x i8> %a, zeroinitializer
|
|
|
|
|
%b = bitcast <16 x i1> %c to i16
|
|
|
|
|
ret i16 %b
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define arm_aapcs_vfpcc i2 @bitcast_from_v2i1(<2 x i64> %a) {
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-LABEL: bitcast_from_v2i1:
|
|
|
|
|
; CHECK-LE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-LE-NEXT: .pad #4
|
|
|
|
|
; CHECK-LE-NEXT: sub sp, #4
|
2021-04-20 22:15:43 +08:00
|
|
|
; CHECK-LE-NEXT: vmov r0, r1, d0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: orrs r0, r1
|
2021-04-20 22:15:43 +08:00
|
|
|
; CHECK-LE-NEXT: vmov r1, r2, d1
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: cset r0, eq
|
|
|
|
|
; CHECK-LE-NEXT: orrs r1, r2
|
|
|
|
|
; CHECK-LE-NEXT: cset r1, eq
|
2021-03-04 16:40:20 +08:00
|
|
|
; CHECK-LE-NEXT: cmp r1, #0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-LE-NEXT: it ne
|
|
|
|
|
; CHECK-LE-NEXT: mvnne r1, #1
|
|
|
|
|
; CHECK-LE-NEXT: bfi r1, r0, #0, #1
|
|
|
|
|
; CHECK-LE-NEXT: and r0, r1, #3
|
|
|
|
|
; CHECK-LE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-LE-NEXT: bx lr
|
|
|
|
|
;
|
|
|
|
|
; CHECK-BE-LABEL: bitcast_from_v2i1:
|
|
|
|
|
; CHECK-BE: @ %bb.0: @ %entry
|
|
|
|
|
; CHECK-BE-NEXT: .pad #4
|
|
|
|
|
; CHECK-BE-NEXT: sub sp, #4
|
|
|
|
|
; CHECK-BE-NEXT: vrev64.32 q1, q0
|
2021-04-20 22:15:43 +08:00
|
|
|
; CHECK-BE-NEXT: vmov r0, r1, d3
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: orrs r0, r1
|
2021-04-20 22:15:43 +08:00
|
|
|
; CHECK-BE-NEXT: vmov r1, r2, d2
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: cset r0, eq
|
|
|
|
|
; CHECK-BE-NEXT: orrs r1, r2
|
|
|
|
|
; CHECK-BE-NEXT: cset r1, eq
|
2021-03-04 16:40:20 +08:00
|
|
|
; CHECK-BE-NEXT: cmp r1, #0
|
[ARM] Fix loads and stores for predicate vectors
These predicate vectors can usually be loaded and stored with a single
instruction, a VSTR_P0. However this instruction will store the entire P0
predicate, 16 bits, zeroextended to 32bits. Each lane of the the
v4i1/v8i1/v16i1 representing 4/2/1 bits.
As far as I understand, when llvm says "store this v4i1", it really does need
to store 4 bits (or 8, that being the size of a byte, with this bottom 4 as the
interesting bits). For example a bitcast from a v8i1 to a i8 is defined as a
store followed by a load, which is how the code is expanded.
So this instead lowers the v4i1/v8i1 load/store through some shuffles to get
the bits into the correct positions. This, as you might imagine, is not as
efficient as a single instruction. But I believe it is needed for correctness.
v16i1 equally should not load/store 32bits, only storing the 16bits of data.
Stack loads/stores are still using the VSTR_P0 (as can be seen by the test not
changing). This is fine as they are self-consistent, it is only "externally
observable loads/stores" (from our point of view) that need to be corrected.
Differential revision: https://reviews.llvm.org/D67085
llvm-svn: 371419
2019-09-10 00:35:49 +08:00
|
|
|
; CHECK-BE-NEXT: it ne
|
|
|
|
|
; CHECK-BE-NEXT: mvnne r1, #1
|
|
|
|
|
; CHECK-BE-NEXT: bfi r1, r0, #0, #1
|
|
|
|
|
; CHECK-BE-NEXT: and r0, r1, #3
|
|
|
|
|
; CHECK-BE-NEXT: add sp, #4
|
|
|
|
|
; CHECK-BE-NEXT: bx lr
|
2019-09-03 03:03:35 +08:00
|
|
|
entry:
|
|
|
|
|
%c = icmp eq <2 x i64> %a, zeroinitializer
|
|
|
|
|
%b = bitcast <2 x i1> %c to i2
|
|
|
|
|
ret i2 %b
|
|
|
|
|
}
|