[PowerPC] Add vec_vsx_ld and vec_vsx_st intrinsics
This patch enables the vec_vsx_ld and vec_vsx_st intrinsics for
PowerPC, which provide programmer access to the lxvd2x, lxvw4x,
stxvd2x, and stxvw4x instructions.
New LLVM intrinsics are provided to represent these four instructions
in IntrinsicsPowerPC.td. These are patterned after the similar
intrinsics for lvx and stvx (Altivec). In PPCInstrVSX.td, these
intrinsics are tied to the code gen patterns, with additional patterns
to allow plain vanilla loads and stores to still generate these
instructions.
At -O1 and higher the intrinsics are immediately converted to loads
and stores in InstCombineCalls.cpp. This will open up more
optimization opportunities while still allowing the correct
instructions to be generated. (Similar code exists for aligned
Altivec loads and stores.)
The new intrinsics are added to the code that checks for consecutive
loads and stores in PPCISelLowering.cpp, as well as to
PPCTargetLowering::getTgtMemIntrinsic().
There's a new test to verify the correct instructions are generated.
The loads and stores tend to be reordered, so the test just counts
their number. It runs at -O2, as it's not very effective to test this
at -O0, when many unnecessary loads and stores are generated.
I ended up having to modify vsx-fma-m.ll. It turns out this test case
is slightly unreliable, but I don't know a good way to prevent
problems with it. The xvmaddmdp instructions read and write the same
register, which is one of the multiplicands. Commutativity allows
either to be chosen. If the FMAs are reordered differently than
expected by the test, the register assignment can be different as a
result. Hopefully this doesn't change often.
There is a companion patch for Clang.
llvm-svn: 221767
2014-11-12 12:19:40 +08:00
|
|
|
; RUN: llc -mcpu=pwr8 -mattr=+vsx -O2 -mtriple=powerpc64-unknown-linux-gnu < %s > %t
|
|
|
|
|
; RUN: grep lxvw4x < %t | count 3
|
|
|
|
|
; RUN: grep lxvd2x < %t | count 3
|
|
|
|
|
; RUN: grep stxvw4x < %t | count 3
|
|
|
|
|
; RUN: grep stxvd2x < %t | count 3
|
2014-12-06 04:32:05 +08:00
|
|
|
; RUN: llc -mcpu=pwr8 -mattr=+vsx -O0 -fast-isel=1 -mtriple=powerpc64-unknown-linux-gnu < %s > %t
|
|
|
|
|
; RUN: grep lxvw4x < %t | count 3
|
|
|
|
|
; RUN: grep lxvd2x < %t | count 3
|
|
|
|
|
; RUN: grep stxvw4x < %t | count 3
|
|
|
|
|
; RUN: grep stxvd2x < %t | count 3
|
[PowerPC] Add vec_vsx_ld and vec_vsx_st intrinsics
This patch enables the vec_vsx_ld and vec_vsx_st intrinsics for
PowerPC, which provide programmer access to the lxvd2x, lxvw4x,
stxvd2x, and stxvw4x instructions.
New LLVM intrinsics are provided to represent these four instructions
in IntrinsicsPowerPC.td. These are patterned after the similar
intrinsics for lvx and stvx (Altivec). In PPCInstrVSX.td, these
intrinsics are tied to the code gen patterns, with additional patterns
to allow plain vanilla loads and stores to still generate these
instructions.
At -O1 and higher the intrinsics are immediately converted to loads
and stores in InstCombineCalls.cpp. This will open up more
optimization opportunities while still allowing the correct
instructions to be generated. (Similar code exists for aligned
Altivec loads and stores.)
The new intrinsics are added to the code that checks for consecutive
loads and stores in PPCISelLowering.cpp, as well as to
PPCTargetLowering::getTgtMemIntrinsic().
There's a new test to verify the correct instructions are generated.
The loads and stores tend to be reordered, so the test just counts
their number. It runs at -O2, as it's not very effective to test this
at -O0, when many unnecessary loads and stores are generated.
I ended up having to modify vsx-fma-m.ll. It turns out this test case
is slightly unreliable, but I don't know a good way to prevent
problems with it. The xvmaddmdp instructions read and write the same
register, which is one of the multiplicands. Commutativity allows
either to be chosen. If the FMAs are reordered differently than
expected by the test, the register assignment can be different as a
result. Hopefully this doesn't change often.
There is a companion patch for Clang.
llvm-svn: 221767
2014-11-12 12:19:40 +08:00
|
|
|
|
[PowerPC 1/4] Little-endian adjustments for VSX loads/stores
This patch addresses the inherent big-endian bias in the lxvd2x,
lxvw4x, stxvd2x, and stxvw4x instructions. These instructions load
vector elements into registers left-to-right (with the first element
loaded into the high-order bits of the register), regardless of the
endian setting of the processor. However, these are the only
vector memory instructions that permit unaligned storage accesses, so
we want to use them for little-endian.
To make this work, a lxvd2x or lxvw4x is replaced with an lxvd2x
followed by an xxswapd, which swaps the doublewords. This works for
lxvw4x as well as lxvd2x, because for lxvw4x on an LE system the
vector elements are in LE order (right-to-left) within each
doubleword. (Thus after lxvw2x of a <4 x float> the elements will
appear as 1, 0, 3, 2. Following the swap, they will appear as 3, 2,
0, 1, as desired.) For stores, an stxvd2x or stxvw4x is replaced
with an stxvd2x preceded by an xxswapd.
Introduction of extra swap instructions provides correctness, but
obviously is not ideal from a performance perspective. Future patches
will address this with optimizations to remove most of the introduced
swaps, which have proven effective in other implementations.
The introduction of the swaps is performed during lowering of LOAD,
STORE, INTRINSIC_W_CHAIN, and INTRINSIC_VOID operations. The latter
are used to translate intrinsics that specify the VSX loads and stores
directly into equivalent sequences for little endian. Thus code that
uses vec_vsx_ld and vec_vsx_st does not have to be modified to be
ported from BE to LE.
We introduce new PPCISD opcodes for LXVD2X, STXVD2X, and XXSWAPD for
use during this lowering step. In PPCInstrVSX.td, we add new SDType
and SDNode definitions for these (PPClxvd2x, PPCstxvd2x, PPCxxswapd).
These are recognized during instruction selection and mapped to the
correct instructions.
Several tests that were written to use -mcpu=pwr7 or pwr8 are modified
to disable VSX on LE variants because code generation changes with
this and subsequent patches in this set. I chose to include all of
these in the first patch than try to rigorously sort out which tests
were broken by one or another of the patches. Sorry about that.
The new test vsx-ldst-builtin-le.ll, and the changes to vsx-ldst.ll,
are disabled until LE support is enabled because of breakages that
occur as noted in those tests. They are re-enabled in patch 4/4.
llvm-svn: 223783
2014-12-10 00:35:51 +08:00
|
|
|
;; Note: The LE test variant is disabled until LE support for VSX is enabled,
|
|
|
|
|
;; as otherwise we fail to get the expected counts.
|
|
|
|
|
|
|
|
|
|
; R;UN: llc -mcpu=pwr8 -mattr=+vsx -O2 -mtriple=powerpc64le-unknown-linux-gnu < %s > %t
|
|
|
|
|
; R;UN: grep lxvd2x < %t | count 6
|
|
|
|
|
; R;UN: grep stxvd2x < %t | count 6
|
|
|
|
|
; R;UN: grep xxpermdi < %t | count 12
|
|
|
|
|
|
[PowerPC] Add vec_vsx_ld and vec_vsx_st intrinsics
This patch enables the vec_vsx_ld and vec_vsx_st intrinsics for
PowerPC, which provide programmer access to the lxvd2x, lxvw4x,
stxvd2x, and stxvw4x instructions.
New LLVM intrinsics are provided to represent these four instructions
in IntrinsicsPowerPC.td. These are patterned after the similar
intrinsics for lvx and stvx (Altivec). In PPCInstrVSX.td, these
intrinsics are tied to the code gen patterns, with additional patterns
to allow plain vanilla loads and stores to still generate these
instructions.
At -O1 and higher the intrinsics are immediately converted to loads
and stores in InstCombineCalls.cpp. This will open up more
optimization opportunities while still allowing the correct
instructions to be generated. (Similar code exists for aligned
Altivec loads and stores.)
The new intrinsics are added to the code that checks for consecutive
loads and stores in PPCISelLowering.cpp, as well as to
PPCTargetLowering::getTgtMemIntrinsic().
There's a new test to verify the correct instructions are generated.
The loads and stores tend to be reordered, so the test just counts
their number. It runs at -O2, as it's not very effective to test this
at -O0, when many unnecessary loads and stores are generated.
I ended up having to modify vsx-fma-m.ll. It turns out this test case
is slightly unreliable, but I don't know a good way to prevent
problems with it. The xvmaddmdp instructions read and write the same
register, which is one of the multiplicands. Commutativity allows
either to be chosen. If the FMAs are reordered differently than
expected by the test, the register assignment can be different as a
result. Hopefully this doesn't change often.
There is a companion patch for Clang.
llvm-svn: 221767
2014-11-12 12:19:40 +08:00
|
|
|
@vsi = global <4 x i32> <i32 -1, i32 2, i32 -3, i32 4>, align 16
|
|
|
|
|
@vui = global <4 x i32> <i32 0, i32 1, i32 2, i32 3>, align 16
|
|
|
|
|
@vf = global <4 x float> <float -1.500000e+00, float 2.500000e+00, float -3.500000e+00, float 4.500000e+00>, align 16
|
|
|
|
|
@vsll = global <2 x i64> <i64 255, i64 -937>, align 16
|
|
|
|
|
@vull = global <2 x i64> <i64 1447, i64 2894>, align 16
|
|
|
|
|
@vd = global <2 x double> <double 3.500000e+00, double -7.500000e+00>, align 16
|
|
|
|
|
@res_vsi = common global <4 x i32> zeroinitializer, align 16
|
|
|
|
|
@res_vui = common global <4 x i32> zeroinitializer, align 16
|
|
|
|
|
@res_vf = common global <4 x float> zeroinitializer, align 16
|
|
|
|
|
@res_vsll = common global <2 x i64> zeroinitializer, align 16
|
|
|
|
|
@res_vull = common global <2 x i64> zeroinitializer, align 16
|
|
|
|
|
@res_vd = common global <2 x double> zeroinitializer, align 16
|
|
|
|
|
|
|
|
|
|
; Function Attrs: nounwind
|
|
|
|
|
define void @test1() {
|
|
|
|
|
entry:
|
|
|
|
|
%0 = load <4 x i32>* @vsi, align 16
|
|
|
|
|
%1 = load <4 x i32>* @vui, align 16
|
|
|
|
|
%2 = load <4 x i32>* bitcast (<4 x float>* @vf to <4 x i32>*), align 16
|
|
|
|
|
%3 = load <2 x double>* bitcast (<2 x i64>* @vsll to <2 x double>*), align 16
|
|
|
|
|
%4 = load <2 x double>* bitcast (<2 x i64>* @vull to <2 x double>*), align 16
|
|
|
|
|
%5 = load <2 x double>* @vd, align 16
|
|
|
|
|
store <4 x i32> %0, <4 x i32>* @res_vsi, align 16
|
|
|
|
|
store <4 x i32> %1, <4 x i32>* @res_vui, align 16
|
|
|
|
|
store <4 x i32> %2, <4 x i32>* bitcast (<4 x float>* @res_vf to <4 x i32>*), align 16
|
|
|
|
|
store <2 x double> %3, <2 x double>* bitcast (<2 x i64>* @res_vsll to <2 x double>*), align 16
|
|
|
|
|
store <2 x double> %4, <2 x double>* bitcast (<2 x i64>* @res_vull to <2 x double>*), align 16
|
|
|
|
|
store <2 x double> %5, <2 x double>* @res_vd, align 16
|
|
|
|
|
ret void
|
|
|
|
|
}
|